Data Modeling - PostHog

Pineapple on pizza

No data available

Feature ownership

Members

Goals

Source control on models
- Transitioning from rapid iteration in app to guarded, PR'd changes against a GitHub repo must be smooth
- We will also provide a CLI to improve the local dev experience for data engineers

Modeling tests
- Data tests: the values of certain data are what they should be or custom assertions on data pass
- Schema tests: the schema conforms to a contract like not null or unique
- Freshness tests: the data from a particular node is no more than <some duration> old
- Relational tests: expectations on referential integrity hold (i.e. one-to-one, many-to-one, etc.)

Endpoints GA
- Beta label is removed as soon as the experience meets our expectations of quality; this includes fixes to current papercuts
- Marketing launch and "true" GA happens once pricing is ready

Modeling on managed warehouse
- We will substantially limit what modeling workflows are allowed to hit our ClickHouse infra
- We will help customers as much as reasonably possible as they migrate to our managed warehouse; this includes UX changes to gently nudge customers toward the (ultimately superior) managed warehouse experience

Competitor research concerning anomaly detection and alerting
- We don't feel any of our competitors nail this and many of our customers agree; that doesn't mean these features aren't incredibly useful, it just means we have to do better
- Anna will ship an RFC with some opinionated recommendations here

AI-first data modeling via skills and MCP
- We already have some internal folks itching to help ship this
- Source control and testing will act as solid guard rails for reigning in agents that will model on behalf of customers in the near future; but we won't let that block us from shipping an ai experience that we can improve later.
- the AI-first modeling experience will include skills that cover a decent chunk of day-to-day modeling tasks: creating and editing views, materializing them, creating endpoints, asking questions about existing models, etc. If our customers are hitting "skill issues" when modeling with AI, that's on us to fix this quarter.