Start With the Workload, Not the Logo
The data platform conversation usually starts in the wrong place: which vendor. Start instead with three numbers — data volume, query concurrency, and freshness requirement — and the architecture largely chooses itself:
- Under ~500GB, hourly freshness, a handful of analysts: a well-indexed PostgreSQL replica or DuckDB over Parquet files does the job for a fraction of the cost. We tell more clients this than any vendor would like.
- Terabytes, mixed BI + data science, many consumers: a cloud warehouse (Snowflake, BigQuery, Redshift) or lakehouse (Databricks, or open-table formats like Iceberg/Delta on S3 with an engine on top) earns its keep.
- Real-time operational analytics: a different problem entirely — see our event streaming guide before buying a warehouse to do a stream's job.
The warehouse-vs-lakehouse war is mostly over: warehouses added open formats and Python; lakehouses added SQL performance and governance. In 2026 the honest differences are pricing model, ecosystem fit, and where your team's skills already are.
The Architecture That Survives Contact With Monday Morning
- ELT with a managed extractor. Land raw data with a connector service (Fivetran-class) or lightweight open-source ingestion; transform inside the platform with dbt. Hand-rolled extraction scripts are where data teams go to die.
- Medallion layering with a straight face. Raw → cleaned/conformed → business-ready marts. The point isn't the bronze/silver/gold branding; it's that analysts query modelled tables with tested definitions, never raw feeds.
- A semantic layer for the metrics that matter. "Revenue" defined once, tested, and consumed by every dashboard — this is what actually ends the weekly "whose number is right?" meeting, and it's the difference between a data platform and an expensive filing cabinet.
- Tests and lineage from day one. dbt tests on keys, freshness and accepted values; lineage so that when a source column changes, you know what breaks before finance does.
Nobody has ever been promoted for the elegance of their ingestion pipeline. Trusted numbers, on time, every morning — that's the entire job.
The Cost Traps
"Pay for what you use" platforms bill for what you waste just as cheerfully:
- Warehouses left running. Auto-suspend at 60 seconds, right-size per workload, and separate the BI warehouse from the transformation warehouse. This alone commonly cuts Snowflake bills 30-50%.
- Dashboards refreshing data nobody reads. Audit query history quarterly; retire zombie reports.
- Full-refresh models that should be incremental. The single biggest dbt cost lever on large tables.
- Per-row connector pricing on high-churn tables. Know your MAR maths before pointing a managed connector at an events table.
Getting Value in Weeks, Not Quarters
The failed pattern is the eighteen-month "foundation" programme that ships no dashboards. The pattern that works: pick the three questions the executive team argues about, build the thin slice — source → model → tested metric → dashboard — end to end in the first month, then widen. Our typical fixed-price engagement stands up the platform, the first two source integrations, the semantic layer for core metrics, and the executive dashboard in 6-8 weeks, with your team trained on dbt as we go so the platform is yours to grow, not ours to invoice.
Reports slow, numbers disputed, dashboards multiplying?
Book 15 minutes and we'll size the platform your workload actually needs — including 'smaller than you think'.
Book a 15-Minute Call →