The Challenge
The company had plenty of data — but no way to get at it. Reporting was served from a daily cache rebuilt from production databases by the engineering team. If the cache failed, dashboards went dark. If someone needed a metric that wasn’t pre-computed, they filed a ticket and waited.
Customer Success couldn’t build their own analytics. The executive leadership team couldn’t see North Star metrics on demand. And the customer-facing analytics — used by over 100K users — were slow, brittle, and constantly at risk of going down.
The engineering team was spending a disproportionate amount of time keeping this system alive instead of building product.
The Approach
I led the design and implementation of a proper data engineering platform to replace the production-database-backed cache:
- Data warehouse: Stood up AWS Redshift as a dedicated analytical store, decoupling reporting from production databases entirely
- Transformation layer: Built a dbt project with modular, tested models — replacing the monolithic cache-rebuild with incremental, auditable pipelines
- Self-serve BI: Deployed QuickSight dashboards so the Customer Success team could slice data themselves without engineering involvement
- Executive reporting: Built a management information layer so leadership could see North Star metrics and operational KPIs on demand
- Customer-facing analytics: Re-architected the data serving layer so 100K+ users got reliable, fast dashboards backed by the warehouse rather than a fragile cache
The Result
- 10x faster reporting: What used to require a full daily cache rebuild now updates incrementally, with most dashboards refreshing in minutes
- 99.9% uptime: Customer-facing analytics moved from “hope the cache worked” to reliable, warehouse-backed serving
- Self-serve analytics: Customer Success built their own dashboards and stopped filing tickets to engineering
- Executive visibility: Leadership got on-demand access to North Star metrics for the first time
- Engineering time reclaimed: The team stopped babysitting cache rebuilds and went back to building product