May 21, 2025How we Built Scalable & Customizable DashboardsA technical deep dive into the core abstractions and architecture On Day 3 of , we introduce customizable dashboards (): a powerful way to visualize LLM usage directly within the Langfuse UI. Whether you want to track latency trends, monitor user feedback, or correlate cost with performance, our new dashboards let you build the charts you need - right where you need them. And for those who prefer working in their own analytics stack, we’ve made the same querying capabilities available via our API (). For context: quick demo of the finished product In this post, I’ll take you behind the scenes: from our product thinking to the technical implementation, testing, and rollout—and share the lessons we learned in building flexible, real-time insights into your LLM pipelines. Motivation Customizable dashboards were among the features within Langfuse. Early on, we had the home screen dashboard that showed users metrics we believed to be relevant and accurate. We expanded that over time based on feedback about the charts users wanted to see. The dashboard grew quite long, and we found that charts were relevant but not necessarily to the same kind of users. A product manager wants to understand which score specific traces receive, while a developer wants to see latencies, error rates, and costs. We realized that we needed to provide dashboards that were adaptable to the needs of different users and personas. This involved the ability to create multiple dashboards, a variety of charts and insights, and a flexible query engine. While exploring custom dashboards, we found that within our data model and constraints, this would become quite complex. Therefore, we included custom dashboards as part of the Superday in our interview process and had many candidates design their version of the dashboard and share their thoughts. That’s how this became the most specced feature we have ever had at Langfuse. Early in March, I set ...
First seen: 2025-05-21 19:21
Last seen: 2025-05-21 19:21