Cutting down on AI model usage costs by 40% with monitoring and evaluation

For one of our clients we recently built an agentic workflow with 10+ distinct chained LLM calls. After the initial build, the latency of the agent was large and the output accuracy insufficient.

We solved this problem by implementing diligent monitoring and evaluation metrics for each step in the workflow, using Helicone (YC W23) as a useful tool in this process. In doing so, we gradually discovered hidden insights to adapt our LLM flow, radically improve output accuracy and cut down model usage costs by over 40%! We're always happy to share learnings about our approach.