How I use data to optimize AI apps
A video collaboration between Find AI and Velvet
Philip Ilic Thomas
At Find AI, we use OpenAI a lot. Last week, we made 19 million requests.
Understanding what's happening at that scale can be challenging. It's a classic OODA loop:
- Observe what our application is doing and which systems are triggering requests
- Orient around what's happening, such as which models are the most costly in aggregate
- Decide how to make the system more efficient, such as by testing a more efficient model or shorter prompt
- Act by rolling out changes
Velvet, an AI Gateway, is the tool in our development stack that enables this observability and optimization loop. I worked with them this week to produce a video about how we use data to optimize our AI-powered apps at Find AI.
The video covers observability tools in development, cost attribution, using the OpenAI Batch API, evaluating new models, and fine-tuning. I hope it's a useful resource for people running AI models in production.
Keep reading
Behind the scenes of the Find AI API
We recently released a Search API for Find AI, and announced its integration with Clay.com. APIs allow developers to interact with a product using code. There are many different ways to build an API, ...
Internal tools of Find AI
Technical presentation at an AI meetup
Claude, Launching, and Tools
For the last week, I’ve been leaning into Claude Code for development. I used to think of it as more of a hobbyist tool, rather than a professional one. Cursor had been the main way I used AI to write...