Solving AI Latency and Cost Problems with Valkey - Aiven Webinar
LLM providers charge you for caching, and they do it on their terms. You can run it yourself for a fraction of the cost.
In this Aiven webinar, Jay Miller (Staff Developer Advocate) shows how to use semantic caching and semantic routing on Valkey to cut prompt costs + latency. You'll see a working demo, "Template Scout", that searches GitHub for deployable repos and picks the cheapest model that can answer each question.
What you'll learn:
Why AI costs + latency compound, and where a cache fits
How a cache hit differs from a semantic hit, and how vector distance sets the threshold
How semantic routing sends each prompt to a cached answer, a cheap model, or a top-tier model
How Template Scout stores 500+ GitHub repos in about 30 MB on a Valkey developer tier plan (around $12/month)
🔗 Links
Blog post, right-sizing model usage with Valkey + semantic routing: https://aiven.io/blog/right-sizing-your-model-usage-with-valkey-and-semantic-routing
Webinar registration page: https://aiven.io/webinar/solving-real-ai-problems-using-valkey
Template Scout demo repo: https://github.com/Aiven-Labs/semantic-cache-python-run-on-runtime-demo
Valkey Admin on Aiven Runtime: https://github.com/aiven-labs/valkey-admin-aiven-apps
Runs on Runtime templates: https://templates.aiven.io/
Valkey: https://valkey.io/
Try Aiven free for 30 days + meet Jay: https://aiven.io/jay
🎁 Claim your $10 Aiven credit (valid until November 7)
In the Aiven Console (https://console.aiven.io/), go to Billing | Billing groups.
Open your billing group and click Details.
On the Credits tab, enter code AIVEN_VALKEY_10 and click Claim credits.
Chapters
0:00 Intro
3:07 The economics of AI
6:20 Why a cache comes first (Valkey)
10:30 Semantic hits vs. semantic misses
12:50 Semantic caching
14:20 Semantic routing
16:20 Tools: BetterDB, Langfuse, LiteLLM, OpenRouter, Semantic Router
18:20 Demo: Template Scout on Aiven Runtime
23:40 Demo: chat + model routing
27:00 Inside Valkey: cached repos + conversations
31:50 Q&A: embedding models
CONNECT WITH US
Website: http://aiven.io
LinkedIn: https://www.linkedin.com/company/aiven/
GitHub: https://github.com/aiven
X: https://twitter.com/aiven_io