Operations | Monitoring | ITSM | DevOps | Cloud

Back in 2016, Jonathan Ross founded Groq, the AI chip startup, which went on to enter a non-exclusive licensing agreement with NVIDIA for Groq’s inference technology (as part of a $20 billion deal). The name ‘Groq’ is commonly confused with X (formerly Twitter)’s Grok, which was launched in 2023 as a Gen AI chatbot. As demand for real-time AI continues to grow, inference has become one of the most important and expensive parts of the machine learning lifecycle.

If you’ve spent any time in developer tooling lately, you’ve probably noticed a pattern: every product is rushing to add a chatbot, an AI summary, or some kind of “magic” button. We get it — it’s tempting. But at Scout, we’ve been deliberately taking a different approach. Instead of building AI into our product first, we’ve focused on making Scout’s data accessible to the AI tools you’re already using.

The best QA engineers are always asking themselves (and others around them) what might break. When engineering teams shifted to agile delivery, that mindset largely moved out of dedicated roles and into the background. Automated testing took over the repetitive work, developers owned quality end-to-end, and velocity improved. What didn't carry over was the habit of looking at a feature and asking how a real user, an edge case, or unexpected load might expose it.

The wrong people got the most attention from Jensen Huang’s comments last week. Huang told the All-In Podcast that he’d be “deeply alarmed” if a $500,000 engineer consumed less than $250,000 in AI tokens annually. Within 48 hours, the discourse collapsed into a compensation debate.

For the past few years, the narrative around Artificial Intelligence has been dominated by what I like to call the "magic box" illusion. We assumed that deploying AI simply meant passing a user’s question through an API key to a Large Language Model (LLM) and waiting for a brilliant answer.