Links/#gen-ai×/2 entries

This is a great write up about how Airbnb ships AI features that actually work. Basically, the answer is they test them. What a concept! It sounds really dumb and obvious when you say it out loud, but it’s been surprising to me how many teams are shipping LLM-based systems to production with no test harnesses or evals at all.

Expect to spend a meaningful share of your total project effort on evaluation. This is not unnecessary overhead, it’s how you build products that actually work.

This is anathema to the way of working that the types of people who love AI default to. The industry is overrun right now with enthusiastic “builders” who care much more about velocity than quality. AI has made a lot of people think shipping software is easy because shipping vapourware demos has become 1000% easier than it used to be. But a prototype is not a product. Making software that works for you on the happy path is genuinely easy now. But making it work for someone else on the actual wild internet is still extremely difficult. As a user of many software products, I can say I wish people cared a bit more about whether their AI features worked in production.

This was a great read, an interesting takeaway is that we should be thinking about interacting with APIs differently when writing "agents" (I like Simon Willison's definition of "agent" as an LLM that runs "tools in a loop to achieve a goal").

Normally a human user of an API knows exactly what they want and makes a specific request for it. LLMs have no idea what they're looking for (they should have some means of validating or benchmarking the responses they get, but that's another too-often-ignored problem for another day), so they need to be able to iterate faster. Instead of sending a single (AI-generated) request, getting some (AI-generated) response, checking it, then repeating, we should be sending multiple requests and evaluating multiple responses in parallel before converging on the "best" answer. This means adding much faster and more robust support for branching queries and rollbacks, since in the world of agents these are extremely common compared to the rare cases they're needed during human interaction.