Links/#ai-agents×/6 entries

I was excited to see that this is the first module in a new Coursera course I'm doing. I've long thought that ethics are missing from software development, and I think it's more important than ever to care about the consequences of what we're doing and building in this age of AI. This course on building agents literally opens with this statement:

This module is for professionals and data scientists aiming to build responsible AI.

I'm really heartened to see that building responsibly is becoming a legitimate concern, at least in some corners of the industry.

By bridging theory and practice, the program empowers you to lead initiatives that prioritize accountability, ensuring your AI systems deliver immense value without compromising integrity or public trust.

I'm looking forward to this.

To understand what a swarm of tens of thousands of automated actions did…

This is a really interesting preview into the scale and type of attacks that are bound to become more common in this age of ai.

Thanks to this approach, we were able to do in hours what would usually take days, and match the adversary's speed.

Useful perspective, if you’re not already using AI to secure production, you’ll have to start. It makes sense that there would be no way for a team of unassisted humans to keep up with the sheer scale of a frontier-model-driven attack.

When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure.

This is the inevitable result of the so called “guardrails” frontier labs have placed on their models. It’s also a bit egregious that they themselves are free to let their most powerful models run in the wild with no such guardrails in place. I don’t think we want to live in a world where the people currently deciding who has access to these tools are the ones who currently do.

This experience points to a gap worth planning for. We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.

This is an extremely important takeaway. If your plan for responding to these incidents is to use a frontier LLM via their commercial API, you are screwed.

I really love seeing more and more serious engineering being put into making LLMs actually work in production. They’re too cool a tool not to use or integrate into consumed software, but nobody actually wants more shitty chat bots. Making them work reliably to actually make software better requires structure and effort, and this is one cool way of thinking about it I hadn’t come across before.

Without additional artifacts beyond what human-driven CD requires, agent-generated code accumulates drift and technical debt faster than teams can detect it.

This is undeniably true at this point and it’s cool to see frameworks and systems emerging for addressing it. This is the first time I’d heard of “agentic CD” and it’s super interesting!

This really hit home for me. Lately I've been feeling very overwhelmed and bored at work, feeling like I'm just drowning in AI slop and cannot possibly keep up. AI-generated PRs are flying through our previously robust process, production problems are way up, most of what I read day to day is not AI-generated, and it's frankly exhausting.

It feels like something has shifted just in the last few months. I'm not sure if it actually has, or if my organization has just taken up AI to a new extent recently, but somehow the day to day enjoyment I used to get out of programming has mostly disappeared. I've been reflecting on why that is and what I might need to change going forward, but I'm not sure what that is yet. For now I enjoy lamenting the old way of doing things with other old fogeys around the internet.

This was a great read, an interesting takeaway is that we should be thinking about interacting with APIs differently when writing "agents" (I like Simon Willison's definition of "agent" as an LLM that runs "tools in a loop to achieve a goal").

Normally a human user of an API knows exactly what they want and makes a specific request for it. LLMs have no idea what they're looking for (they should have some means of validating or benchmarking the responses they get, but that's another too-often-ignored problem for another day), so they need to be able to iterate faster. Instead of sending a single (AI-generated) request, getting some (AI-generated) response, checking it, then repeating, we should be sending multiple requests and evaluating multiple responses in parallel before converging on the "best" answer. This means adding much faster and more robust support for branching queries and rollbacks, since in the world of agents these are extremely common compared to the rare cases they're needed during human interaction.