Links/2026×/#quality×/5 entries

It's difficult to find reasonable, pragmatic takes on using coding agents these days. Ray Myers is a good voice in that space right now and I thought this was a great survey of what people in the industry are actually saying about reading AI-generated code.

but those seem like weak-sauce countermeasures [my addition for clarity: (training employees to recognize LLM manipulation and deploying judge agents)] if you mean to help someone “fight back” against a “bombing”. They suggest pushing workarounds onto the user and adding more agents. Think for a second. What’s missing?

We could use fewer agents!

This always drives me crazy. More or less every week at work now people are complaining about some negative effect of AI, and the standard answer from leadership is universally and consistently "use more AI". Instead of trying to use more AI to solve our AI-generated problems, what if we just tried using less AI in the first place? Why is nobody asking that?

I understand why the LLM companies don't want us to use less AI. But so far I do not understand why executives who are paying through the nose for their ever-increasing LLM inference bills do not want us to use less AI.

Where are the women?

When we find ourselves citing exclusively men, something may have gone wrong.

I appreciate this take. Most industry conversations are male dominated. My take on this is that it's mostly because being female in public really, really sucks, and speaking up in these conversations is almost never worth the inevitable harassment, bullying, and gatekeeping you're in for. Men simply do not understand what it's like to have their very reasonable technical opinions regularly met with "you should not exist in this industry". I know that's not true, but hearing it too often is bad for my mental health, so I mostly stick to my read-only corner of the internet over here and do not engage in public conversations. I've been bullied my whole life and learned a long time ago that the best way to deal with bullies is to just remove their access to you. You can't get bullied if the bullies can't contact you.

Frowning on those who live in the present is a luxury that evaporates with proximity to the pager.

This is the first time I've seen this "proximity to the pager" phrase, but I love it. I think it cuts to the core of what is so divisive in the industry right now. In my experience, the people who are most enthusiastic about "AI" and agentic coding and all of that stuff are the ones who are nowhere to be found when prod goes down. Conversely, everyone I know who has ever had to work after hours to fix a downed system that people are paying for is much more realistic and honest about the current capabilities of coding agents.

They then hand those constraints down to our department to do their homework for them work out the details.

It does feel strange that OpenAI and Anthropic constantly claim they are building a Superintelligence capable of doing real work, yet most of the industry is now spending most of its time doing the real engineering work of making the LLMs they produce do anything actually useful. Un-guided and unrestrained, LLMs do far more harm than good, but we've all been effectively mandated to use them day in and day out, so now we spend most of our time just learning how to build better harnesses, guardrails, and systems to try to cajole them into doing meaningful work.

Rigorous automated testing is much more feasible than is generally realized. This is largely a failure to budget for coaching and training.

This is definitely true in my experience. At the risk of sounding like a gatekeeping asshole, I do believe that people who write poorly tested code (or did before AI could do it all for them), both fundamentally misunderstand how to write good test-driven software and dramatically underestimate the cost of all the manual verification steps they do instead. All the engineers I know who write bad or inadequate tests spend ridiculous amounts of time manually QAing their own software. There are some true agents of chaos out there who just yolo code into production, but I would say most software engineers, even very mid ones, do want to confidently answer the question "does this code work" before they ship it. I think the people who don't believe that automated testing is the fastest way to do that just don't really know how to do it well and at this point are too embarrassed or set in their ways to admit they need to upskill.

They emphasize speed of code gen as a proxy for value.

This is a really deeply embedded misunderstanding in the industry. The velocity at which your software is produced doesn't tell you anything at all about the value it delivers to users. But because it is easy to measure many teams have massively over-indexed on it and mistake velocity for productivity, which is unfortunate because it's really easy to juice your own velocity metrics while actually causing damage to project outcomes. But that doesn't matter when nobody is measuring outcomes and only velocity is rewarded.

This is the best “AI usage” policy I’ve seen yet. I would love it if projects I worked on would adopt similar conventions and standards.

you as an author need to take the time to make sure that all of the ideas in the writing are the ideas that you personally intend to convey (including the structure and wording that determines which ideas are emphasized) and that the documents are a good use of your readers’ time.

The thing that frustrates me the most about how AI is used in the contexts where I’m exposed to it is when people try to pass off what is clearly unedited slop as their own work. Rather than seeing AI as a tool to assist with human productivity, too many people are trying to make it replace human productivity, but all they’re really doing is unloading their work on some other human. I feel pretty strongly that asking someone to read AI generated output is rude and inconsiderate. You are effectively asking them to fact-check, edit, summarize, and distill a jumbled bag of claims that may or may not be true for you. To the author, if feel like they saved a ton of time producing the document or message or whatever they needed to write. To the person on the receiving it’s another chore. To be fair I have mostly stopped tolerating this behaviour.

You will confuse your readers (and waste their time) if you present them things that are not genuinely representative of your thoughts. In some cases, readers will recognize and call out the incongruous parts; in others, they will be misled as to what your actual thoughts are.

This cuts to the core of the issue, I think. I'd strongly argue that we shouldn't be outsourcing our actual thinking to AI. It's so tempting to just post whatever it generates because it creates the illusion of productivity which is highly rewarded in many white collar jobs. Most managers reward velocity and not quality. The problem is certainly that the incentives are wrong in the first place, but until those are corrected we have to rely on our own self-discipline to resist the urge to just ship more slop for the sake of seeming more productive.

Pascal once wrote, “I have made this [letter] longer than usual because I have not had time to make it shorter.”

I love this quote, but I always thought it was Mark Twain's. I guess a lot of quotes are misattributed to him.

If you are producing a longer piece of writing from a shorter prompt, consider instead just sharing the prompt itself.

I like this suggestion.

This was a pretty scathing indictment of the tech industry but I think on point. It says a lot of things that you’re not supposed to say out loud right now but that everyone is thinking. Like:

AI makes projects with weak engineering culture fail much faster.

and

In every team, there are competent people who make the project possible. There are also people who essentially make it harder for everyone else.

I don’t actually think anyone benefits from being mean about it, but this is generally true in my experience. Not just in software but in life generally, sadly.

There used to be a time when people sat down and talked about how they'd do something. Now they can just prompt an agent for a few hours and open a PR.

The most tragic aspect of this way of working is that, to the untrained eye, it works.

It does appear to work and I think this is the most dangerous part. Something I really struggle with lately is finding the right balance between excitement and gatekeeping. On one hand it’s really cool to see more people with different ways of seeing things coming up with software solutions to their problems. On the other hand I die a little inside every time I see what’s under the hood, especially when I’m asked why it doesn’t quite work.

This project has become so convoluted, with so many layers and services, that no one on your team could possibly start to understand what's going on.

So, what do you do?

Fixing it would require such a colossal amount of work that it would be impossible to even start justifying it to anyone in management.

It’s always been frustrating to me that “management” (in my experience it's more often product teams than management who control engineering time, but same concept) don't understand the importance of untangling these kinds of architectural messes. "Tech debt" work often gets prioritized alongside regular product development and engineers are constantly forced to justify its "business value", but this makes absolutely no sense and fundamentally misunderstands the cost and consequences of carrying technical debt.

More often than not there is no direct value (in the business sense) in paying down tech debt, but it's a non-negotiable prerequisite for getting literally anything else done. Like real debt, you don't get anything new when you pay it off -- you already consumed the value when you took on the debt. But until you do pay it off you can't buy anything else.

At some point, someone still has to know what is going on. And that's the most valuable person on the team.

It has become very annoying to be this person. I used to just go in and fix things and make software work. Now I have to stage an intervention first.

I’d never heard about this conference before, the talks look super interesting. I wish more people in the industry cared about working software. The bar seems to only be getting lower in this age of AI. I always love finding a batch of new people to follow and new conferences to daydream about attending.

This is a great write up about how Airbnb ships AI features that actually work. Basically, the answer is they test them. What a concept! It sounds really dumb and obvious when you say it out loud, but it’s been surprising to me how many teams are shipping LLM-based systems to production with no test harnesses or evals at all.

Expect to spend a meaningful share of your total project effort on evaluation. This is not unnecessary overhead, it’s how you build products that actually work.

This is anathema to the way of working that the types of people who love AI default to. The industry is overrun right now with enthusiastic “builders” who care much more about velocity than quality. AI has made a lot of people think shipping software is easy because shipping vapourware demos has become 1000% easier than it used to be. But a prototype is not a product. Making software that works for you on the happy path is genuinely easy now. But making it work for someone else on the actual wild internet is still extremely difficult. As a user of many software products, I can say I wish people cared a bit more about whether their AI features worked in production.