#ai/22 entries

Well this is awkward. Literally yesterday I wrote “the hard part of software engineering was never writing the code.” I concede that that way of describing what’s happening to the software industry is a bit lazy and exaggerated.

Nobody knows how the AI revolution will play out in the end, but it is clear many aspects of work and life will be transformed—including programming.

This definitely feels true to me. My day to day is already completely unrecognizable compared to a year ago and I wouldn’t have known how to even begin describing the nature of my job now if you’d asked me to only a few months ago.

If coding is easy, how come programmers were in high demand, and have demanded large salaries for years (even before ZIRP)?

This is answerable though, in terms of economics. How hard a skill is isn’t what determines its price. It’s true that a very small proportion of the labour force is willing to put up with solving abstract symbolic logic puzzles for 40 hours a week, but the main reason software engineers are so highly paid is more because the marginal cost of scaling software is effectively zero than anything to do with the work itself. Lots of jobs are harder and pay less.

I have met many programmers throughout my career, and very few of them want to talk to stakeholders, much less customers (exceptions are freelancers and founders, especially of software development shops). And, “having clarity on the priorities” boils down to “just tell me what to do and don't switch it up every two days

Relatable.

Whoever you are, don't outsource your understanding, judgement, empathy and taste to AI.

This feels like the most important takeaway for this moment. What drives me more crazy than anything else these days is people trying to pass off AI generated output as their own. It’s lazy and inconsiderate to ask someone to read piles of slop on your behalf, but that’s not even what I hate the most about it. It’s the abdication of our unique human ability to relate to other humans that gets me. I wish people would stop pretending like generated AI output could ever replace human interaction.

This is a great write up about how Airbnb ships AI features that actually work. Basically, the answer is they test them. What a concept! It sounds really dumb and obvious when you say it out loud, but it’s been surprising to me how many teams are shipping LLM-based systems to production with no test harnesses or evals at all.

Expect to spend a meaningful share of your total project effort on evaluation. This is not unnecessary overhead, it’s how you build products that actually work.

This is anathema to the way of working that the types of people who love AI default to. The industry is overrun right now with enthusiastic “builders” who care much more about velocity than quality. AI has made a lot of people think shipping software is easy because shipping vapourware demos has become 1000% easier than it used to be. But a prototype is not a product. Making software that works for you on the happy path is genuinely easy now. But making it work for someone else on the actual wild internet is still extremely difficult. As a user of many software products, I can say I wish people cared a bit more about whether their AI features worked in production.

The thing I hate most about AI is that it has made it very difficult to actually learn something new anymore. I have become convinced that if you already know what you are doing, AI can be quite cool. It definitely allows me to develop more and better software than ever before (which I'm not convinced yet is necessarily a good thing, but it is cool). The problem is, though, if you don't know what you're doing, AI has made it more or less impossible to tell what is true or not anymore. This is actually a huge problem for learning. If you don't know what you don't know, how can you possibly sift through the sheer volume of slop that has flooded the internet on whatever topic you're trying to learn about? This is actually a frustrating problem.

One of the main ways I work with AI since I had a baby 5 months ago is prompting genies from my phone while I'm naptrapped or breastfeeding (did you know newborns feed for ~5 hours per day?! I did not). It’s very sweet and cozy napping with a baby but eventually also boring.

Anyway it’s very cumbersome to review code when building things this way, but I decided that’s mostly fine since for a lot of little personal utilities I have it build I don't really care whether the code is particularly good or not (although I do find lately it writes perfectly passable Clojure, still my language of choice in this age of AI).

I always have it set up CI from the start so the project ships somewhere I can try it out from my phone and give feedback, but sometimes the tools are CLIs or other utilities I want for when I'm back at my desk. In those cases I have Claude write a book about how the project works for me, then I give feedback based on that about what needs to change (either in the project itself or just the prose). The result is that I understand the project and end with an edited book I can reference as documentation. This has been working great for me and has been resulting in many useful personal tools that I'm really enjoying building and using.

To make the process of giving feedback on books (or any other text) easier, I had the AI build itself this little annotation tool. It lets me paste a link in (to e.g. one of the deployed book chapters), then I can highlight particular passages and copy the feedback in a way that is easy for a genie to act on. You could use it for anything that requires taking notes on specific passages of a webpage though.

The problem is that intelligence and trustworthiness are different properties of a model. We measure the first obsessively and the second barely at all, and this week produced a pile of evidence that they can move in opposite directions at the same time.”

This was a great read. I also believe this to be true and am equally concerned about the consequences.

The reality is thus: the people in charge either have no plan, or see no path forwards other than keeping their heads down.

This is interesting and unsurprising, it seems like it’s just human nature to fall in line, regardless of how absurd the groupthink is. I’ve definitely seen it in every project I’ve been a part of over the last 3 years or so since LLMs became mainstream. Suggesting that we are not, in fact, on the brink of a revolution and should just focus on building something that actually works is heresy.

Frequently the failure is not related to AI itself, but rather that companies are terminally bad at running software projects effectively, and as I have remarked previously, AI projects are subject to all the failure modes of normal projects plus you can get everything right and then still fail because of the method's novelty.

This is what people like me have been saying since the very beginning. The first DORA report on AI adoption came to the same conclusion — AI is just another tool. It will amplify whatever practices you already have in place, good or bad. Most companies have no idea how to ship software and AI doesn’t change that.

They cannot buy sensible software, hire competent talent, communicate honestly with executives about the state of projects, or undertake any sort of sensible initiative.

It’s a shame to hear from someone who has sat at many tables with the big wigs that it has come to this. Again, not really surprising, but it’s unclear what the path forward is.

The intersection of AI tooling and open source projects is interesting to me. I’ve also come around to the reality that LLM-powered coding agents like Claude code or codex are pretty clearly game changing when it comes to how software gets produced.

AI is a tool, just like other tools we use. And it's clearly a useful one.

It may not have been that "clearly" even just a year ago, but it's no longer in question today.

I also agree this wasn’t true even a year ago, but the last couple of generations of models have seen some sort of threshold crossed that we didn’t know was there and they are legitimately very useful in my day to day work now.

They also fundamentally change the nature of open source software development. Now that writing code by hand is virtually an obsolete skill, where does that leave maintainers? And what is the point of open source?

Sure, the social angle of working on open source is important and often a very motivating part of the project, but in the end that's a side benefit, not the point of the project.

This was part of it, and a big part of what I love about it. To me it’s really enjoyable working on interesting projects with a globally distributed team of people who also enjoy the work. Part of it was also performative, a way to demonstrate publicly that you could write decent software. I think that is mostly over.

In the kernel community we do open source because it results in better technology, not because of religious reasons.

Maybe that just leaves us with this — writing open source because it produces better results. This doesn’t strike me as universally true though. I guess it depends what you mean by “better”, but I can think of many examples of software where the proprietary version is much better than the OSS one.

I always appreciate these kinds of candid analyses. I’ve become convinced that all the various things people refer to as “AI” are indeed very useful, but they don’t obviate the need for actual engineering. At this point it’s been many years since LLMs became broadly available and the benefits remain largely hypothetical and in the future but the harms are very much real and present today.