When AI writes almost all the code
This week I read a piece of news that made me stop for a moment. Anthropic, the company behind the AI Claude, has updated the prospectus for its stock market listing. Among the financial figures, which is what almost everyone looked at, there were two facts that seem far more important to me.
The first: in September, more than 90% of contributions to the company's code were written by its own AI. And the amount of code each engineer contributes per day is more than 45 times what it was in 2024.
The second: the company itself admits it could be very close to a threshold defined in its own internal safety rules. In other words, using AI to build a better AI, which in turn helps build an even better one, faster and faster.
“We may imminently meet our Responsible Scaling Policy's definition of ‘dramatically accelerated progress attributable to AI.’”
In the same document they add that they don't expect to be able to let their AI design the next generations on its own, without people overseeing it and understanding what it does. And on their own website they make a point I liked: counting lines of code almost certainly overstates the real productivity gain.
The risk that struck me most
It isn't that AI becomes very clever. It's something simpler and more of an engineer's concern: that progress moves faster than the ability to measure it. The data the company uses to track that pace arrives with a delay, so it might not notice a sharp acceleration until it has already happened.
Imagine driving at night with a speedometer that shows how fast you were going ten minutes ago. On a quiet straight road, nothing happens. The trouble starts exactly when you speed up.
What I learned in aviation
When a new aircraft is tested, nobody goes out on day one and flies it to the limit. You “expand the envelope” step by step: a bit faster, a bit higher, a slightly tighter turn. And at every step, before taking the next one, you look at the data. The underlying rule is simple: you never fly faster than you can measure. If the instruments lag behind, you don't accelerate. You stop and fix the instruments.
That's why I think it's a good sign that an AI company says this in an official document, in front of investors, rather than in a nice press release. What remains to be seen is whether it acts on it when doing so costs speed against the competition.
What does this mean for aviation?
Aviation is probably the industry that has learned best to distrust speed. And I think this news raises two uncomfortable questions for it.
We certify things that stand still. An aircraft is certified with a frozen design: you prove that this specific version is safe, and any change goes through the process again. In on-board software, every line has to be explainable: which requirement it comes from and which test verified it. The standard doesn't care who wrote it. What it won't accept is that nobody knows why it's there. An AI that improves itself is, by definition, something that doesn't stand still. The European aviation safety agency already has a roadmap with levels, from AI that only informs the pilot to advanced automation with little or no real-time human intervention. And it warns of something I agree with: simply putting a person “in the loop” is not enough on its own.
The bottleneck moves to verification. If writing software is 45 times faster but certifying a change still takes months, the gap between what can be built and what can be flown will grow enormously. There will be pressure to speed up certification. My prediction is that AI itself will end up generating much of the tests and documentation. And then the age-old question gets harder: who checks the checker?
And for defence?
Here the conversation is more serious, because in defence speed isn't just a commercial advantage. It's a military one. Whoever observes, decides and acts first usually wins. And in that chain, the slowest link is almost always the person.
That's the dilemma. If your adversary automates more and decides faster, the temptation is to move the person aside so you don't fall behind. An analysis I read on this explains it very well: as AI gains autonomy, the human goes from deciding to supervising, and if nobody validates continuously, the system ends up reasoning on its own conclusions. It becomes circular.
Lawmakers are starting to react. In the United States, the draft 2027 annual defence bill approved by a Senate committee requires an appropriate level of human judgement at all times, the ability to interrupt or shut down a system, records of how targets are selected, and that AI never decides to launch a nuclear weapon. It's still a draft, but it's heading in the right direction.
What makes me think most is the resemblance to arms control. Anthropic says on its website that slowing down would probably be a good thing, and that it would do so if others slowed down too, verifiably. That is, almost word for word, the logic of disarmament treaties: nobody stops unless they can verify the other side has stopped too. What made those treaties work wasn't trust. It was inspections.
On Eurofighter I worked with four countries that had to agree on every change. From the inside, it sometimes felt incredibly slow. Over the years I understood that this slowness was also a safety system: it forced someone else to look at every decision before it became reality.
Where I think this is heading
- In aviation, AI will move quickly into everything around flight (maintenance, planning, documentation, testing) and slowly into what actually flies. I think that's how it should be.
- In defence, the big debate of the coming years won't be whether to use AI, but where to put the person and how to prove they really decide rather than just sign.
In aviation, we never leave the autopilot alone in the cockpit. Not because it isn't good, but because someone has to watch the instruments and someone has to answer for what happens. A machine can do the work. The responsibility is still ours.
A question for you: in your work, which part are you already handing over to AI, and how do you check that it turns out right?
Sources: Anthropic's stock market prospectus, as reported in the press, and its page on AI that helps build itself; the European aviation safety agency's AI roadmap; the draft 2027 US defence bill and an analysis published in a US defence journal. The opinion and predictions are my own.
← Back to the blog