Estimating a Drupal upgrade is tedious, and it is important to get right. It takes hours of composer exploration, module inventorying and risk thinking before you land on a range the client can trust. So when I was facing a Drupal 10 to 11 upgrade on a client project, I tried something I had been thinking about for a while: I let two AI agents produce independent estimates on the same codebase, with the same instructions, and compared them against my own gut feeling.
The models were Claude (Fable 5, high) for the short window we had access to it, and Codex (GPT-5.5, high), both run as agents in the terminal with an estimate skill I had built. The skill describes how an estimate should look: ranges rather than point numbers, a staged plan if needed, QA steps and open questions. Neither of them got any tool tips beyond that. The full prompt was a few lines:
We will estimate this drupal 10 to drupal 11 upgrade.
Also contrib modules and our two custom modules.
Remember contrib modules can exist in multiple placed.
Include plan on if we need to split upgrade in different stages.
Write the estimate to correct place in the vault and add claude/codex
to title so we know its yours.
Ask questions if needed and plan and document QA steps.
Also feel free to check themes and db if needed.The project in brief, without details that identify it: a larger corporate site on Drupal 10, multilingual and multidomain, with around fifty contrib modules and custom modules written by different developers over several years.
The numbers
My own gut estimate, written down before I read either result, came in below both of them. Claude landed roughly a quarter above my number, with a coherent range and a weighted total. Codex landed around double, with a wider range and a recommended quote interval on top of that.
Token usage was the inverse of what I expected: Claude used around 100k tokens, Codex around 169k. The model that did the more thorough job used considerably fewer tokens. Where they went is what this article is about.
The interesting part is not the gap, but why
Two estimates that differ by nearly a factor of two on the same codebase can feel like something went wrong. But when I put them side by side, almost the entire difference came down to one thing: how they handle uncertainty.
Claude converted uncertainty into facts. It installed upgrade_status, ran a deprecation scan across the existing custom code, looked up release status on drupal.org for every blocked contrib module and checked the database. The scan showed that the custom code was essentially clean, a single deprecation. The database lookup revealed that several of the apparent blockers were not blockers at all: a module whose function had already moved into core, an old editor no longer used by any text format, a library whose only feature was disabled. Each discovery shrank the estimate.
Codex priced the uncertainty instead. It stayed strictly read-only, running composer why-not and drush status, never scanning the code. The custom modules got a medium-high risk level based on assumptions, and the blocker list was the composer output taken at face value. That is not wrong, different models solve problems differently, but it is cautious. Every assumption Claude verified stayed in Codex's range as risk that costs time.
The cheaper estimate came from the model that did the most research.
Same sentence, two interpretations
What makes this more than a numbers comparison is that both got exactly the same latitude. The prompt said feel free to check themes and db if needed. Claude read that as permission to install an analysis tool, use it and clean up afterward. Codex read the same words as a read-only mandate.
Neither interpretation is wrong. In another context I would be grateful for the agent that does not install things in a client project without asking. But it means the difference between the estimates was not fundamentally about Drupal knowledge, it was about initiative: how much an agent is willing to do to avoid guessing.
A small detail in the same vein: the prompt ended with ask questions if needed. Claude asked about who handles the deploy. Codex chose not to ask anything. Both delivered a section of open questions at the end of their document. Agents can sometimes be better at writing questions than asking them.
What each did well
Claude produced the estimate I would probably hand to a client: a clear release breakdown, risks priced in time, a QA checklist referencing actual details in the project, and a plan that decoupled our work from the external vendor's timeline. It also used context from our knowledge base, including a scope decision from the same day, and scoped the estimate accordingly.
Codex caught things Claude downplayed: contrib modules duplicated in two places in the repo structure that need a decision on action, a version difference between local code and production that should be sorted out before estimating further, and a cleaner three-step release model. Its QA command list was also the better of the two.
And on the big picture they were in striking agreement: split the upgrade into a hardening release on Drupal 10 before the actual 11 upgrade, domain handling is the biggest risk, the same modules should go entirely. When two models working independently land on the same plan structure, I trust the shape of the plan more. The hours are still a separate question.
What I take from this
My own estimate was closest to Claude's number, and I don't think that is a coincidence. I have worked with Claude across several projects and built my processes with it in mind, so it is not surprising that it matches my gut better than Codex. Codex assumed the opposite and priced it. An estimate is essentially a list of assumptions with a cost attached to each, and the value in AI estimation turned out to lie less in the final number and more in which assumptions get turned into facts before you land on it.
The gap against my own number was instructive in the other direction too. Claude came in above my gut, and when I read why I have to give it credit: coordination, runbooks and two QA rounds that my estimate tends to downplay when I am thinking in pure development time.
And the cost? Both runs together used tokens for a fraction of what the equivalent manual walkthrough would have cost in senior time. Letting two models estimate independently and triangulating against your own experience is cheap extra perspective. I will do it again on the next larger upgrade, and by then it will probably be the estimates that act as each other's baseline.
What this does to the skill
Reading the estimates side by side, I noticed something I should have seen earlier: my estimate skill has gaps. It describes how to inventory a Figma file and find what is not shown in the sketches. It has sections on risks, QA and upgrades, but not a single line telling the model to do research, or how to handle uncertainty.
That explains more than I first thought. The skill gave both models the output contract: ranges instead of point numbers, weighted total, assumptions, exclusions, risks and open questions. Both followed it almost exactly, and it shows in the result: two estimates with nearly identical form and completely different method. The skill governed form. Method was left to the models to improvise, and that is where they diverged.
So the next version of the skill gets its own mode for upgrade estimates, built on what this comparison taught me:
A verification ladder instead of free initiative. Check whether a compatible release already exists for each blocked module, check whether the blocker is even in use, run a deprecation scan on the code we are responsible for. Initiative should not be a personality trait of the model, it should be written into the instruction.
A clear mandate. What the agent may install locally and clean up afterward, and what it must never touch. The difference between the models' interpretations of the same open-ended phrasing became this entire article.
An assumptions register in the output. Every assumption is either verified with evidence or priced in hours. The uncertainty should be visible in the estimate, not baked into the range.
I added a decision split. If an unanswered question can move the total by more than a fifth, the agent should stop and ask, not write a tidy questions section at the bottom of a finished document.
And a rule that has nothing to do with the agent: write down your own gut estimate before you read its answer. That discipline was the closest thing I had to a control group in this experiment, and it cost nothing.
I should also be honest that the new version is itself an assumption. What made the better estimate good was initiative the model took on its own, and it is not impossible that a tightly steered skill suppresses exactly that kind of judgment. The next real estimate will show. And the real verdict comes only when the upgrade is delivered and the hours can be compared against all three estimates.
I am considering putting the skill up publicly once the new version has been tested on another real estimate. Get in touch if you want it, and I will know whether it is worth cleaning up for public use.
I also wrote a separate article about what the token usage taught me about clearer agent instructions.
