Half the Four-Hour Tasks Still Needed a Human. That Is the Number That Matters.

OpenAI announced on Saturday that it had hit its "automated research intern" milestone, and hit it ahead of schedule. The number everyone quoted was 3.1: as of mid-August, OpenAI's research organisation was running 3.1 agent-workdays for every human workday.

The number nobody quoted was in the same disclosure. Over the previous six months, more than half of the successful tasks in the four-to-eight-hour range still involved at least one human intervention.

Successful ones. The failures are not even in that denominator.

That second number is more useful than the first, and it deserves to be the one the industry starts reporting.

OpenAI debunked its own headline, which is to their credit

The 3.1 figure measures agent-workdays of runtime, not agent-workdays of progress, and OpenAI said so directly. Their own caveat is that research progress will not necessarily scale with raw activity metrics, because runtime includes parallelisation, redundancy, failed attempts and heavy human steering.

Read that list again. Two of the four things inflating the number are the agent doing work that did not count, and a third is a human holding its hand.

Very few companies publish a metric and then immediately tell you why you should not over-read it. Most vendors would have shipped the 3.1 on a slide and stopped. So credit where it is due, and then let us take them at their word, because the caveat is the actual finding.

Why intervention rate beats success rate

Almost every agent evaluation you will see reports completion. Did the task finish. Did it produce the right answer. That framing comes from software testing, where a passing test costs you nothing to observe.

Agent work is not like that, because the unit that gets consumed is not compute, it is your attention.

Say an agent takes on a task that would have cost you six hours. It completes. Somewhere in the middle it needed you: an ambiguous instruction, a wrong turn, a permission it did not have, a judgment call it could not make. That is one interruption, and an interruption is not a five-minute cost. It is five minutes plus the twenty it takes you to get back to where you were, multiplied by whatever you were doing when it arrived.

Two interruptions across a six-hour delegated task means the task did not save you a morning. It fragmented one. The work still got done, and you are still tired.

Success rate cannot see any of this. It records a win. Intervention rate is the only number that tells you whether the delegation was real or whether you just supervised something slowly.

And the corollary, which is where most people get this wrong in practice: an agent that runs unattended for two hours and produces something 80% right is often worth less than one that produces something 60% right in ten minutes, because the second one leaves you time to fix it and the first one has already eaten the window.

Look at where the delegation actually grew

OpenAI listed the areas where delegation is growing fastest: coding, infrastructure work, technical troubleshooting, monitoring, experiment execution. High-level planning remains minimal.

There is a single property running through that list, and it is not difficulty. Infrastructure work is not easy. The shared property is that verification is cheap. The test suite passes or it does not. The service comes back up or it does not. The experiment produces a number you can read.

High-level planning sits at the other end. Checking whether a plan is good costs about as much as making the plan, which means delegating it saves nothing even when the output is decent.

That is a genuinely portable rule, and it is more useful than any capability benchmark: delegate the work where you can tell in seconds whether the output is right. Not the easiest work, not the most repetitive work. The most cheaply verifiable work. It is why agents took over inbox triage and code review scaffolding before they touched strategy, and it will keep being why.

Instrument your own intervention rate

If you are running agents in a business, the good news is that this metric is trivially available to you and almost nobody bothers to collect it.

Log every time a human has to touch a delegated task, and sort what you find into three buckets.

Clarification. The agent came back because the brief was ambiguous. This is your fault, it is cheap to fix, and it should trend to near zero within a couple of weeks of a task becoming routine. If it does not, the task is not as well-defined as you think.

Correction. The agent produced something wrong and you fixed it. This is the one to watch. A flat correction rate over months means the agent is not learning your context, which for us is a memory problem rather than a model problem.

Rescue. The agent got stuck, looped, or went somewhere it should not have. These are rare and expensive, and each one should produce a scope change rather than a shrug.

Then track the trend rather than the absolute. Nobody should expect zero. The question is whether last month needed fewer touches than the month before for the same class of work. If it did not, adding more agents will not help, and you are about to discover that thirteen half-supervised agents are worse than three well-scoped ones.

The honest version of the milestone

OpenAI defined the research intern as a system that completes well-scoped tasks under human direction. "Under human direction" is carrying most of that sentence. The system explicitly does not set research agendas, does not decide which unanswered questions deserve resources, and does not judge whether a result is significant enough to scale.

Their next target, a fully automated AI researcher by March 2028, is defined by exactly the capabilities that are missing: research judgment, hypothesis selection, experimental design, long-horizon execution, and interpreting ambiguous evidence.

Which tells you what has to happen between now and then. Not longer runs, not bigger context windows, not more agent-workdays. The intervention rate on long tasks has to fall, because every one of those missing capabilities is just a description of the moments where a human currently has to step in.

That is the scoreboard. Everything else is runtime.

Want to test the most advanced AI employees? Try it here: https://Geta.Team