Insight / signal
AI adoption is becoming a scoreboard, whether teams are ready or not
Do not ask whether your team is using AI. Ask what work AI changed, what it cost, what risk it added, and whether you can prove the result was better.
The most useful AI announcement this week was not a new model.
It was a field in an API.
On 29 May, GitHub added AI adoption phases to the Copilot usage metrics API. Companies can now see whether developers are code-first, agent-first, or multi-agent users — based on actual Copilot surface usage over a rolling 28-day window.
Boring? A bit.
Important? Very.
Most companies are still treating AI adoption as a binary question. Did we buy the tool? Did people log in? How many active users this month?
That is not adoption. That is attendance.
GitHub’s new metric is more honest because it separates behaviour. Someone using autocomplete in an IDE is doing something completely different from someone sending work to a cloud coding agent, reviewing AI-written pull requests, or running tasks through the CLI. Different workflow. Different cost. Different risk profile. Different trust level.
GitHub now describes four phases:
No cohort — did not meet engagement criteria.
Code-first — usage mainly completion or IDE agent mode.
Agent-first — engaged with one GitHub agent surface.
Multi-agent — two or more agent surfaces, or the Copilot app.
You can feel the management dashboard forming.
Not the polished one from a vendor deck. The real one. Who is still using AI as fancy autocomplete? Who is handing off actual tasks? Who is letting agents touch pull requests? Who is building dependency? Who is racking up cost without any measurable improvement?
That cost question is getting harder to sidestep.
GitHub also confirmed this week that Claude Opus 4.8 in Copilot launches with a 15x premium request multiplier until usage-based billing rolls out on 1 June. Auto model selection routes based on task, utilisation, quality, reliability, and token efficiency — which is another way of saying “this is complex, it has a price, and you should pay attention.”
TechCrunch covered developer backlash on Copilot’s token-based billing at the same time.
Strip out the noise. The message is simple: AI work has a meter attached now.
That is fine. Metered work can be excellent work. But it means you have to know what you are buying. A flat software licence is easy to understand. A team of people triggering premium model calls, background agents, memory writes, code reviews, and CLI sessions is not the same thing.
For the last two years, the loudest AI conversation has been about capability. Can it write code? Can it build a site? Can it replace a role?
The more useful conversation now is operational.
Did cycle time improve?
Did review burden go up or down?
Did senior people get more time for the work that matters, or more time cleaning up plausible nonsense?
Did costs move from payroll to tokens without anyone noticing?
Did the company gain capacity, or just a more expensive way to look busy?
Those are harder questions. They are also the right ones.
There is a memory angle here worth flagging too.
A few days before the cohort update, GitHub added more controls for Copilot Memory: better deletion options, a repository-level off switch, clearer prompts showing whether memory is a user preference or a repository fact, CLI controls to turn it on, off, or check status.
Again, dry. Again, important.
Memory is where AI tools stop being disposable chat windows and start becoming company infrastructure. If a tool is remembering user preferences, repository facts, client context, decision history, or team habits — that memory is a trust boundary. You need to know what it stored, who can see it, when it is stale, how you delete it, and whether it should have been captured at all.
That is not a prompt-engineering problem.
It is an operating model problem.
The average business owner does not need another AI webinar about embracing change.
They need answers to plainer questions:
Where should AI be in our workflow, and where should it be blocked?
Which tasks are safe for agents to run without a human checking the output?
What does human approval still need to cover?
What memory is useful, and what memory is dangerous?
What are we actually spending?
What got better?
That is the practical gap. And it is where the real post-agency opportunity lives.
The old agency model sells outputs: campaigns, assets, reports, content. AI makes output cheaper, which gets awkward if your model depends on charging premium fees for production theatre.
The next useful thing to build is the operating layer. The system that decides what AI touches, how, who approves it, what evidence gets kept, and how you know the whole thing is working instead of just spinning.
For any business trying to make this practical right now, here is where I would start:
Adoption — who uses AI, how often, and at what depth?
Work type — what tasks are being assisted, delegated, or fully agent-run?
Cost — token costs, licence costs, infrastructure costs per workflow?
Quality — what gets accepted, rejected, rewritten, or rolled back?
Risk — what can agents touch without a human approving the result?
Memory — what is being stored, where, and who controls deletion?
Outcome — what actually improved for the business?
That last one is the one most teams skip.
If you cannot connect AI use to an operational outcome, you are probably measuring activity and calling it adoption.
The big AI story is not that agents are coming.
They are already here. GitHub has enough surface area to classify how people move from autocomplete into multi-agent workflows. Google is adding background information agents into Search. Memory is becoming a product feature, not a research curiosity. Token costs are creating public arguments.
The work now is less exciting than the demo videos.
Dashboards. Permissions. Budgets. Memory controls. Review loops. Handoff rules. Proof.
Good. That is where serious businesses can actually do something.
Do not ask, “are we using AI?”
Ask: what work did AI change, what did it cost, what risk did it add, and can we prove the result was better?
That is the scoreboard.
And whether teams like it or not, it is coming.