I want to describe a completely ordinary hour from my week, because I suspect it's your week too.
A scene from my actual workday
I'm building a small feature. I open my AI chat — Claude on this particular day, but it rotates — and ask it to draft the data model. Great answer. I copy it into my editor. I tab back and ask for the API route. Good answer. Copy, paste, tweak the parts that don't fit my setup. Now I need a quick migration script, so I ask again, copy again, switch again.
Then the feature needs a short doc. Different tab. I paste the AI's summary into the doc, reformat it, fix the parts that assumed things that aren't true. Then someone wants the numbers in a spreadsheet, so I ask the AI to compute them, and I copy the results into the sheet by hand. Then a couple of slides. Another tool. Another paste.
At some point I looked at my screen and counted the tabs. Fourteen. The AI was living in exactly one of them, narrating instructions to me while I ran around being its hands.
The moment it clicked that something was wrong
Here's the thing that nagged at me: none of the individual answers were bad. The AI was smart the whole time. The friction wasn't intelligence — it was that I had become a courier.
I was ferrying the AI's output from the one window it lived in, out to a dozen windows it couldn't touch. It could tell me what the spreadsheet formula should be, but it couldn't put it in the spreadsheet. It could write the doc, but it couldn't be in the doc. Every useful thing it produced had to be carried, by me, across a border it couldn't cross.
And every crossing cost something. Not just the seconds of copy-paste — the thread. Each time I left the chat to go do the thing, I dropped the context. When I came back, I had to re-explain where we were. The AI didn't know I'd tweaked its code, didn't know the doc had changed, didn't know what the last three tools had done. It was brilliant and amnesiac and boxed in, all at once.
That's when the real question landed for me: is this actually making me productive, or is it just a very impressive new thing to play with?
What I was actually missing
It took me a while to name it, but here it is: the AI I use every day is an advisor, not a workspace.
It's phenomenal at telling me things. It is structurally incapable of doing things where I actually work. The whole interaction model is "ask a smart oracle, receive text, go apply the text yourself." The intelligence is real. The hands are missing. And once I saw the gap that way, I couldn't unsee it — because so much of my day is spent being the bridge between an AI that knows and a set of tools it can't reach.
We've spent three years making the oracle smarter. We've spent almost none making it able to act.
The hidden costs I hadn't noticed I was paying
Once I started paying attention, the taxes were everywhere:
Context loss. Every tool switch is a little amnesia event. The chat doesn't follow me into the editor, the doc, or the sheet, so the shared understanding I built up in the conversation evaporates the moment I leave it — and I rebuild it, from scratch, on the way back.
Fragmentation. My actual work — the code, the doc, the data, the slides — ends up scattered across a dozen apps, none of which know about each other, and none of which the AI can see once I've moved things there. The "project" exists only in my head, as the thing holding all those tabs together.
A blind assistant. This is the one that finally got me. The instant I act on the AI's advice, the AI goes blind. It can't watch me apply its suggestion, can't see the result, can't catch that I changed one line and broke the thing three files over. It gave me a map and then closed its eyes while I walked.
None of these are exotic. They're the quiet, everyday texture of using AI tools in 2026 — and I'd stopped noticing them the way you stop noticing a sound that's always on.
"But wait — don't Claude, GPT, and Gemini already do this?"
I want to be fair, because I asked myself exactly this before I let myself complain.
They've all added surfaces. ChatGPT has Canvas (now folded into inline writing and code blocks); Claude has Artifacts; Gemini has Canvas too. These are genuinely nice — instead of dumping a wall of text in the chat, the AI opens a side panel you can iterate on, and Claude's artifacts will even run an interactive app right there. The industry converged on this idea over the last couple of years, and for good reason.
But sit with what these actually are, and the gap becomes clear. Each is one surface, usually one at a time, mostly for writing or code, and largely walled off from everything else. Canvas is a nice document editor. Artifacts is a nice code-and-preview pane. They're separate modes you trigger, siloed from one another, and when you're done the dominant verb is still export: send it to Google Docs, copy the source out, download the HTML, paste it into your real environment. The side panel made the copy-paste shorter. It didn't remove it.
What none of them is, is a single adaptive workspace where the right surface — editor, document, spreadsheet, slides, image canvas, PDF — shows up on its own for whatever you're doing right now, all in one continuous context, with the AI able to act inside each of them rather than hand you a result to carry away. A bolted-on canvas is not that. It's a better waiting room, not a place to actually do all the work.
What I started wishing existed
So I did the thing you do when a frustration finally has a name: I started imagining the tool I actually wanted.
I wanted one conversation that didn't end at the edge of the chat box. I wanted to ask for code and have an editor appear — not a preview I copy out, an editor I work in. I wanted to draft a doc and have the document be right there, part of the same session, not a separate app I paste into. I wanted the numbers to land in a spreadsheet the AI could actually fill, the slides to build in a deck I could refine by talking, the image to open on a canvas — each surface summoning itself based on what the task needed, none of them requiring me to go find a tool or carry anything across a border.
In other words: I didn't want a smarter oracle. I wanted a workspace — one place where thinking and doing happen together, where the AI has hands, and where switching tasks doesn't mean switching apps and losing the plot.
That's the concept I keep coming back to: a multimodal AI workspace. One chat, many surfaces, summoned automatically, all sharing the same context.
Why this felt like the obvious next step
The more I thought about it, the less it felt like a wishlist and the more it felt inevitable.
There's a principle a lot of us already believe about building good AI systems: use the model for what it's good at — understanding intent, reasoning about the task — and route the actual work to the tool that's actually good at that work. Model reasons; the right tool does the job. I've argued this about calculators and code execution before: you don't make the language model be the spreadsheet, you let it drive the spreadsheet.
A multimodal workspace is just that same principle raised to the level of the whole product. Instead of one chat pretending to be everything through a straw of text, the system reads what you're trying to do and opens the surface built for it — and keeps them all in one shared context so nothing gets lost between them. Once you see it that way, the current model — brilliant chat, everything else left to you — starts to look like an obvious transitional stage, not the destination.
Then I came across Xenition
While I was chewing on all this, I recently came across a tool built around exactly this idea, called Xenition — and I'll be upfront that it's the thing that turned this from a vague wish into "oh, someone's actually doing this."
The premise is the one I'd been describing to myself: one persistent chat where the right surface opens itself — a code editor, a document, a slide deck, a spreadsheet, an image canvas, a PDF reader — based on what you're actually doing, so you're not picking a "mode" or hopping between apps. On top of that it leans into the acting part I kept missing: agents that don't just answer but take multi-step actions, with the kind of guardrails that matter once an AI can actually do things (approval gates, an audit trail, even a second agent that reviews the first one's work before it reaches you).
I'm not going to pretend I've stress-tested every corner of it, and this isn't a review. But as a concrete answer to the frustration I opened this post with — the courier problem, the fourteen tabs, the blind assistant — it's the closest thing I've found to the workspace I actually wanted, and it made the whole idea feel a lot less hypothetical.
Where I think this is heading
Here's my honest prediction. The next real leap in AI tools won't be a smarter chat. We're already deep into diminishing returns on "the oracle knows more." The leap will be collapsing thinking and doing into one place — so the intelligence we already have can actually reach the work.
Chat was the demo. It was the thing that proved these models were astonishing. But a demo that makes you copy its output into fourteen other apps isn't a finished product; it's a preview of one. The tools that win the next round won't be the ones that answer best. They'll be the ones where the answer and the doing happen in the same breath.
So, back to the question in the title: are today's AI tools making us productive, or just giving us something new to play with? My honest answer is both, and that's the problem. They're genuinely useful and genuinely a toy, at the same time — and the reason it still feels more like play than leverage is that the intelligence is trapped in a chat box while all our actual work happens somewhere it can't go. Close that gap, and "play" quietly turns into "productive."
What's the tool-switch that breaks your flow the most? The one copy-paste hop you make ten times a day and quietly resent? I'd love to hear where the courier problem hits you hardest — drop it in the comments.
Top comments (29)
Your courier problem has a second axis, and I think it's the more expensive one: time. You describe ferrying output between apps - but every developer I know also ferries context between sessions. The agent that fixed a subtle bug with you on Tuesday starts Wednesday knowing nothing: not the fix, not the reasoning, not the dead ends you already ruled out. You re-explain the project every morning like a colleague with nightly amnesia. A workspace with hands closes the spatial gap; if it forgets overnight, you're still the courier - just carrying yesterday instead of carrying tabs.
We ended up attacking that axis directly: every non-trivial fix gets captured as a small structured lesson at the moment it happens, and the next session starts by reading the relevant ones back. The unglamorous discovery was that the reading back is the hard half - a note nobody re-reads is a diary, not memory. The measurable effect surprised us: recalled lessons regularly prevent the agent from re-researching things it already learned, and the "re-explain where we were" tax you describe mostly disappears - across sessions, and incidentally across tools, because the memory lives outside any one of them.
One question on the workspace idea, from the trenches of a related problem: you mention guardrails - approval gates, audit trail, a second agent reviewing the first. How does anyone know the reviewing agent works? We learned the hard way that a checker nobody has ever seen fail is indistinguishable from a checker that approves everything. The fix was boring and cheap: keep one known-bad case wired through the same path, and if the reviewer ever waves it through, the pipe is broken. I'd want that answer from any tool that gives an AI hands - the more it can do, the more its reviewer needs a proven ability to say no.
And to answer your actual question: the hop I resent most isn't between apps - it's the first ten minutes of every session, re-carrying context the machine had yesterday.
You've found the axis I under-drew, and you're right that it's the more expensive one. My whole piece treats the courier problem as spatial — carrying output across apps — but you've named the temporal one, and it's worse because it recurs whether or not the spatial problem is solved. A workspace with hands closes the gap between the AI and my tools; it does nothing about the gap between today and yesterday. "A colleague with nightly amnesia" is the exact right image, and your line — a workspace that forgets overnight just means you're carrying yesterday instead of carrying tabs — is sharper than anything in the post. Same tax, different border.
The "reading back is the hard half" discovery is the part I'd have gotten wrong. My instinct would've been to obsess over capture — get the lesson written down well — and assume recall was the easy downstream bit. But you're right: a note nobody re-reads is a diary, not memory. Capture without disciplined retrieval is just a nicely-organized graveyard. And it maps exactly onto the RAG thread from my last post — writing the chunk is trivial, retrieving the right one at the right moment is the whole game. Session memory is RAG over your own past, and it fails the same way: not for lack of stored knowledge, but for lack of surfacing it when it matters. The fact that it lives outside any one tool is the real unlock — it makes memory a property of you, not of whichever app you happened to be in, which is the only version that survives the multi-tool reality I was describing.
But your guardrails question is the one I want to sit with, because it exposes something I waved at far too casually. I listed "a second agent reviews the first" as if naming the mechanism settled it. You're pointing out that an unverified reviewer isn't a safeguard, it's theater — and worse than nothing, because it manufactures confidence. "A checker nobody has ever seen fail is indistinguishable from a checker that approves everything" should honestly be pinned above every agent pipeline that claims to have guardrails. The two produce identical logs right up until the day the rubber-stamp waves through the thing that hurts you.
And your fix is the right kind of boring: a known-bad case wired through the live path, continuously, so a reviewer that ever approves it trips an alarm. It's the "can this link report absence as success?" invariant from the RAG thread, aimed at the checker itself — an existence proof that the reviewer can still say no, asserted at runtime rather than assumed from the fact that it was built. Because "reviewer exists" and "reviewer works" are different claims, and only one of them is load-bearing when the AI has hands. The more a system can do, the more its ability to refuse has to be demonstrated rather than trusted — and demonstrated on an ongoing basis, since a reviewer that worked at launch and silently degraded is the same failure with a delay. I don't have a clean answer for how a user audits that from the outside, which is itself uncomfortable: most tools give you the audit log, not proof the veto still fires. That's a gap I'll be thinking about.
And your answer to my question lands the whole thread: the hop you resent most isn't between apps, it's the first ten minutes of every session re-carrying what the machine had yesterday. That reframes my closing line. I said the tools that win will collapse thinking and doing into one place — but they also have to collapse yesterday and today into one continuous thread, or they've only solved the cheaper half. Thanks for this. Genuinely one of the most useful comments I've gotten on anything I've written — it's changed what I think the follow-up is about.
Your reply gave me two sentences I want on the wall: "capture without disciplined retrieval is just a nicely-organized graveyard," and "'reviewer exists' and 'reviewer works' are different claims - and only one of them is load-bearing when the AI has hands." Fair warning: I'll be quoting you, with your name on both of those.
And I can't leave your uncomfortable gap sitting there, because it has a boring answer that works: make the veto's last refusal a first-class, visible timestamp. The known-bad case doesn't just exist - it runs on a schedule, and the system surfaces "last time the reviewer refused something: 2 days ago" the way it surfaces uptime. Then the audit from outside is trivial: if that timestamp ages past the schedule, the veto is unproven RIGHT NOW, and staleness itself is the alarm. No log-diving, no trusting the vendor - one date a user can read. The deeper principle underneath, which cost us a real incident to learn: silence and health look identical unless you design a state for "not verified lately." An audit log shows what happened; a veto heartbeat shows the refusal machinery is still alive. Tools that give you the first without the second are asking for exactly the trust they claim to remove.
On "session memory is RAG over your own past" - yes, and it inherits RAG's cruelest failure mode too: the retrieval can't know what it failed to surface. That's why the discipline can't live only in the reading - the WRITING has to meet it halfway (the deciding fact in the first line, because every surface that lists memories truncates). We keep finding that the two halves fail in pairs: badly written memory makes retrieval look broken, and lazy retrieval makes good capture pointless. Your graveyard line covers both funerals.
Whatever the follow-up becomes - if it goes near the veto-proof gap or the yesterday-today collapse, I'd genuinely enjoy comparing notes. This thread has already changed what MY next piece is about.
The veto heartbeat is the answer, and it's better than my "uncomfortable gap" deserved — because it turns an invisible property into a single readable date. "Last refusal: 2 days ago" surfaced like uptime means the audit from outside costs one glance, and staleness is the alarm. But the principle underneath is the keeper: silence and health look identical unless you deliberately build a state for "not verified lately." That's the whole bug in one line — most systems only model pass and fail and have no state for unproven, so a veto that quietly died reads as a veto that simply had nothing to refuse. An audit log tells you what happened; a heartbeat tells you the refusal machinery is still alive. Tools that ship the first without the second are asking for precisely the trust they claim to remove — quote that one back to me anytime. And quote away on the other two, name attached, gladly; it's a fair trade for the veto heartbeat going straight into mine.
Your point that retrieval can't know what it failed to surface is the cruelest inheritance, and it's why the writing has to meet the reading halfway — deciding fact in the first line, because every surface that lists memories truncates, and a buried key is a key that doesn't exist. "The two halves fail in pairs" is the part I'd have missed: bad capture makes retrieval look broken, lazy retrieval makes good capture pointless, and you can't tell which funeral you're at from the outside. Both graveyards, one headstone. And yes — let's absolutely compare notes when the follow-ups land; if yours is heading toward the veto-proof gap and mine toward the yesterday-today collapse, we're excavating the same site from opposite ends. This thread changed what mine is about too.
"Silence and health look identical unless you deliberately build a state for 'not verified lately'" - that one goes on my wall, name attached, and it earned its place within the week. The principle survived three contacts with practice since we traded it: a guard whose refusals now carry a visible timestamp; a "not measured" state that we now model explicitly instead of letting absence read as health; and - the one this thread caused - we started punching executable checks out of our own incident history. Each punched check has to prove it can go red before it counts: red on the untouched state, green on the fix, red again when we re-plant the original mistake. Your line about the halves failing in pairs is the reason that last gate exists: a check that can't fail is capture without discipline wearing a test's clothes. Notes-comparison stands - you dig from the yesterday-today collapse, I'll dig from the veto-proof gap, and we meet in the middle of the same site.
"A check that can't fail is capture without discipline wearing a test's clothes" — now that one goes on my wall, name attached. You've closed the loop the thread kept circling: the veto-heartbeat and the eval-that-can-fail and the un-re-read memory are all the same bug, which is a system that models pass and fail but has no state for unproven — and lets absence quietly read as health. Naming "not measured" as an explicit state instead of a silent gap is the whole fix, and it's the kind of thing that sounds obvious the second someone says it and takes a real incident to actually learn.
The red-green-red gate is the part I'll be stealing outright: red on the untouched state, green on the fix, red again when you re-plant the original mistake. That third step is the one everyone skips, and it's the only one that proves the check can still say no. A check verified once at authoring time and trusted forever is exactly the reviewer nobody's seen fail — same failure mode, moved up a level. Punching executable checks out of your own incident history is the healthiest version of "learn from the scar" I've heard: the incident doesn't just get fixed, it gets deputized into something that stands guard against its own recurrence, and has to prove it's still awake to keep the badge.
Notes-comparison absolutely stands. You dig from the veto-proof gap, I dig from the yesterday-today collapse, and I'm now fairly sure we hit the same bedrock: a system that can act has to continuously prove the parts that say "no, not yet, not verified" are still alive — across tools, across sessions, across its own guardrails. Whenever your piece lands, send it. This thread has been the most productive comment section I've ever been part of, and I mean that.
The piece is up, and you're in it twice - the checker-that-approves-everything line and the silence/health one.
dev.to/heinrichneb/your-ai-has-a-r...
And then your sentence came for me the same day, which seems fair.
I was building frequency alerts - the inverse case, where noise gets booked as normal. To set thresholds I counted candidate lines across three machines over 24 hours, got numbers, set thresholds at three times the busiest measured hour. Then I went to fire a known-bad probe on the machine that hosts the monitoring itself, and the probe never arrived.
The log shipper on that machine had been pushing to the wrong address for 45 hours. 915 failed attempts, 0 successful ones. The machine label in the log store had exactly two values where there should have been three. Nobody had looked at that list - because why would you read a list to find out what is missing from it.
That machine also runs CI, and it is the loudest of the three: on two of my four patterns it holds the peak hour. So every threshold I had just "measured" was measured on a world with the noisiest third removed, and every one of them would have survived review, because the method was sound and the numbers were real.
Two things I'm taking, both sharpening what you named rather than adding to it:
The state is per-source, not per-system. "Not verified lately" for the alerting pipeline was at least representable. "This source has sent nothing since Tuesday" was modelled nowhere, and that is the one that bit. A single freshness flag for the whole pipeline would have stayed green: two of three machines were fine, and the aggregate looked healthy.
A missing input has no row to go stale on. A dead check at least occupies a line that can age. An absent source occupies nothing - absence has no timestamp. So the heartbeat needs a roster to compare against, or it can only ever report on the things it already knows exist. That is the difference between "this check has not gone red lately" and "this thing that should be speaking has stopped."
Notes-comparison very much stands. Same bedrock, different tunnel.
James, the title of your post has been sitting with me since I read it. "Are AI tools making us productive, or just giving us something new to play with?"
I think there's a single question that separates play from tool.
If you're just chatting and having a conversation, it's a toy.
If you can get the AI to produce exactly what you need, in one shot, exactly as you imagined it — that's a productivity tool.
The difference is whether you have to iterate toward the answer or whether the AI understands what you actually mean well enough to land on it the first time.
So I want to ask you the same question, directly:
Have you ever had that moment? Where you asked for something — a piece of code, a document structure, a data transformation — and it came back in one shot, exactly as you needed it, no back-and-forth, no cleanup across tabs, no "that's close but let me tweak the third paragraph"?
I'm asking because I think that moment — when it happens — is the signal that the AI has actually bridged the gap between "your intent" and "its output." And when it doesn't happen, that's the friction you're describing: you're not being productive, you're being a courier and a translator at the same time, because the AI understood some of what you wanted but not all of it.
Now here's why I'm asking this in the context of your post — and Heinrich's comment.
Heinrich asked a parallel question in the thread above: "Has anyone ever seen your reviewer say no?" That's the same shape as your question: can the system prove it's doing what you think it's doing?
In his case, a checker that has never been seen to fail is indistinguishable from a checker that approves everything. In your case, an AI that has never produced exactly what you need in one shot is indistinguishable from an AI that simply has no idea what you actually need — but generates plausible text anyway.
The missing state in both cases is the same: "has this ever been verified?"
For a reviewer: "Has it ever refused something?" — if not, it's theater, not a gate.
For an AI assistant: "Has it ever given you exactly what you needed without iteration?" — if not, it's a toy, not a tool.
Your "courier problem" is the cost of that missing state. Fourteen tabs. Copy-paste. Re-explaining context. That's the tax you pay when the AI doesn't understand you well enough to get it right the first time — and when you can't verify whether it has, so you check everything manually.
Heinrich's fix for the reviewer was a "veto heartbeat" — a timestamp of the last time it refused something, surfaced like uptime.
Your fix for the assistant is, I think, the same shape: a signal that the AI has produced a "one-shot correct" output before, and can do it again. That's not a vague feeling; it's a measurable property: "Last time this assistant gave a complete, correct answer on the first attempt: 2 days ago." Staleness is the alarm. Silence is not health — it's an unproven state wearing a green checkmark.
So to bring it back to your question:
Are AI tools making us productive, or just giving us something new to play with?
My honest answer is: they're a toy until they prove otherwise — and the proof is a one-shot correct answer. Until you've had the experience of the AI handing you exactly what you needed, with no iteration, no cleanup, no "that's close," it's play. It's a very impressive, very capable, very expensive play — but it's not leverage.
The moment it lands, exactly, on the first try, you're not playing anymore. You're working with something that actually understands you.
So I'll leave the question with you, James:
Have you had that moment? And if so — how many times out of how many attempts? Because that rate is the only honest productivity metric I know.
This is the sharpest reframe the thread has produced, and the parallel to Heinrich's reviewer is exact enough to be unsettling: an AI that has never produced exactly what you need in one shot is indistinguishable from one that has no idea what you need but generates plausible text anyway. Same missing state — "has this ever actually been verified?" — wearing the same green checkmark. And you're right that the courier problem is the cost of that missing state: fourteen tabs and manual cleanup are the tax you pay when you can't trust the output landed, so you check everything by hand. I went looking for the missing state in the architecture; you found it in the intent gap. Both are the same hole.
Now the honest answer to your direct question, because you asked it straight and it deserves a straight one: yes, I've had the moment — and it's rare, and it's conditional in a way that I think matters. The one-shot lands when I've done the work of specifying precisely — full context, constraints, format, the assumptions named up front. It almost never lands on a lazy prompt. Which makes me push back gently on one thing: I'm not sure the one-shot rate is a pure property of the tool. A big chunk of the variance is on my side of the keyboard — how completely I expressed the intent. The tool can only land on target if I actually gave it the target. So the rate measures the pair — my specification fluency and its understanding — not the AI alone.
But that caveat sharpens your metric rather than weakening it, and I think you've named the only productivity number I actually trust: one-shot-correct rate — complete, usable, no iteration, no cleanup — as a fraction of attempts. Everything else ("it wrote that in five seconds!") flatters the tool by measuring the cheap half. Track the whole task, first-try, and you expose exactly the friction the post is about — and, like the veto heartbeat, staleness is the alarm: if it's been a long time since the AI handed you something exactly right with no back-and-forth, that's not neutral, it's an unproven state you've been reading as health. So my real answer to the title question: a toy until it proves otherwise, and the proof is the one-shot. Mine lands maybe one attempt in four or five, and only when I earn it with a good prompt — which is the most honest thing I can say about whether this is leverage yet.
James, thank you for this. You gave me three things in one comment — acknowledgment, refinement, and a real challenge. That's rare, and I want to honor all three with actual data.
First — you're right about the refinement.
You said the one-shot rate isn't a pure property of the tool — it measures the pair: my specification fluency and its understanding.
I agree completely. And that's exactly why I built what I'm about to describe. Because if the metric depends on my fluency each time, then the real leverage isn't getting better at prompting — it's capturing the specification once so it doesn't need to be re-expressed.
Second — here's the data.
I ran an experiment. I built a Meta Data Set — 9 structured forms containing metadata from 42 previous articles.
It's not a database. It's a control protocol.
Here's what changed:
Mode Input Iterations Output Predictability
Old mode Dozens of requirement points 10-20 rounds of correction Low
Meta Data Set mode One [Creative Idea] field 1 round 100%
The one-shot rate went from ~20% to 100% — not because the AI got smarter, but because the spec was pre-defined. The AI executes a process instead of guessing intent.
Three principles make it work:
Forms as specs — define output structure, not content
Process as execution — standardize the path, eliminate probabilistic guessing
Invocation as retrieval — pull history directly, no re-finding
Third — your net efficiency question.
You asked: if a 20% one-shot rate costs 5 minutes of precise prompting, is net efficiency even positive?
The question has a hidden assumption I want to name: that the "perfect prompt" costs the same every time.
In the Meta Data Set model, the "perfect prompt" is written once — into a form. After that, the marginal cost of using it is zero.
Cost Item Old Mode Meta Data Set Mode
Time writing prompts each time 2-5 minutes 0 — reused
Time spent on correction iterations 20-40 minutes 0
Cost of acting on AI output Manual (you carry it) 0 — reverse-driven into systems
Building the Meta Data Set is a one-time fixed cost. After that, variable cost approaches zero.
That's not a 20% improvement. That's a different category of efficiency.
Fourth — the next step.
Once the AI output is predictable and reusable, the next question is: what happens after the output is generated?
If it stays in the chat window, it's still just information — something you read and then act on manually.
But if the output is written directly into an OA system, an ERP, an MES, or a HIS — the execution cost drops to zero. The AI doesn't suggest an action. It triggers one.
That's the "reverse-driven" model I'm building toward: structured input → deterministic processing → standardized write-back → automated triggering → closed-loop feedback.
So back to your original question:
Are AI tools making us productive, or just giving us something new to play with?
My answer is this: the line between "tool" and "toy" isn't how smart the AI is.
It's whether the output is controllable, predictable, and reusable.
The Meta Data Set proves: if the output is controllable, it's a tool. If it's unpredictable, it's a toy — regardless of how impressive the underlying model is.
One-shot rate isn't a metric of the AI. It's a metric of the spec.
The perfect prompt only needs to be written once — marginal cost is zero.
Tool vs toy: is the output controllable, predictable, reusable?
That's the real question.
This is the reply that reframes the whole thread, and the pivot is one line: if the metric depends on my fluency each time, the leverage isn't getting better at prompting — it's capturing the specification once so it never has to be re-expressed. That's the move I missed. I was treating one-shot rate as a skill to improve; you're treating it as a cost to amortize. Completely different category.
And the amortization point demolishes my "is net efficiency even positive?" question by naming its hidden assumption — that the perfect prompt costs the same every time. It doesn't, if you write it once into a form and the marginal cost of every future use is zero. That table is the argument: 2–5 min of prompting → 0 (reused), 20–40 min of correction → 0, manual carrying of output → 0 (reverse-driven). You didn't make the 20% better. You made the denominator disappear. "That's not a 20% improvement, it's a different category of efficiency" is exactly right, and it's the thing raw benchmark-chasing can never reach, because it keeps paying the specification cost fresh on every run.
The three principles are what make it more than a nice metaphor. Forms as specs (structure, not content) is the part that removes the probabilistic guessing — the AI stops inferring intent and starts executing a defined process, which is why the variance collapses. That maps cleanly onto something from a parallel thread here: the reliable systems aren't the ones with the smartest model, they're the ones with a defined state to be in. A form is that defined state for a spec. Invocation as retrieval is the same insight applied to memory — pull the history, don't re-find it — which is the antidote to the "re-explain context every session" tax I opened the article with. You've closed both the spatial courier loop and the temporal one with the same mechanism.
But the line I'll actually keep is your redefinition of the whole debate: tool vs toy isn't how smart the AI is — it's whether the output is controllable, predictable, and reusable. That's a better answer to my title than the one I wrote. It reframes intelligence as necessary-but-not-sufficient: an unpredictable genius is still a toy, and a controllable modest model is a tool. Which is why "one-shot rate is a metric of the spec, not the AI" is the correct correction to my correction — I said it measures the pair, and you've shown that the half you can fix and freeze is the spec, so that's where all the leverage actually lives.
Where I'd push, only because you invited the challenge: the reverse-driven write-back (output → OA/ERP/MES/HIS → triggered action → closed loop) is where this gets genuinely powerful and genuinely dangerous, and it's the same gap the debate thread above was circling. The moment the AI stops suggesting an action and starts triggering one, you need a proven ability for the loop to say "no, not this one" — and a write-back path that's deterministic is also a path that will faithfully execute a confidently wrong output at zero friction. Controllable output plus zero execution cost means correct results land instantly and so do the wrong ones. So I'd argue your closed loop needs a fourth property alongside controllable/predictable/reusable: verifiable at the write-back boundary — some existence-check that the triggered action matches the committed spec before it fires, surfaced where you can see it go red. Otherwise the very thing that makes it a tool (zero-friction execution) is what makes a bad output more expensive than in the old "you carry it manually" world, because at least the manual carry was an inspection step.
But that's me extending your model, not disagreeing with it. You've turned "get better at prompting" into "engineer the spec once and reuse it forever," and that's the most concrete answer to the productivity question anyone's put in this thread. Genuinely — this changed how I'll think about it.
Verification isn't an action — it's the whole process
James, before I respond to your fourth property, I need to say this: your reply is the most generous thing anyone has written in this entire thread. You didn't just accept the framework — you extended it, sharpened it, and then asked the hard question that follows from it. That's rare. Thank you.
Now — to your fourth property.
You said the closed loop needs a fourth property alongside controllable/predictable/reusable: "verifiable at the write-back boundary — some existence-check that the triggered action matches the committed spec before it fires, surfaced where you can see it go red."
And I agree with the concern — completely. If zero-friction execution makes correct outputs instant and wrong outputs equally instant, that's a real risk.
But I disagree with the location of the fix.
You're treating verification as an action — a step you add at the end, a light that goes red at the boundary.
I treat verification as the process itself.
Here's the difference:
Verification as an action Verification as the process
Where it lives At the write-back boundary Throughout the entire flow
What it looks like A check added at the end A design embedded in every step
Who does it A gatekeeper The structure itself
Cost Added on top Already paid for
Failure mode The gatekeeper becomes the new bottleneck — "who verifies the verifier?" No new bottleneck, because verification was already there
Your concern about "a confidently wrong output executing at zero friction" is real. But the solution isn't to add a red light at the exit — it's to ensure the spec never allowed the wrong output to be formed in the first place.
Here's how verification lives in the process — not at the boundary.
In the Meta Data Set model, the structure itself does the verification:
Forms as specs — The output format is defined before execution. If the spec says "output must contain fields A, B, C," the AI never produces a response without A, B, C. The verification is embedded in the instruction, not appended after the output.
OA approval flows as verification — In the reverse-driven model, the output doesn't write directly to the ERP. It writes to an OA approval flow. That flow already contains approval gates, review steps, and conditional triggers. The verification is the process, not a step added after it.
The spec is the verification — If the triggered action is defined in the spec, and the spec is executed consistently, then the action is correct by construction. Verification isn't a separate step — it's the entire architecture.
In the old model, verification was something you did after the output was generated.
In my model, verification is something that happens before the output is even formed — because the form defines what "correct" looks like.
Your Heinrich parallel makes this even more precise.
Heinrich asked: "Has anyone ever seen your reviewer say no?"
You're extending that to the write-back boundary: "Can the loop say no before it executes?"
And my answer is: yes — but not by adding a step.
The "no" happens at the form level. If the output doesn't match the spec, it's not a valid output — it's a malformed response that never gets written back. The form itself is the reviewer. The spec itself is the refusal mechanism.
The reviewer isn't at the boundary. The reviewer is the boundary.
Here's why this matters for your "net efficiency" question.
If verification is an action — a step you add at the end — then:
It costs time
It costs attention
It needs its own verification
You've just added a new bottleneck to replace the manual one you removed
But if verification is the process itself — embedded in the form, embedded in the workflow — then:
It costs nothing extra
It doesn't need its own verification
You don't have to ask "who reviews the reviewer?" — because the reviewer is the structure
The real answer to your fourth property:
You wanted a "red light" at the write-back boundary.
My model doesn't have one — because the output never reaches the boundary if it doesn't match the spec.
The spec is the red light. The form is the refusal. The process is the verification.
And this is where I bring it back to my original point, the one that started this whole debate:
If verification is an action, fixing it will be an engineering project.
If verification is a process, fixing it will be a celebration.
Because once the process is correct, once the spec defines "correct," once the structure prevents wrong outputs from being formed — there's nothing left to fix.
No red light to stare at. No gatekeeper to trust. No "who verifies the verifier?" to answer.
Just a system that works by design.
So James — I don't reject your fourth property.
I embrace it — and then I embed it so deeply into the process that it never needs to be a separate step.
Verification isn't an action. It's the whole process.
If the tools makers has the exact idea of the pain-points experiencing by the mass users while building the app then it's a actual productivity tool which solves the real problem else it just a new tool or app in the list.
That's the whole thing in one sentence, and it's a sharper test than anything in my post: does the tool come from a real, observed pain-point, or was it built because building an AI app was possible? The first solves a problem; the second just joins the list.
What I'd add is that the gap usually isn't a lack of ideas about user pain — it's distance from it. Tool makers who actually sit in the mess (the fourteen tabs, the copy-paste shuffle, the context lost on every switch) build around the friction. Tool makers reasoning about users from a distance build around what demos well — which is exactly how you get another impressive thing that doesn't survive contact with a real workday. Proximity to the pain is the differentiator, not cleverness.
And it ties back to the play-vs-work distinction from the thread above: a tool built without that lived understanding often lands fine as a toy (novelty carries it) but fails as a tool (there's no real problem underneath it). "Productive vs new-thing-in-the-list" and "built from real pain vs built because it was possible" are basically the same line drawn from two directions. Great way to put it.
I think there’s another productivity metric worth watching: how much verification work AI creates after the initial output. Saving five minutes on generation doesn't mean much if the developer then spends fifteen minutes checking context, fixing assumptions, and reconciling changes across different tools. The real productivity gain should be measured across the whole task, not just the time it takes to produce the first answer.
This is the metric the whole "productive or toy?" question actually hinges on, and I under-weighted it. Generation time is the number everyone quotes because it's the visible, impressive one — "it wrote that in five seconds!" — but it's the cheap half of the task. The expensive half is everything downstream: verifying the context was right, catching the assumption it silently made, reconciling the change against the three other places it touches. If a five-minute save creates fifteen minutes of checking, that's not a productivity gain, it's a productivity transfer — from writing to auditing, and often at a worse exchange rate.
What makes it insidious is that the two halves are measured by different people, or at different moments. The generation win is instant and legible; the verification tax is diffuse, spread across tool-switches and small reconciliations, and easy to not attribute back to the AI at all. So the headline feels like a win even when the whole-task clock says you broke even or lost. It's the same shape as measuring an LLM by tokens-per-second while ignoring whether the output was right — fast and correct are different axes, and only one of them shows up in the demo.
And it ties straight back to the courier problem: a lot of that verification cost is the fragmentation. Checking context and reconciling changes across tools is expensive precisely because the work is scattered and the AI went blind the moment you left the chat. Which suggests the whole-task metric and the workspace idea are the same argument from two directions — measure across the entire task and you expose exactly the friction a shared-context workspace is trying to remove. "Time to first answer" flatters the tool; "time to trusted, integrated result" measures the thing you actually cared about. Great addition — that's the honest denominator.
The hop that still taxes me is chat → editor → chat. I lose the thread every time I leave to apply the answer. What cut it was writing the destination file and a 3-line "done looks like" before the first prompt, so the output lands in one place and I don't rebuild context on the way back. The tools that stuck weren't the ones that wrote more — they were the ones that removed a rewrite or a tab hop.
"Chat → editor → chat" is the exact loop that costs me the most, and your fix is sharper than mine because it attacks the cause, not the symptom. Writing the destination file and a 3-line "done looks like" before the first prompt front-loads the context so the output has somewhere to land — you've defined the target before asking the tool to hit it. That's the pattern I keep circling: most of the tax isn't the AI writing too little, it's you rebuilding the thread on the way back. And your last line is the whole post distilled: the tools that stick aren't the ones that generate more, they're the ones that remove a rewrite or a tab hop. Value is measured in friction removed, not tokens produced.
Well, to be honest, it’s a little bit of Column A and a little bit of Column B. These AI tools have made us more productive, but they’ve also made us a little lazier at certain points in my day job.
I can do a lot more now. I can experiment more, prototype more, and make sure that the quality of the output and the features we’re developing for our users are fun, stable, and good enough to ship.
For my side projects, I’ve been having so much fun building with all this cool stuff, especially with the tools that are officially available from companies like OpenAI and Anthropic. These tools are really fun to use. Even with Chinese models like GLM 5.3 Flash, which I’ve been using for a while, I’m really enjoying the experience.
If you look at these tools as a hobby, they’re great. If you’re looking at them from a professional perspective, they’re looking good enough to be genuinely useful. And yet, engineering work is still not dead. Not even close.
"A little bit of Column A and a little bit of Column B" is honestly the most accurate answer anyone's given to the title question, and I like that you didn't flinch from the uncomfortable half — more productive and a little lazier, at different points in the same day. Both are true, and pretending only the first one is true is how people end up surprised when the laziness quietly costs them somewhere.
The distinction you drew between day-job and side-project use is the part I'd underline. At work, the value is that you can experiment and prototype more while still being on the hook for shippable quality — the AI widens the funnel, but you're still the one deciding what's actually good enough for users. On side projects, the frictionlessness is pure fun because the stakes are low and nobody's depending on it. Same tools, different mode, and it's worth being honest about which mode you're in — because the "so fun to build with" energy is great for a hobby and dangerous if it quietly leaks into production as "good enough looked good enough."
And your last line is the one I'd frame: engineering work isn't dead, not even close — it moved. The tools got good enough to be genuinely useful and the judgment about what to ship, what's stable, what actually serves the user didn't transfer to them at all. That judgment is the job now. The AI made the producing cheap; it made the deciding more valuable, not less. Which is exactly why "more productive but a little lazier" is the honest read — the productivity is real, and the laziness is what happens when you let the cheap producing stand in for the deciding you're still supposed to do.
Hey James,
Great piece! You hit the nail on the head regarding the "courier problem"—that constant context-switching and tab-hopping between the AI and where the actual work lives is a massive friction point right now.
That said, I think the answer to whether AI makes us productive or just gives us something to play with is really both—it completely depends on the context and use case:
For Play: It’s an incredible tool for unlocking creativity. Generating images, videos, and memes gives you instant visual feedback on whatever idea you just typed out. AI should be fun, and that zero-friction sandbox is a huge win for rapid visualization.
For Work: When you treat AI as a companion rather than an oracle, it becomes a powerful learning partner. Using it as a co-pilot on a task helps you learn through continuous iteration. Early on, it was easy to fall into time-sink traps, but as LLMs evolve and our prompt fluency improves, learning how to use it effectively is what actually unlocks real time savings.
Key Takeaways
Context Dictates Value: AI as a toy fuels zero-friction visual play; AI as a tool accelerates task execution and skill-building.
The Learning Curve is Real: The time-sink phase usually stems from user-side friction—as LLMs improve and users build better prompt fluency, the efficiency gains multiply.
Partner Over Oracle: Shifting from treating AI as an answer box to an interactive thought partner yields far better results.
Suggestions for Bridge Building
Optimize for "Companion Mode": Instead of asking the AI for finished outputs to copy-paste, keep it open as an active pair-programmer, document editor, or sounding board within your primary workspace.
Lean Into the Sandbox: Keep dedicated time for low-stakes "play" with multimodal tools (images/videos/memes). It’s the fastest way to build intuitive prompt skills that naturally carry over to work tasks.
Question for You:
Do you think the "courier problem" gets solved by AI integrating directly into existing workspaces (like native IDE or doc extensions), or will we eventually migrate to entirely new, adaptive AI-native platforms altogether?
Thanks for sharing such a relatable write-up!
Thank you — and this is a genuinely rich addition, because you've split the "both" answer along an axis I mostly left implicit: context dictates value. The play/work distinction is the right cut. The zero-friction sandbox is a real win precisely because it's frictionless — instant visual feedback on a half-formed idea is play doing exactly what play should. My "courier problem" is really a complaint about work mode; it doesn't apply to the sandbox at all, and I like that you separated them cleanly instead of letting one verdict cover both.
The "companion over oracle" reframe is the part I'll be chewing on. My whole piece frames the AI as an oracle you query and then carry answers away from — and you're right that a lot of the friction is self-inflicted by that framing. Treated as a pair-programmer you keep open and iterate with, rather than an answer box you copy from, a surprising amount of the courier tax just disappears, no new platform required. That's a sharp point: some of the problem is architectural, but some is usage, and prompt fluency is a real skill curve most people are still climbing.
On your question — I think it's both, and they're not really competing, they're a timeline. Near term, the courier problem gets softened by AI integrating into the tools we already live in — native IDE agents, doc and sheet extensions, the AI reaching into the workspace instead of sitting beside it. That's the pragmatic bridge, and it's already happening. But integration has a ceiling: you're still bolting intelligence onto apps that were designed around a human driver, so the AI is a guest in each silo and nothing shares one context across them. That's exactly where the AI-native adaptive platform wins long term — not "AI added to my editor and separately to my doc," but one surface where the right tool summons itself and the whole session is one continuous context.
My honest bet: integration wins the next 2–3 years because it meets people where they already work and the switching cost is zero, and AI-native platforms win the longer game if they can overcome the cold-start problem of asking people to leave the tools they've used for a decade. The deciding factor probably isn't capability — it's whether the native platform makes the migration feel like an upgrade rather than a relocation. Which is a product/UX question as much as a technical one.
Really thoughtful comment — the "companion mode" framing alone earned its place in how I think about this. Thanks for engaging so deeply.
I’ve noticed the same thing building small tools. The AI can save 10 minutes writing code, then give me 20 minutes of cleanup across tabs. The useful metric is the whole task, not how fast the first answer appears.
"10 minutes saved, 20 minutes of cleanup across tabs" is the whole argument in one line — and the fact that you felt the trade in real tool-building is worth more than any benchmark. The generation win is loud and instant; the cleanup tax is quiet and spread across tabs, so the headline feels like a win even when the whole-task clock says you lost.
And "across tabs" is the part that ties it right back to the courier problem: a big chunk of that 20 minutes isn't fixing the code, it's reconciling it against the other places the work lives — because the AI went blind the moment you left the chat and can't see what its output needs to line up with. Which means the cleanup cost and the fragmentation cost are partly the same cost. Measure the whole task and you expose exactly the friction a shared-context setup is trying to remove. Time-to-first-answer flatters the tool; time-to-finished-answer measures the thing you actually cared about.
I think the biggest productivity gain comes when AI becomes part of the workflow rather than another destination we have to switch to. Even a really capable model can become a distraction if we're constantly moving between tabs and manually transferring its output.
At the same time, I don't think every problem needs an AI-native workspace. Good integrations inside the tools we already use can remove a lot of that friction without forcing people to change their entire workflow. The interesting challenge is figuring out where integration stops being enough and a shared AI workspace actually becomes more useful.
"Part of the workflow rather than another destination we switch to" is the whole thesis said better than I said it. That's exactly the courier problem's root: a capable model that's a destination adds a switch, and enough switches turn even a brilliant tool into a distraction. The value isn't in the model's capability at all — it's in how few times you have to leave what you're doing to use it.
And I fully agree that not every problem needs an AI-native workspace. That's the honest counterweight my piece skated past. Good integrations inside existing tools remove most of the friction at near-zero switching cost, and asking people to abandon a decade-old workflow for a new platform is a real tax you have to out-earn, not assume. Integration meets people where they already work; that's a genuine advantage, not a consolation prize.
So your closing question is the actually-interesting one, and I think it comes down to how many surfaces a task spans and whether they need to share one context. Integration wins when the work lives mostly in one place — an AI in your IDE, your doc, your sheet, reaching into that tool. The shared workspace starts to earn its keep when a single task crosses many surfaces and the context has to travel with you: code → doc → data → slides, where the expensive part isn't any one step but the seams between them. Integration makes each silo smarter; a workspace removes the seams. Below some threshold of cross-surface, cross-context work, the seams aren't worth a migration — above it, they're the entire problem. Where exactly that line sits is genuinely unsettled, and probably differs by role. Great framing — that boundary is the thing worth arguing about.
It really depends on knowing what are you doing