There is a very specific kind of frustration that comes from staring at a loading spinner.
You have submitted something. The page is doing something. You are not sure what. You do not know if it is 10% done or 90% done. You do not know if it crashed. You are just sitting there, watching a circle spin, wondering if you should refresh, knowing that if you do, you will probably have to start over.
That was the experience on the GoGroup Partner Portal every time someone used the Initial Estimates feature. A partner would drop in a rough project brief and get back a structured software estimate: features, timelines, ballpark costs. The feature worked. It was just invisible while it worked. This is how we fixed the spinner.
What the old flow actually looked like
The Initial Estimates feature lets a technology partner submit a rough project description and receive a structured software estimate. The original pipeline had six sequential steps:
- Partner submits requirements
- Backend packages the request
- Claude AI generates estimates (10-20 seconds)
- JSON is converted to HTML and PDF
- PDF is saved to cloud storage
- Link is sent to the partner.
Every step had to be completed before the partner saw anything.
The Old Way: Claude could be 80% done generating the estimates, and the user would still be looking at a blank screen. The data existed. The progress was real. None of it was visible.
The GoGroup Way: Show content as Claude generates it. Every piece of useful output, the moment it is ready.
This is called streaming. Claude’s API supports it natively. Rather than receiving one large response at the end, the client gets a trickle of small pieces in real time, like reading a message as someone types it rather than after they hit send. For regular text, streaming is straightforward. For structured data, it is a different problem entirely.
The problem nobody tells you about: streaming JSON
The GoGroup estimates come back as JSON. JSON is only valid when complete. A half-open structure is not partial JSON. It is invalid text.
When Claude streams, it sends the response in fragments. Mid-stream, the client might receive:
{ “header”: { “project_name”: “Partner Por…
That is not JSON. That is half a word inside an incomplete structure. It cannot be parsed, converted, or displayed. You cannot do anything useful with it yet.
The engineering challenge: how do you make something useful out of incomplete structured data, in real time, before it is finished?
Two approaches, one problem
GoGroup built two streaming versions.
Version 1
Section-by-section delivery. The server watches for completion of whole sections: the project header, the features list, the timeline, the summary. The moment a full section arrives and validates as complete JSON, it converts to HTML and pushes to the browser immediately. The partner sees the header appear first, then the features, then the summary. Each section checks in when it is ready.
Version 2
Item-by-item delivery. Instead of waiting for a whole section, the server sends each individual item the moment it becomes usable. Rather than waiting for all ten features before showing any, the first feature appears as soon as Claude finishes generating it. Then the second. Then the third.
When a fragment arrives that is not yet complete JSON, the server artificially completes it, closing unclosed brackets, filling in missing structure, then running it through a validator. If it parses correctly, there is something displayable. If not, the server waits for the next fragment and tries again. It sounds like a hack. It works precisely because JSON structure is predictable enough that missing pieces can be inferred from what has arrived.
What the partner experiences now
Before: Submit requirements, stare at a spinner for 15-30 seconds, PDF appears.
After: Submit requirements, project header appears in roughly 3 seconds, features start building out one by one, summary loads, complete in roughly 15 seconds.
Partners stopped refreshing the page thinking it had crashed.
The total processing time did not change dramatically. The experience changed completely. Partners started commenting that the tool felt fast, even though the underlying computation was doing roughly the same amount of work.
This is one of those engineering lessons that sounds like a UX principle until you see it in the data: perceived speed matters as much as actual speed. When something is visibly building, people trust it. When something is invisible, people assume it broke.
What it cost
GoGroup tracks Claude compute budget consumption per task. The Initial Estimates streaming upgrade, covering both versions, real-time JSON reconstruction, frontend integration, and test coverage, consumed 26% of a five-hour Claude session budget.
That is a meaningful number. It reflects real complexity: two streaming architectures, ten-plus JSON edge cases, a live production codebase. But it shipped, with two variants, with tests.
26% of a 5-hour Claude session budget. 2 streaming versions built. 10+ JSON edge cases handled. ~3 seconds to first visible content (was 30 seconds).
The relevant metric is not tokens consumed in isolation. It is the ratio of delivered functionality to session budget consumed. On this feature, that ratio was strong.
Three lessons that transfer
Design your data structures with streaming in mind before you write any code. Flat, simple structures are dramatically easier to stream piece by piece than deeply nested ones. If the schema makes streaming difficult, the schema is the problem. Version 2 became feasible partly because the estimates structure was shallow enough that partial completion was predictable.
Build the simpler version first. Version 1 alone is a significant improvement over a 30-second spinner. Version 2 is a refinement on top of that. Starting directly with the item-by-item approach would have meant more risk, more debugging, and more time before anything was in front of users. The right sequence was Version 1 to production, then Version 2 as an iteration.
Streaming changes how users feel about software, not just how fast it is. There is a qualitative difference between “the app returned a result” and “the app is working with me.” The spinner communicates the former. Streaming communicates the latter. For any feature that involves waiting on AI generation, the question is not just whether streaming is technically feasible. It is whether the experience of waiting changes what users believe about the product.
Frequently asked questions
AI streaming sends model output to the client as it is generated rather than waiting for the complete response. For structured data like JSON, it requires additional logic to handle incomplete fragments. The UX impact is significant: users see progress immediately rather than staring at a blank screen, which reduces abandonment and increases perceived speed even when total processing time is unchanged.
JSON is only valid when complete, so streaming requires two approaches. The first buffers until a whole logical section arrives and then sends it. The second attempts to complete partial fragments by closing open brackets and validating the result, sending each item the moment it passes validation.
Version 1 waits for complete sections before sending anything to the browser. More reliable, easier to implement. Version 2 sends each individual item as soon as it becomes a valid JSON fragment, using artificial completion and re-validation on partial data. Version 2 produces a more continuous live-build experience but requires significantly more edge case handling. Build Version 1 first.
The GoGroup Initial Estimates streaming upgrade consumed 26% of a five-hour Claude session budget. The relevant benchmark is not tokens consumed in isolation but the ratio of shipped functionality to session budget consumed.
Streaming is worth implementing whenever AI generation exceeds roughly five seconds and the output is progressively renderable. If the output only makes sense when complete, for example a final calculation that depends on all inputs, streaming intermediate state adds complexity without user benefit.
The bottom line
The spinner was not a technical constraint. It was a choice about when to share information.
Claude’s API streams. The data was always being generated in real time. The question was whether to hold it until it was finished or share it as it arrived. GoGroup built the infrastructure to share it, and partners stopped thinking the tool had crashed.
GoTeams AI Bootcamp covers this class of problem directly: how AI-assisted engineering teams make architectural decisions about where AI fits in a production pipeline, how to measure the cost-to-value ratio of AI-assisted development, and how to build features that use AI generation in ways users can actually feel.
Leave a Reply