Blog
Why 87% of ML Projects Never
Ship to Production — And the MLOps
Gartner says it. VentureBeat repeats it. Every backend engineer who’s been in the trenches for more than two years has lived it: between 70 and 80 percent of APIs never make it past their first real-world integration. The endpoint works in Postman. The demo to the client is clean. The documentation looks sharp. And then, six months later, the API is timing out under load, third-party partners are complaining about breaking changes, and the codebase is so inconsistent that onboarding a new integration takes three weeks.
I’ve watched this happen at five companies in the last year alone. Each one had a different product, a different industry, and a different reason why “this time would be different.” None of them were different. The pattern is structural, and the fix is structural too. This piece is about what the fix actually looks like — and the API development stack we’ve put in production at STALLYONS TECHNOLOGIES across 30+ shipped API platforms.
The first instinct most leaders have when API projects stall is to blame talent. “We need better backend developers.” “We need to hire someone from AWS.” “We need to bring in a top-tier agency.” None of these are wrong, exactly, but none of them address the actual gap.
The API failure is an engineering discipline gap. Most API projects fail not because the logic is bad — the logic is usually fine — but because nobody scoped the work between “working endpoint in Postman” and “API serving real integrations at scale.” That work includes versioning strategy, rate limiting, authentication layers, schema validation, pagination standards, error handling contracts, OpenAPI documentation, and the unglamorous gateway wiring that ties all of it together. None of that work shows up in a requirements doc. All of it shows up when your payment API starts returning 500 errors under Black Friday load because nobody stress-tested beyond five concurrent requests.
“The endpoint is usually fine. What’s missing is everything around the endpoint — and that’s where 80% of API engineering work actually lives.”
— Dmitri Holsworth, Head of Backend Engineering, STALLYONS TECHNOLOGIES
Across the projects I’ve audited, the same three failure modes show up over and over. If you’re early in your API journey, recognizing these now will save you a year of dead-end work.
A senior backend developer builds the API fast to hit a deadline. The code works, but there’s no versioning strategy, breaking changes ship directly to /api/endpoint, and third-party partners wake up to integrations that stopped working overnight. There’s no changelog, no deprecation policy, no backward compatibility layer. When a new feature requires a schema change — usually a month after launch — it silently breaks every existing consumer. The fix always takes 5x longer than scoped and the trust lost with partners rarely comes back.
The API ships handling 100 requests per second without a hiccup. Engineering celebrates. Six months later, somebody notices partner complaints are spiking. After two weeks of investigation, the team discovers the API has been returning 503 errors during peak hours for the last three months because there’s no rate limiting, no circuit breakers, and no autoscaling — and one noisy consumer has been hammering the endpoints unchecked. By the time anyone realizes, two enterprise integrations have already churned.
The team shipped API keys in plain text responses, left internal endpoints exposed without authentication, and skipped input validation because “it’s just an internal tool for now.” The API works great in development — until a security researcher finds an unprotected endpoint exposing customer PII, or a bad actor brute-forces an unthrottled auth endpoint. Now the legal team is in the room, the API is taken offline, and there’s no hardened, audited, or patched version ready to deploy.
Here’s the architecture pattern we’ve put in production across 30+ engagements. None of these tools are required — you can swap Kong for AWS API Gateway, Swagger for Redoc, PostgreSQL for MongoDB — but every category below is required. Skip any one and your API becomes a liability instead of an asset.
We worked with Sentinel Fintech Partners earlier this year to rebuild their partner-facing API platform. They had zero versioning, no rate limiting, authentication handled differently across every endpoint, and three enterprise partners threatening to churn over reliability issues. Their backend team was talented and frustrated. Their partner success team had stopped promising integration timelines because nothing shipped clean.
We didn’t start with new endpoints. We started with the foundation. In week one we deployed an API gateway via Kong, implemented OAuth 2.0 across all endpoints, enforced schema validation at the gateway layer, and wired up a Datadog observability stack. By week three, uptime was at 99.95% and the three at-risk partners had confirmed they were staying.
Then we iterated. The backend team could finally focus on what they were good at: building product logic. They introduced /v2/ with proper deprecation notices for /v1/, added webhook support with retry logic and delivery guarantees, and shipped a self-serve developer portal with auto-generated OpenAPI docs. Every change was contract-tested and deployed through the CI/CD pipeline. By week 14, new partner onboarding time dropped from three weeks to two days, API error rate fell by 84%, and infrastructure cost per million requests dropped by 57%.
The crucial thing: the backend team did the same work they had been doing for the previous 18 months. The difference was that now their work could ship without breaking every existing integration.
Before you call an API project “done,” run it through this list. If any item is missing, your API is one bad integration away from an enterprise churn:
The backend community has spent a decade getting better at building features. We have not spent that decade getting better at shipping reliable APIs. The gap between a working Postman collection and a production-grade API platform is enormous, and closing it is mostly engineering discipline — not feature work.
If you’re a founder or engineering leader betting on API-first products, here’s the practical recommendation: hire the feature developer second. Hire the backend engineer who understands API design, gateway architecture, and DevOps first. The shape of the team that ships a reliable API platform looks very different from the shape of the team that builds a clean Postman collection, and most companies still hire as if those two things were the same. They’re not.
If you’d like to talk through where your API stack is and where it needs to be, that’s literally the conversation we have on every free strategy call.