AI engineering company vs dev shop: how to tell them apart

An AI engineering company and a traditional dev shop look alike in the sales call. They diverge the day the pilot has to run on real data without someone watching. The difference is not vocabulary. It is whether the team has already shipped the production engineering a model needs around it, has priced the work accurately enough to commit to it, and will hand you a system your own people can maintain after they leave.

Three tests catch it before you sign. Can they describe, in specific technical terms, how they handle malformed input, retries, evaluation, and rollback on a system they have in production right now? Will they commit to a fixed price per phase before a line of code is written? Will the source, infrastructure, and documentation sit in your accounts the day the engagement ends? A dev shop that put AI in its deck can rarely say all three.

The MIT NANDA review in 2025 placed the share of enterprise generative AI pilots delivering a measurable business return at roughly five percent. Gartner's 2025 forecast is that over forty percent of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. The failure numbers are no longer novel. The pattern underneath them is.

The failures do not cluster where buyers expect. The model is almost never the problem. The engineering around it is. US B2B teams end up in those numbers for a specific reason: they bought AI work from a vendor that is good at software and new to AI, and the production gap did not show up until the pilot was supposed to become a system. By then the budget was spent, the team was tied up supporting a demo that would not scale, and the project was quietly reclassified as a learning. For more detail on the pattern itself, see [why AI pilots fail to reach production](/blog/why-ai-pilots-dont-reach-production).

That is what this guide answers. Not which firm has the best website. Which firm has already shipped production AI, and which one is learning on your project.

Traditional software is deterministic. Given the same input, a well-tested function returns the same output every time, and the test suite can prove it. AI is probabilistic. The same input returns a different output tomorrow because a dependency version changed, because the model vendor updated its routing silently, or because a slightly different phrasing lands in the retrieval set. A system built on determinism breaks on this and never fully recovers.

That is why an AI engineering company spends most of its build time on infrastructure a dev shop barely plans for. Evaluation that runs continuously against production traffic rather than a one-time acceptance test. Retrieval design that still works a quarter later when the knowledge base has doubled. Guardrails that intervene at runtime instead of a policy that lives in a document. Observability that captures prompt, context, model version, output, and decision together, so when a result is wrong you can trace why. Rollback that lets a bad prompt or a bad model configuration get reverted without touching anything else. Cost ceilings that stop a runaway loop from producing a surprise invoice at 3am.

In plain English for a non-engineer: a dev shop ships the demo that works and calls it done. An AI engineering company ships the demo plus the parts that keep it running when nobody is watching, which is the only version that stays usable in production. That difference is roughly where eighty percent of the budget lives in an honest AI build, and it is the part that gets quietly cut when a generalist team scopes the work to hit a price.

The talent profile is different too. An AI engineer has shipped systems where the model is a probabilistic component inside a larger application, not the application itself. They know prompt engineering as a controls problem, retrieval as a data problem, and evaluation as an ongoing engineering problem rather than quality assurance. A good software engineer can learn these; they do not already know them, and the first project is where the learning happens, usually on the client's budget.

The gap shows up in how the engagement is priced. A traditional dev shop usually sells time: hourly rates, time and materials contracts, or a soft fixed price with liberal change orders. For deterministic software this works, because scope is relatively discoverable up front. For AI work, scope is a function of the data and the reliability bar, and both of those take real engineering to pin down. An hourly contract on AI work transfers all that scope uncertainty to the buyer, and the invoice is where the uncertainty surfaces.

An AI engineering company that has shipped this before scopes the work in phases and quotes each phase as a fixed price. The first phase is a short, paid discovery that converts an open problem into a defined one: a specification, an architecture, a data readiness assessment, and a quoted price for each phase that follows. You approve each phase before it starts, and you can stop at any boundary. That is only possible when the team already knows what the hard parts are and how to estimate them. More on the economics in [how much custom AI development actually costs](/blog/how-much-does-custom-ai-development-cost).

Ownership is the second difference. The dev shop model is often repeat engagement: they keep the code on their accounts, run the infrastructure, and the client renews to keep the system alive. An AI engineering company ships the source, the infrastructure as code, and the runbooks into your accounts and documents the operation so your team can take it. One sells access, the other sells a system. For most US B2B buyers the second is the only outcome that makes financial sense past the first year.

Handover is the third. A dev shop handover is usually a password and a repository link. An AI engineering company handover includes the evaluation set, the prompt and model versions in use, the observability dashboards, the on-call playbook for the specific failure modes the system can produce, and training time for the engineers who will own it. That is a different artifact, and asking to see a redacted sample of it in the sales process is the clearest single test of which kind of vendor you are talking to.

A rubric that works regardless of how glossy the deck is. Score each answer: specific and technically grounded, vague, or deflected. A pattern of vague answers across the first five is the signal, not any one of them in isolation.

A list that fits in a single sales call. These predict a stalled project reliably enough that any one of them should pause the shortlist. Several of them in the same conversation is the whole answer.

Not every project needs an AI engineering company, and selling you one when you do not need it is itself a red flag. The honest version of this framework includes the cases where a generalist dev shop is the better fit.

A dev shop is the right call when the AI work is bounded, well-trodden, and does not need to run unattended. Adding a chatbot behind a managed service to an existing marketing site. Wiring up a well-documented API from a model provider for a feature that a human reviews every time. Integrating an off-the-shelf copilot into a workflow where the vendor already solved the hard engineering. In those cases the deterministic application work is most of the project, and a generalist team is as competent as any specialist. Paying for AI-engineering overhead on a job that is really a web build is wasted budget.

An AI engineering company is the right call when the system has to make a probabilistic decision that someone will act on without a human in the loop every time; when the data is yours, messy, and specific to the business; when there is a reliability or compliance bar it has to clear; and when you need to own the result rather than renew access to it. If the project description includes words like production, unattended, audit, compliance, custom data, or scale, the technical risk is in the eighty percent, and a generalist dev shop is not the vendor that will carry that risk well.

The hardest version of the call is a project that genuinely sits between the two. The pattern that works is to split it: a dev shop for the parts that are deterministic application work, and an AI engineering company for the AI system at the center. The seams between the two are a planned engineering problem rather than an accident, and both vendors together usually cost less than one vendor pretending to be both. The build-versus-buy and in-house-versus-outsource version of this choice is covered in [custom AI development vs an in-house team](/blog/custom-ai-development-vs-in-house-team).

That is what Stallwart is built around. If the question is whether your project needs an AI engineering company or a dev shop, the first conversation should be about which parts actually need which. If it is all dev shop work, we will say so and point you at the right kind of vendor. If part of it is a production AI system that has to run on your data unattended, that is the part we would own.