Most robotics programmes do not stall because of motors, sensors or vision stacks. They stall because the software that commands those machines still travels down a delivery route designed years ago. This field guide walks through the three symptoms we see most often in warehouse and industrial automation teams, traces each one back to its cause, and describes what experienced DevOps practitioners do about it.
Symptom one: a firmware release that eats a quarter
The first sign is calendar-shaped. An integrator promises a coordination upgrade for the AGV fleet in January, and the change finally reaches the floor in April. When teams map the journey of a single change with value stream mapping, the pattern is almost always identical: the hands-on work takes days, while the change itself spends weeks parked in queues. Waiting for a merge window. Waiting for a test bench. Waiting for a maintenance slot between shifts.
Two of the DORA delivery metrics make this visible. Lead time for changes measures the elapsed span from commit to running in production, and deployment frequency shows how often the team actually ships. Robotics operations groups that improve these numbers rarely do it by typing faster. They shorten the path itself: trunk-based development, so branches stay measured in hours instead of weeks, and a CI/CD pipeline that builds once, then promotes the same artifact from bench to cell to floor. Flow and feedback — the first two of the Three Ways — are doing the heavy lifting here.
Tooling is the least interesting part of the exercise. Whether a team wires up Jenkins, GitHub Actions, GitLab CI or Tekton matters far less than whether every commit travels the same automated road. As the Ops Atlas DevOps reference frames it, the discipline exists to arrange delivery so that builders and the crew running the machines chase one set of goals with shared tooling and open feedback; the pipeline is simply where that cooperation becomes mechanical.
There is also a price for waiting that never appears on a project plan. Take a change’s weekly value, multiply by the idle weeks, then by the number of similar changes in flight: a fleet feature worth 5,000 in weekly margin that waits three weeks burns 15,000 before a single line of it runs, and twelve such delays across a quarter quietly remove 180,000 from the business. Queues feel free because nobody invoices for them.
Symptom two: a rollback nobody has ever practised
The second symptom arrives at 2 a.m. Something in a new build makes the fleet manager misread bay occupancy, and pick rates sag across the shift. Everyone agrees the previous version behaved fine. Nobody can say precisely how to get back to it — the schema has moved on, the installer lives on a laptop that went home, and the engineer who signed the release is mid-flight.
A worked example that circulates in the delivery literature makes the stakes concrete. One team shipped a deployment that the incident record tags d06, and it drove a four percent spike in payment errors. With no rehearsal of the rollback, recovery stretched to roughly 190 minutes. The same failure, practised in advance, closes in about twenty. That is a tenfold difference manufactured entirely by rehearsal.
The fix is a runbook short enough to recite:
- Declare the deployment. Every rollout carries an identifier and a named owner before it starts.
- Roll back when it is safe. If no irreversible migration has executed, restore the previous artifact immediately.
- Contain instead, when needed. Switch the feature off via a flag, or fix forward if toggling is impossible.
- Review without blame. Hold the blameless review within five working days and fold the lessons back into the runbook.
Notice what the runbook omits: an escalation meeting, a hunt for a scapegoat, a decision by committee. Speed in the first ten minutes is purchased with decisions made months earlier.
Symptom three: DevOps islands in one warehouse
The third failure is subtler. Each robot cell has a tidy build pipeline. The fleet dashboard has its own. The warehouse-management integration has a third, and the three never speak. Examined individually, every island looks healthy; examined together, they reproduce the very silos the automation was meant to remove.
A five-stage maturity model describes the climb: siloed, automated islands, integrated, continuous delivery, and finally a learning system. Stuck teams usually sit at stage two and assume stage three arrives with another platform purchase. It does not. It arrives when culture, automation, lean flow, measurement and sharing — the five CALMS dimensions — advance together. Team Topologies contributes the structural piece: because Conway’s law guarantees your software mirrors your communication paths, redrawing who talks to whom buys more integration than any new tool ever will.
It helps to see DevOps as the hub of a family of practices rather than a movement with firm borders. SRE brings reliability targets, DevSecOps shifts security checks left, GitOps keeps infrastructure declared in version control, and PlatformOps, ReleaseOps and their relatives carve out further slices. A robotics team does not need all of them on day one. It needs to know the hub exists, so that when the drone-inspection group asks how to govern a rollout there is a body of practice to borrow from instead of a blank page.
A ten-minute audit for your operation
Run this diagnostic against your own fleet programme:
- Can any engineer ship a change to a test cell on the day they write it?
- Has the rollback path been executed as a drill within the last quarter?
- Do cell, fleet and integration changes share one pipeline standard?
- Does a written, blameless review exist for the most recent incident?
- Does anyone track change failure rate, or only deployment counts?
Every “no” marks a queue, and every queue is where engineering salary quietly waits.
Your first ninety days of DevOps on the floor
The sequence that works is unglamorous:
- See and stabilise. Map the value stream, publish baseline delivery metrics, and relieve the most painful queue first.
- Automate the route. Build once, drill the rollback, and drive feedback on any commit down to roughly ten minutes.
- Govern and spread. Add least-privilege access, replace standing keys with short-lived OIDC credentials, and run a quarterly audit that samples about ten random changes instead of inspecting everything.
One more metric deserves wall space. In 2024 the DORA research set added rework rate — the share of capacity consumed redoing work — alongside the classic four, having redefined recovery in 2023 so it counts only change-caused failures. For a robotics team, rework is the gap between engineers tuning cell behaviour and engineers re-testing releases that should never have left the bench.
AGV fleets, inspection drones and articulated arms all reward the same treatment: a short, automated, rehearsed route from idea to floor. The machines already run on feedback loops. The delivery process should too. For the fleet side of the house, the complete guide to AGV warehouse automation is a solid companion, the robotic KPIs every factory manager needs pairs naturally with the metrics above, and how robotic arms are built is worth a skim before your next cell upgrade.