The wireless connection disappears as an inventory cart approaches a warehouse crossing. A forklift enters from the next aisle, and the cart stops. Its next assignment, still waiting on a server elsewhere in the building, never arrives. A report being uploaded for the weekly fleet review pauses halfway through.
In this fictional warehouse, the outage interrupts some work and leaves other functions running. That difference reflects a design choice made well before the cart entered service: which decisions depend on a connection, and what the machine does when that connection fails.
The cart counts fully visible cartons on single-depth shelving. Shelf labels identify the storage location and expected SKU; any stock hidden from view remains uncounted. A local server assigns work across the fleet, while a cloud service compares operating records from several warehouses. These activities use different information and tolerate different delays. Their placement should follow those requirements.
Start with how long the decision remains useful
A cart approaching a crossing has a limited time to respond to an obstacle. Its protective control path must detect the relevant conditions and bring the machine to a stop within the limits established for that machine and its operating environment. Waiting for a remote interpretation would add a dependency to that response.
This protective function is distinct from the application runtime: the software that manages inventory jobs, invokes models, and evaluates incoming assignments. Protective control may use a separate processor and sensors. An application model recognizing a forklift does not, by itself, establish that the machine can stop safely. The protective design needs its own validation.
For the inventory application, the same attention to time applies at a different pace. A shelf observation remains useful while it still describes the stock at that location. An aisle assignment may remain useful until another cart reserves the work or a loading crew closes the route. Weekly damage analysis can tolerate a much longer delay because it concerns patterns that develop over many shifts.
The relevant deadline includes the entire path to the required outcome. Capturing an image, waiting in a queue, running inference, transmitting a message, and processing the response all consume time. If the result triggers movement, the remaining motion and control delays also matter. Measuring model inference alone leaves much of this path unexamined.
Average response time can conceal the requests that cause trouble. In The Tail at Scale, Jeffrey Dean and Luiz André Barroso describe how occasional slow responses become increasingly consequential in large online services. The warehouse presents a related engineering concern: even a small deployment can receive an answer after the situation has changed. A fast average offers little comfort when the cart needs this particular answer now.
Percentile measurements help characterize that variation, but an observed percentile is not a guaranteed upper bound. The design still needs a deadline for abandoning the request and a defined response when that deadline passes. Moving computation onto the cart removes the network round trip; it leaves local scheduling, memory contention, and thermal behavior to be measured under the complete workload.
Specify what happens while an answer is missing
Suppose the cart loses contact while counting a shelf. Its application could finish that observation and store the result locally. Continuing to another shelf would depend on the authority already granted to it, the validity of its route, and its current operating limits. If that authority expires, the cart needs a defined holding behavior, with its protective functions remaining active.
That behavior might involve stopping where it is or proceeding to a designated waiting location, provided the machine is authorized and able to reach it. The choice belongs in the design. A waiting location cannot help if reaching it requires the very instruction that is missing. The team must also consider whether the cart can remain in that state as the outage continues and its battery declines.
Reconnect behavior deserves equal attention. An assignment sent before the outage may arrive after another cart has completed the work. The application runtime should check the assignment’s identity, validity period, and relevant site state before accepting it. A repeated message should not create a second job, and a delayed cancellation should not silently undo a newer instruction.
A fresh assignment still passes through local checks before execution. The runtime can confirm that the cart is at the expected location and that the requested work is permitted. The protective control path continues to govern motion within its own validated limits. These responsibilities remain distinct even if the implementation places some components on the same hardware.
With this arrangement, the server can become temporarily unavailable without leaving the cart to invent its next action. The acceptable loss may be a few minutes of inventory work. That operational cost should be stated explicitly rather than hidden inside a generic promise of graceful degradation.
Send the information the next decision needs
Sending every camera stream to a server would make the network part of every analysis that depends on those images. It would also consume capacity that the warehouse may need for other equipment. For shelf counting, the cart can often send a result containing the shelf identity, the visible count, and the observation time, with an explicit indication that hidden stock was not counted.
An obstruction report needs different information. The site coordinator may need its position, extent, and age to decide which assignments to change. A short description can help a supervisor understand the event, while an image crop may be necessary to resolve ambiguity. The right amount of data follows from what the recipient must decide and how it can request further evidence.
Privacy adds another reason to make that choice deliberately. Keeping continuous video on the cart reduces where identifiable images travel. Yet a compact event record can still reveal a person’s movements or work pattern. Access controls and retention choices apply to those records too. On-site processing creates options for limiting exposure; it does not settle the site’s data-handling decisions.
Some tasks genuinely benefit from richer inputs on a more capable processor. A fixed camera may send video to a local server because it lacks the resources to perform the analysis itself. The architectural question is whether that transfer preserves the required response time and data controls, and whether its failure has an acceptable, defined consequence.
Use the wider view where it changes the answer
The server has a useful advantage over an individual cart: it can see work already assigned across the warehouse. It can avoid sending several carts to the same shelf and account for an aisle that a loading crew has closed. Keeping that coordination on site can reduce dependence on an external connection, although the site’s own network and server still need failure handling.
| Decision | Information that matters | Suitable placement in this example |
|---|---|---|
| Respond to an obstacle | Current local conditions and validated stopping requirements | The machine’s protective control path |
| Assign the next inventory task | Fleet availability, aisle status, and work reservations | An on-site coordinator, with a defined waiting behavior on the cart |
| Investigate recurring counting errors | Records from many shifts and warehouses | A cloud service whose results can arrive later |
Ambarella’s N1 family includes processors designed for on-premise AI boxes and multi-camera analysis. In the warehouse example, such a platform could support analysis of fixed-camera feeds alongside the site-coordination application. The team would need to measure the combined workload and design its scheduling and failure behavior around those results.
The cloud becomes useful when the question extends beyond one building. Repeated counting errors for a particular package type may become apparent only after comparing observations from several sites. Those findings can guide a model update or a change in the inspection process. Neither task requires the cloud to authorize the cart’s next movement, and a proposed model update still needs evaluation before deployment.
Model size enters the decision after these requirements are understood. A compact model can run remotely when its answer can wait. A more capable model can run on the machine if the workload fits its resources and the application needs the result there. The deployment may use two tiers or three; the decisions determine which are useful.
Test the placement by breaking the connection
The earlier article From Decisions to Silicon argued for evaluating the complete path to an outcome. Placement decisions deserve the same treatment. Run the cart’s full workload, interrupt its network at different stages, and observe whether each function meets its own deadline or enters its specified waiting state.
Restore the connection with old assignments still queued. Introduce duplicated messages and a server restart. Confirm that stored counts retain their original capture times and that rejoining the fleet does not repeat completed work. These tests expose dependencies that a diagram showing three compute tiers can easily conceal.
The useful result is a clear account of what the system can continue doing, what it must postpone, and how it resumes. At the crossing, the cart stops for the forklift while its completed counts remain stored and its next assignment waits. When connectivity returns, the fleet can resume coordinated work without repeating the count or mistaking an old instruction for a current one.