Debugging Humidity: Lessons from deploying software in the physical world

Summary

The first time I deployed code to an actual factory floor, I learned that "edge compute" doesn’t live in climate-controlled racks. It lives next to dust, grease, and forklifts. The idealized world of distributed systems I had studied-clean APIs, reliable networks, consistent power-was a comforting fiction.The reality was far more brutal. Power flickers. Networks throttle to near-zero bandwidth because a welder turns on. Devices disappear mid-packet because a forklift driver unplugs them to charge their phone.Every assumption from a decade in cloud engineering melts the moment you connect to something that moves.The Cloud is a Lie (in the Physical World)In the cloud, we build on abstractions designed to create infinite, reliable resources. On the factory floor, you have finite, hostile resources. The mismatch is violent.Idempotency isn't about APIs, it's about Physics. A common pattern in distributed systems is the idempotent request. If a POST request fails, just send it again. The server is designed to handle the duplicate.Now, imagine your request is actuator.rotate(90). The network cuts out after you send it. Did the arm rotate? You have no 200 OK. Do you send it again? If the first one worked, you're about to crash a multi-thousand-dollar robotic arm by rotating it another 90 degrees. Suddenly, you're not debating RESTful principles; you're implementing a complex state reconciliation protocol over an unreliable network just to prevent physical damage."Offline-first" is not a UX pattern; it's survival. On a mobile app, "offline-first" means a user can like a photo without a connection, and it syncs later. That’s nice. On an assembly line, the PLC controlling a conveyor belt must function for its 8-hour shift even if the facility's internet goes down completely. The device cannot be a thin client for the cloud. It needs local autonomy. Computation, state, and decision-making have to live on the metal because the link to the mothership is a privilege, not a right.T...