Uptime Industries wants to boost localized AI usage with an ‘AI-in-a-box’ called Lemony AI

Summary

Uptime Industries is taking a bet on on-premise AI infrastructure with an “AI-in-a-box” handheld device called Lemony AI. The size of a sandwich, Lemony AI can supposedly run a large language model (LLM), AI agents and AI workflows all on one node. Each Lemony requires only 65 watts of power to run, the equivalent of charging a laptop, and can be stacked and connected to form bigger AI clusters. Sascha Buehrle, co-founder and CEO of Uptime, told TechCrunch that each node can sustain an LLM up to 75 billion parameters in size, and can host open-source models or retrofitted versions of closed models. In a cluster of Lemony nodes, each device can run a different model. Uptime is launching with partnerships with IBM and JetBrains to provide its customers easier access to AI models, including IBM’s closed models. Like most tech startups, the idea for Lemony AI stemmed from a side project: Uptime’s co-founders, Buehrle and Ivan Kuleshov, were trying to figure out if they could distribute language models on Raspberry Pi’s micro computers, and while this wasn’t originally related to generative AI, when it worked, they started trying to see what else these Raspberry Pis could handle. When they realized running models on local devices could prove the key to unlock more AI adoption by organizations — say, enterprises that don’t want to use cloud-based models — they started building a device of their own. They figured going small with an emphasis on data privacy could get companies to adopt AI quicker. “We need to build something small which can go easily into the teams, and which [does] not require any organization-wide decision making — bringing on-premise generative AI solutions into the business teams, basically,” Buehrle said. Clusters of small and powerful devices can help the system grow according to a customer’s needs, he added. The company says it has already seen strong demand from business in heavily-regulated industries like finance, healthcare and law. “Everything ...