Much of the conversation around artificial intelligence today focuses on building cloud capacity and massive data centers to run models. Companies like Apple and Qualcomm are in the early stages of making in-device AI more useful. Amidst all this, London’s 14-strong technical team Mirai is working to improve how models work on phones and laptops.
Mirai, which is backed by a $10 million round led by Uncork Capital, was founded by Dima Shvets and Alexey Moiseenkov last year. Both founders have experience building scalable consumer applications. Shvets co-founded the face-swapping app Reface, which was backed by a16z. Shvets later also became a scout for the venture capital firm. Moiseenkov has been the CEO and co-founder of viral AI filter app Prisma for the past decade.
As consumer developers, both were thinking about AI and machine learning in devices even before genetic AI became popular, Shvets said.
“When we got together in London, we started talking about technology and realized that in the hype of GenAI and the adoption of artificial intelligence, everyone is talking about the cloud, about servers, about the coming AGI. But the missing piece is in the device [AI] for consumer hardware,” he told TechCrunch.
Shvets and Moiseenkov wanted to use artificial intelligence to create a pipeline that would allow them to enable complex tasks on the phone, which led them to launch Mirai. When they asked others who developed consumer apps, they heard that many also wanted better cost optimization and profit margin per token.
Today, Mirai is developing a framework for models so they can perform better on devices. The company has built an inference engine for Apple Silicon that optimizes performance on the device. With its upcoming SDK, developers can integrate the runtime into their apps with just a few lines, the company says.
“One of the visions of why we started the company was that we wanted to give developers, like this Stripe, eight lines of code [integration] experience… you basically go to our platform, embed the key and start working with your summarization, classification or any other use case,” Shvets said.
The startup built this engine in Rust, which can increase the speed of producing a model by up to 37%, they claim. The company said that while tuning the model for a platform, it doesn’t mind the model’s weights to ensure there is no loss in production quality.
Mirai’s stack is currently focused on improving text and voice modes on the platform, with plans to support vision in the future. The team has begun working with edge model providers to tune their models for use at the edge and is in discussions with different chipmakers. Later, it plans to bring its engine to Android as well.
In addition, Mirai aims to release benchmarks on the device so that model makers can test performance on the device. However, Shvets acknowledges that not all AI tasks can be done on the device. To enable a hybrid mode, the team is building an orchestration layer to send requests that cannot be fulfilled on the device up to the cloud.
While the startup doesn’t yet work directly with apps, its engine could power on-device assistants, transcribers, translators, and chat apps, we’re told.
Andy McLoughlin, managing director of Uncork Capital, noted that he has invested in a leading machine learning company for the past decade. He said the company was early and eventually sold its business to Spotify. In today’s world, the situation is different, he believes.
“Given the cost of cloud inference, something has to change… For now, VCs are happy to keep funding rocket ship companies, spending exorbitant amounts of money on cloud inference. But that won’t last — at some point, people will focus on the underlying economics of these businesses and realize that something has to change,” he said. “It seems that every modeler will want to perform some of the inference workload at the limit, and Mirai feels very well positioned to meet that demand.”
Mirai’s first round also included Dreamer CEO David Singleton, YC partner Francois Chaubard, Snowflake co-founder Marcin Żukowski, ElevenLabs co-founder Mati Staniszewski, former Google AdSense product manager and Coinbase board member Gokulunor Rajaramyco, C. Brajaramyco. Krishnan, Ben Parr and Matt Schlicht of Theory Forge Ventures and former Netflix technical lead Aditya Jami.
