AI is all the rage — particularly AI that generates text, aka big language models (think models along the lines of ChatGPT). In a recent one overview of approximately 1,000 business organizations, 67.2% say they see the adoption of large language models (LLM) as a top priority by early 2024.
But obstacles stand in the way. According to the same survey, a lack of customization and flexibility, combined with an inability to preserve corporate knowledge and intellectual property, prevented—and is—many firms from developing an LLM in manufacturing.
That got Varun Vummadi and Esha Manideep Dinne thinking: What might a solution to the business LLM adoption challenge look like? In search of one, they founded Giga MLa startup building a platform that allows companies to develop LLMs on-premise — ostensibly cutting costs and maintaining privacy in the process.
“Data privacy and customization of LLMs are some of the biggest challenges businesses face when adopting LLMs to solve problems,” Vummadi told TechCrunch in an email interview. “Giga ML addresses both of these challenges.”
Giga ML offers its own set of LLMs, the “X1 series,” for tasks such as code generation and answering common customer questions (eg, “When can I expect my order to arrive?”). The startup claims that the models, built on top of Meta’s Llama 2, outperform popular LLMs in a number of benchmarks, particularly MT-Bench trial set for dialogs. But it’s hard to say how the X1 compares qualitatively. this reporter tested the Giga ML’s online demonstration but faced technical problems. (The app ended regardless of the message I typed.)
Even if the Giga ML models is Superior in some aspects, however, can they really take a dip in the ocean of open source, offline LLMs?
In talking with Vummadi, I got the feeling that Giga ML isn’t so much trying to build the best performing LLMs out there, but instead building tools that will allow businesses to fine-tune LLMs locally without having to rely on third-party resources and platforms.
“Giga ML’s mission is to help enterprises securely and efficiently deploy LLM on their own on-premises infrastructure or virtual private cloud,” said Vummadi. “Giga ML simplifies the LLM training, refinement and execution process by taking care of it through an easy-to-use API, eliminating any associated hassle.”
Vummadi emphasized the privacy benefits of running models offline — benefits that may be compelling for some businesses.
Predibase, the low-code AI programming platform, found that less than a quarter of businesses are comfortable using commercial LLMs due to concerns about sharing sensitive or proprietary data with suppliers. Nearly 77% of survey respondents said they either do not use or do not plan to use commercial LLMs beyond prototyping in production — citing issues related to privacy, cost and lack of customization.
“IT managers at the C-suite level find Giga ML’s offerings valuable due to secure on-premise LLM deployment, customizable models tailored to the specific use case, and rapid inference that ensures data compliance and maximum performance.” said Vumadi.
Giga ML, which has raised ~$3.74 million in VC funding to date from Nexus Venture Partners, Y Combinator, Liquid 2 Ventures, 8vdx and many others, plans in the near future to grow its two-person team and enhance product R&D. A portion of the fund is earmarked to support Giga ML’s customer base, Vummadi said, which currently includes unlisted “entrepreneurial” companies in the financial and healthcare sectors.
