ChatGPT launch has started in the age of the great language models. In addition to OpenAI offerings, other LLMs include Google’s LaMDA LLM family (including Bard), the BLOOM project (a collaboration between teams at Microsoft, Nvidia, and other organizations), Meta’s LLaMA, and Anthropic’s Claude.
No doubt more will be created. Actually, one Arize survey April 2023 found that 53% of respondents planned to develop an LLM within the next year or sooner. One approach to doing this is to create a “vertical” LLM that starts with an existing LLM and carefully retrains it in domain-specific knowledge. This tactic can work for life sciences, pharmaceuticals, insurance, finance and other business sectors.
Developing an LLM can provide a strong competitive advantage — but only if done well.
LLMs have already led to newsworthy issues, such as their tendency to “fake” incorrect information. This is a serious problem and can distract leadership from key concerns about the processes that produce these results, which can be just as problematic.
The challenges of training and developing an LLM
One problem with using LLMs is their huge operational overhead, because the computational demand for training and running them is so intense (they aren’t called large language models for nothing).
LLMs are exciting, but their development and adoption requires overcoming many feasibility hurdles.
First, the hardware to run the models is expensive. The H100 GPU from Nvidia, a popular choice for LLMs, sells on the secondary market for around $40,000 per chip. One source estimated it would take approx 6,000 tokens to train an LLM comparable to ChatGPT-3.5. That’s about $240 million for GPUs alone.
Another major cost is powering these chips. Just training a model is estimated to require approx 10 gigawatt hours (GWh) of electricity, equivalent to the annual electricity use of 1,000 US homes. Once the model is trained, its electricity cost will vary, but can become excessive. This source estimated that the energy consumption to run ChatGPT-3.5 is about 1 GWh per day, or the combined daily energy use of 33,000 households.
Power consumption can also be a potential pitfall for the user experience when running LLM on mobile devices. This is because heavy use of a device could drain its battery very quickly, which would be a major barrier to consumer adoption.
