This week in AI: Apple won’t say how the sausage is made

Hey guys, and welcome to TechCrunch’s regular AI newsletter.

This week in AI, Apple stole the spotlight.

At the company’s Worldwide Developers Conference (WWDC) in Cupertino, Apple unveiled Apple Intelligence, its long-awaited ecosystem-wide push to build artificial intelligence. Apple Intelligence has a range of features, from an upgraded Siri to AI-generated emojis to photo editing tools that remove unwanted people and objects from photos.

The company promised that Apple Intelligence is built with security at its core, along with highly personalized experiences.

“It needs to understand you and build on your personal context, like your routine, your relationships, your communications and more,” CEO Tim Cook noted during Monday’s keynote. “All of this goes beyond artificial intelligence. It’s personal intelligence and it’s the next big step for Apple.”

Apple Intelligence is classic Apple: It hides flawless technology behind obvious, intuitively useful features. (Not once did Cook utter the phrase “large language model.”) But as someone who writes about AI for a living, I wish Apple had been more transparent — this time — about how the sausage was made.

Take, for example, Apple’s model training practices. Apple revealed in a blog post that it trains the AI ​​models that power Apple Intelligence on a combination of licensed datasets and the public web. Publishers have the option to opt out of future training. But what if you’re an artist curious about whether your work was scanned at Apple’s initial training? Tough luck – the word is mum.

Secrecy may be due to competitive reasons. But I suspect it’s also to protect Apple from legal challenges — especially copyright-related challenges. Courts have yet to decide whether vendors like Apple have the right to train on public data without compensating or crediting the creators of that data — in other words, whether the fair use doctrine applies to genetic artificial intelligence.

It’s a bit disappointing to see Apple, which often describes itself as a champion of common sense technology politics, tacitly embrace the fair use argument. Cloaked behind the veil of marketing, Apple can claim to take a responsible and measured approach to AI, while it may well have been trained in the works of unlicensed creators.

A little explanation would go a long way. It’s a shame we haven’t gotten one — and I don’t hope we will anytime soon, barring a lawsuit (or two).

News

Apple’s top AI features: Yours truly rounded up the top AI features Apple announced during its WWDC keynote this week, from upgraded Siri to deep integrations with OpenAI’s ChatGPT.

OpenAI is hiring executives: OpenAI this week hired Sarah Friar, the former CEO of hyperlocal social network Nextdoor, to serve as chief financial officer and Kevin Weil, who previously led product development at Instagram and Twitter, as chief product officer.

Mail, now with more artificial intelligence: This week, Yahoo (the parent company of TechCrunch) updated Yahoo Mail with new AI features, including AI-generated email summaries. Google recently introduced a similar digest feature — but it’s behind a paywall.

Controversial views: A recent study from Carnegie Mellon finds that not all generative AI models are created equal—particularly when it comes to how they deal with the polarizing issue.

Sound Generator: Stability AI, the startup behind AI-powered art generator Stable Diffusion, has released an open AI model for generating sounds and songs that it claims was trained exclusively on royalty-free recordings.

Research paper of the week

Google believes it can create a productive AI model for personal health — or at least take preliminary steps in that direction.

In a new newspaper featured on the official Google AI blog, Google researchers are pulling back the curtain on the Personal Health Large Language Model, or PH-LLM for short — a refined version of one of Google’s Gemini models. The PH-LLM is designed to provide recommendations for improving sleep and fitness, in part by reading heart and breathing rate data from wearable devices such as smartwatches.

To test PH-LLM’s ability to make useful health recommendations, the researchers created nearly 900 sleep and fitness case studies involving US-based individuals. They found that the PH-LLM provided sleep recommendations that were near — but not as good as — the recommendations given by sleep experts.

The researchers say PH-LLM could help shape physiological data for “personal health applications.” Google Fit comes to mind. I wouldn’t be surprised to see PH-LLM eventually powering some new feature in a fitness-focused Google app, Fit or otherwise.

Model of the week

Apple has devoted several blog posts detailing the new AI models built on the device and in the cloud that make up the Apple Intelligence suite. However, despite how long this post is, it reveals precious little about the models capabilities. Here’s our best attempt at breaking it down:

The anonymous model on the device that Apple points to is small in size, no doubt it can work offline on Apple devices like the iPhone 15 Pro and Pro Max. It contains 3 billion parameters—the “parameters” are the parts of the model that essentially define its capabilities in a problem, such as text generation—making it comparable to Google’s Gemini model on the Gemini Nano device, which comes in parameters 1, 8 billion and Sizes 3.25 billion parameters.

The server model, meanwhile, is bigger (how much bigger, Apple won’t say exactly). What we I am doing i know it is more capable than the device model. While the device model performs on par with the likes of Microsoft’s Phi-3-mini, Mistral’s Mistral 7B and Google’s Gemma 7B on Apple’s benchmark lists, the server model “compares favorably” with OpenAI’s older flagship model GPT-3.5 Turbo, Apple claims.

Apple also says that both the device model and the server model are less likely to derail (ie, mouth toxicity) than similarly sized models. That may be so — but this writer reserves judgment until we’ve had a chance to test Apple Intelligence.

Grab bag

This week marked the sixth anniversary of the release of GPT-1, the ancestor of GPT-4o, OpenAI’s latest flagship AI model. And while Deep learning can hit a wallit’s incredible how far the field has come.

Consider that it took a month to train GPT-1 on a data set of 4.5 gigabytes of text (the BookCorpus, which contains about 7,000 unpublished fiction books). GPT-3, which is nearly 1,500 times the size of GPT-1 by number of parameters and much more sophisticated in the prose it can generate and analyze, took 34 days to train. How’s that for scaling?

What made GPT-1 groundbreaking was its approach to training. Previous techniques relied on massive amounts of manually labeled data, limiting their usefulness. (Manually labeling data is time-consuming — and laborious.) But GPT-1 didn’t. trained mainly in no label data to ‘learn’ how to perform a series of tasks (eg writing essays).

Many experts believe we won’t see a paradigm shift as significant as GPT-1 anytime soon. But then again, the world didn’t see GPT-1 coming either.