Keeping up with an industry as fast-paced as artificial intelligence is a tall order. So, until an AI can do it for you, here’s a helpful roundup of recent stories in the world of machine learning, along with notable research and experiments we didn’t cover on their own.
This week, Meta released the latest in the Llama series of AI models: Llama 3 8B and Llama 3 70B. Able to parse and write text, the models are “open source,” Meta said — intended to be a “foundational piece” of systems that developers design with their unique goals in mind.
“We believe these are the best open source models in their class,” Meta wrote in a suspension. “We embrace the open source ethos of releasing early and often.”
There’s just one problem: The Llama 3 models aren’t Really open source, at least not in stricter definition.
Open source means that developers can use the models as they choose, without restrictions. But in the case of Llama 3 — as with Llama 2 — Meta has imposed some licensing restrictions. For example, Llama models cannot be used to train other models. And App developers with over 700 million monthly users must request special permission from Meta.
Debates about the definition of open source are not new. But as companies in the artificial intelligence space play fast and loose with the term, it injects fuel into long-standing philosophical arguments.
Last August, a study Writing by researchers at Carnegie Mellon, the AI Now Institute and the Signal Foundation, many AI models labeled as “open source” come with big catches — not just Llama. The data needed to train the models is kept secret. The computing power required to run them is beyond the reach of many programmers. And the work to perfect them is prohibitively expensive.
So if these models aren’t truly open source, what exactly are? This is a good question. Defining open source in relation to artificial intelligence is not an easy task.
A related unresolved question is whether copyright, the fundamental IP mechanism underlying open source licensing, can be applied to the various elements and pieces of an AI project, particularly the internal scaffolding of a model (e.g. integrations). Then there’s the need to overcome the mismatch between the perception of open source and how AI works: Open source was invented in part to ensure that developers could study and modify the code without restrictions. With AI, though, the ingredients you need to study and tweak are open to interpretation.
Cutting through all the uncertainty, the Carnegie Mellon study does clear up the harm inherent in tech giants like Meta who choose the phrase “open source.”
Often, “open source” AI projects like Llama end up starting news cycles — free marketing — and providing technical and strategic benefits to project maintainers. The open source community rarely sees these same benefits, and when they do, they are marginal compared to maintainers.
Rather than democratizing AI, “open source” AI projects — especially those from big tech companies — tend to entrench and expand centralized authority, the study’s co-authors say. This is good to keep in mind the next time a major “open source” model release is released.
Here are some other notable AI stories from the past few days:
- Meta updates its chatbot: Coinciding with the debut of Llama 3, Meta upgraded its Facebook, Messenger, Instagram and WhatsApp AI chatbot – Meta AI – with a back end powered by Llama 3. It also introduced new features, including faster image generation and accessing web search results.
- AI Generated Porn: Ivan writes about how the Supervisory Board, Meta’s semi-independent policy board, is turning its attention to how the company’s social media platforms handle clear AI-generated images.
- Snap Watermarks: Social media service Snap plans to add watermarks to AI-generated images on its platform. A translucent version of the Snap logo with a glitter emoji, the new watermark will be added to any AI-generated image extracted from the app or saved to your camera roll.
- The New Atlas: Hyundai-owned robotics company Boston Dynamics has unveiled its next-generation Atlas humanoid robot, which, unlike its hydraulic-powered predecessor, is all-electric — and much friendlier to look at.
- Humanoids to Humanoids: Not to be outdone by Boston Dynamics, Mobileye founder Amnon Shashua launched a new startup, MenteeBot, focused on building robotic biped systems. A demo video shows the MenteeBot prototype walking across a table and picking fruit.
- Reddit, translation: In an interview with Amanda, Reddit CPO Pali Bhat revealed that an AI-powered language translation feature to bring the social network to a more global audience is in the works, along with a helpful supervision tool trained on past decisions and actions of Reddit moderators.
- AI-Generated LinkedIn Content: LinkedIn has quietly begun testing a new way to increase its revenue: a LinkedIn Premium Company Page subscription, which — for fees that appear to be as high as $99/month — includes artificial intelligence to curate content and a series of tools to increase the number of followers.
- A Bellwether: Google parent Alphabet’s Factory X this week unveiled Project Bellwether, its latest bid to apply technology to some of the world’s biggest problems. Here, that means using artificial intelligence tools to identify natural disasters like fires and floods as quickly as possible.
- Protecting children with AI: Ofcom, the regulator charged with enforcing the UK’s Internet Safety Act, plans to launch an investigation into how artificial intelligence and other automated tools can be used to proactively detect and remove illegal content online, especially to protect children from harmful content.
- OpenAI lands in Japan: OpenAI is expanding into Japan, with the opening of a new office in Tokyo and plans for a GPT-4 model optimized specifically for the Japanese language.
More machine learning
Image Credits: DrAfter123 / Getty Images
Can a chatbot change your mind? Swiss researchers have discovered that not only can they change your mind, but if they’re pre-armed with some personal information about you, they can also be real more persuasive in a conversation than a person with the same information.
“This is Cambridge Analytica on steroids,” said project leader Robert West from EPFL. The researchers suspect that the model – GPT-4 in this case – drew from the vast reserves of arguments and facts on the internet to present a more compelling and confident case. But the result speaks for itself. Don’t underestimate the persuasive power of LLMs, West warned: “With the US election coming up, people are worried because that’s always where the first battle of this kind of technology is tested. One thing we know for sure is that people will use the power of large language models to try to swing elections.”
Why are these models so good at language anyway? This is an area that has a long history of research, dating back to ELIZA. If you’re curious about one of the guys who was there for a lot of it (and performed it himself), check out this profile to Stanford’s Christopher Manning. He has just been awarded the John von Neumann Medal. Congratulations!
In a provocatively titled interview, another longtime AI researcher (who has also graced the TechCrunch stage), Stewart Russell and postdoctoral fellow Michael Cohen speculate “How to stop artificial intelligence from killing us all.” Might be good to figure it out sooner rather than later! It’s not a superficial discussion though – these are smart people talking about how we can really understand the motivations (if that’s the right word) of AI models and how regulations should be built around them.
The interview is essentially relevant a paper in Science published earlier this month, in which they suggest that advanced artificial intelligences capable of acting strategically to achieve their goals (what they call “long-term planning agents”) may be impossible to test. Essentially, if a model learns to “understand” the test it must pass in order to succeed, it may well learn ways to creatively negate or bypass that test. We’ve seen it on a small scale, so why not on a large scale?
Russell suggests limiting the material needed to build such agents… but of course, Los Alamos National Laboratory (LANL) and Sandia National Labs just received their deliveries. LANL just had the ribbon cutting ceremony for Venadoa new supercomputer intended for artificial intelligence research, consisting of 2,560 Grace Hopper Nvidia chips.
Researchers examine new neuromorphic computer. Image Credits: Craig Fritz/Sandia National Labs
And Sandia just took delivery of “an extraordinary brain-based computing system called Hala Point,” with 1.15 billion artificial neurons, built by Intel and believed to be the largest such system in the world. Neuromorphic computing, as it is called, is not intended to replace systems like Venado, but rather to pursue new methods of computation that are more brain-like than the rather statistics-focused approach we see in modern models.
“With this system of billions of neurons, we will have the opportunity to innovate at scale both new AI algorithms that can be more efficient and smarter than existing algorithms, as well as new brain-like approaches to existing computer algorithms such as optimization and modeling,” he said. Sandia researcher Brad Aimone. It sounds like a rabbit…just a dandy!
