Arcee, a US open-source AI lab, says Chinese models are not inherently dangerous

As China’s open-weight AI models grow in capability and popularity, arguments about what to do about them have reached a fever pitch again.

There is speech that the Trump administration may try to ban them (although he hasn’t acted yet in the idea). Meanwhile, proprietary model makers, notably OpenAI and Anthropic, appear are increasingly worried about them.

Open-source models such as Moonshot AI’s Kimi K3 or Alibaba’s Qwen offer inference at a fraction of the token cost of closed-source models from these major US labs. The fear is that they also pose some kind of threat. They certainly threaten the profit margins of large proprietary AI labs.

But should businesses running these models in their own data centers succumb to the fear that they could become a vehicle for Chinese hackers?

No, says Lucas Atkins, its CTO Arceewhich builds open models to give American companies a domestic alternative to Chinese models.

If any startup benefits from the ban on Chinese models, Arcee will benefit. But Atkins says China’s open models are no more dangerous than any other open source software a company might use. In fact, he says, they even offer benefits even to his own company.

“A lot of people see it as similar to a Chinese software program. Like, it was coded with these x, y, z intentions” that a bad actor could just command, he said.

“That’s not essentially how these models are trained. There’s really no way for an Arcee or an Alibaba to build a model, have someone run it in their own environment, and for us to have any access to it,” he explained.

While most of these models are what is known as “open weight” and are not truly fully open source software, the source code (the part that will actually run on servers), if obtained from open source sites such as Hugging Face, is similarly highly visible and assessable. (What is not available are the methods and data used to train the models.)

Large organizations should put the core of any model through their testing and safety review processes, and often train the models for their specific uses and can look at areas such as bias, toxicity, hallucinations, and sensitivity to certain subjects. So they work, optimize and understand the models before people start sending them prompts.

Could a model used for coding somehow drop malicious backdoors into the code it writes? Again, while this is theoretically possible, it would require acrobatic feats to achieve.

“There’s no reason that a fairly sophisticated actor couldn’t train a model to be an absolutely amazing coding model in every situation, but when presented with a certain type of code base … some hidden training would kick in,” argued Atkins, who spends his days training models. But he adds, “I don’t know how you would do that.”

Because large language models are inherently creative, the chances of a modern model destroying malware in response to a pre-engineered perfect storm of context and prompting are slim to none. Even less are the chances that any business will subsequently use this code.

Could it happen in the future? That’s anyone’s guess. However, enterprises are also building their AI applications to be model-agnostic and use multiple models. So even if the Chinese models are the best for the price today, businesses won’t be locked into using them forever.

“I think instead of the conversation being about how to ban Chinese models, it should be about how do we cultivate a good, open ecosystem here in the US,” Atkins says.

Arcee also gains advantages from Chinese models. Because they’re open, the startup “benefits from the fact that these models are good because we can learn what they’ve done. We can build on them. Then they can learn what we’re doing,” he says. “We have tremendous respect for the people who build these models, the individual researchers.”

Ultimately, the way to compete with Chinese models “is to release a model that’s better,” Atkins says. “We have to give them something to talk about.”