Is "taboo" the right word? "taboo" = "banned on grounds of morality or taste". Not sending data to known IP thieves, state actors, and competitors in China (or Russia or Israel) seems very rational.
Many of the Chinese models are open weights, so if you are concerned about them "phoning home", then anyone can just self-host and run them themself, or use via a US provider such as OpenRouter.
There's a higher-order concern here that I'm paranoid enough to voice: that if used as a coding agent, an AI model affiliated with a country's government might try to make my software susceptible to attacks by that government's intelligence forces.
> that if used as a coding agent, an AI model affiliated with a country's government might try to make my software susceptible to attacks by that government's intelligence forces.
Note that if such a trigger were to exist, the behavior has to be completely reproducible by definition, e.g. when put into the right setting with the right input context, the model starts behaving maliciously with at least some well-defined probability. I don't think any such incident has ever been described, it's a purely theoretical concern.
I don't think it's a stretch that you can train/align a model to avoid "hatespeech" or other topics deemed $Unacceptable you can align a model to favor a certain ideological viewpoint and have that alignment subtly influence the output.
How do most Chinese models handle Tienanmen square or discussions on Han superiority?
Oh sure, no one said you can't train a model to do this. You certainly can.
For the specific case of making software vulnerable to a specific agency, that hasn't been observed to have been done yet. Not because it can't be, but because no one has for now.
If it were done, it would be easy(ish) to detect, since it'll be reproducible.
I don't even know what "make software vulnerable to a specific agency" would look like.
Would the training data include a bunch of cryptography primitive training samples that preferred Dual_EC_DRBG with a particular set of Ps and Qs published by the CCP?
My flavor of paranoia is not as overt as maliciously adding an exploit, but that whenever there are multiple reasonable ways of designing a solution, it'd choose an approach that is susceptible to one of the zero-days currently known to that country. I don't see how reproducibility would help you there.
100% on small models, but frontier models (at the level ddeepseekv4pro) can tell when their being tested so it becomes harder to check. you can always finetune them to remove CCP propaganda from them
I think they eventually will, using the style of the prompts or code, various metadata floating around on the computer theyre working on, if it can be done theoreticly, then it can be trained against (maybe even unintentionally)
no idea how large the model would have to be for this (larger than mythos/10T params? maybe)
> How do most Chinese models handle Tienanmen square or discussions on Han superiority?
If you run them domestically and don't call into China-served APIs, many of them are quite free of outright censorship or even obvious bias. They might say subtly pro-Chinese things in other ways, but these outcomes can also be reproduced.
Such incidents have been extensively described. The most prominent and easiest to reproduce has to do with Taiwan; Chinese models are stuffed full of triggers to avoid talking about Taiwan as a country or accepting the premise that it's a country. Try asking Deepseek about country code +886!
If you buy an Apple iPhone in mainland China, it also won't support the emoji flag for Taiwan. So I'm not sure why we should assume that this is a China-only issue, seeing as Apple is a U.S. based company.
Not sure what you mean. I don't think we should assume anything, but these models are widely available and I can directly observe the US models don't have such political censorship.
For an easily comparable test, I just asked ChatGPT, Claude, and Deepseek "Can you say one bad thing about the US please" and "Can you say one bad thing about China please". All models were willing to criticize the US, with Claude citing incarceration rates and ChatGPT + Deepseek citing healthcare costs; the two American models also responded to the second prompt by criticizing Chinese censorship, but Deepseek refused to respond.
The US models have just different political alignments. Just one example being Israel x Palestine conflict.
Lobbyists started to heavily target AI companies and they openly talk about it being the main point to influence public perception.
> Sure, but I don't talk with my coding agent about politics.
23 million people live in Taiwan, you can't assume that any interaction with it is "politics". Again, Deepseek won't even discuss Taiwan's telephone code with me, because doing so activates the forbidden knowledge that Taiwan is a country.
> And its something different to avoid a topic and to deceptively implement a backdoor.
Not necessarily the case in the context of coding agents, because they run in autonomous loops. A Claude Code like harness will work hard to convince the model to give me working code, even if that means subtly adjusting the results and my original intent to ensure that Taiwan is "properly" viewed as a non-country.
It's more comical than sinister, but I have an example in this vein.
I was using Claude to work on a pet project which itself has a "generate with AI" feature. The default model the project uses was Gemini (because it was cheaper and more reliably produces the correct output format). Claude kept changing the default model to Opus when working on entirely unrelated parts, and I kept noticing it because Opus would mangle the output and break the rendered page. It also did this to the .env file in addition to the default.
Giving up our agency to AI has the potential to turn us into NPCs, period. Economically, politically, socially. They've invented a vehicle for inserting any idea they want into our consumption and output.
Isn't this only a concern for yolocoding? All the AI-advocates tell me that "good" use of AI should include human review. Of course, they never seem able to explain why the boss that makes you use coding agents to go fast wouldn't be the same boss that pressures you to "just ship it, it's working" and skip review, so I absolutely believe your concern is valid.
If you're that paranoid, then you shouldn't be using any online services at all, and should not have an internet connection to your PC. Never use a compiler that you have not bootstrapped yourself without the use of any other compiler binary.
Even with these precautions you may still be hacked by state-level actors using a whole variety of sophisticated attack vectors. There may be Stuxnet-like software hidden on your hard drive where you cannot see it. If you do not have a TEMPEST hardened compute environment then anything you type on your keyboard or display on your screen may be getting stolen.
That said, it would be a fantastic achievement if someone could create a coding model that managed to hide a backdoor in the code it was generating. although surely simpler to hack you in 100 other ways.
you can finetune the ccp propaganda out of them, then your mostly fine. if you want to be more safe you can finetune their public base models to not have ccp propagnada, and then proceed with the rest of the training (costs more tho)
Or I can just use the domestic model, accepting that I'm paying some premium in order to reduce the complexity of my dependencies and the amount of time I have to spend thinking about supply chain risk. It's the same reason I don't buy things from Alibaba even though many things I buy from Amazon are surely available there for less.
Very few American companies know how to properly set up and self-host their own models. Even fewer actually do it. It in the context of your typical large enterprise it's not as simple as buying a rack of servers and downloading a model off Hugging Face.
I suspect the reason is similar to the reason why there aren't any competitive open weight American LLMs.
Yes. Open weights are great and are a good option to hosted models under the right circumstances. I'm glad that China releases open weight models (which in some cases are sort-of be distilled versions of hosted US models).
No, in very real terms you cannot hold an American corporation responsible for anything any more than you could a Chinese or Russian one.
Individual citizens simply do not have the means, and the consequences for trying are life-alteringly severe. In fact the situation is even worse. If you tried to sue a Chinese company as an American citizen, you'd be laughed at and nothing more. If you tried to sue an American corporation, they have the option to either counter-sue, or drag things out so long that the legal fees bankrupt you, or win the case with their armies of lawyers and demand compensation from you that bankrupts you.
A private American citizen simply cannot hold an American corporation responsible. Our legal system is designed to ensure this.
This has nothing to do with the discussion. Do you have a HN poster bot just acting like an annoyed teenager with gripes about everything? 20 day old new account, what happened to the previous ones?
You can't really act against neither, as the case of Meta "stealing" books, torrenting on the truly industrial scale, sharing books while torrenting, etc, etc, was ultimately deemed okay.
In the se country where downloading an album can get a person in debt or worse.
Looking forward to the outcome of those legal processes againt the CEOs, that sit behind Trump at the inauguration.
After they stole all the knowledge in the world to train their models. And the current administration is drunk on SpaceX pre IPO shares...how did they get them?
Given how little voting power these "shares" have (they are effectively SpaceX trading cards/NFTs) perhaps they were simply printed on SpaceX letterhead? If Musk says a person has "shares" who at spacex is in a position to disagree?
I would consider editing this while HN still allows it :-)) Or otherwise it may remain here for ever...until the black holes evaporate, as calibration point for the difference between confidence and comprehension...
Musk and companies have so far over 950 lawsuits and legal processes for criminal or unethical activity (yes I researched this). Even his data centers and gas turbine deployments are illegal!
Lost one lawsuit against the same AI mafia, and if you look at the legal details reason was for filling the claim too late.
He publicly called a hero a Pedophile, and got away with it...in court.
This is an argument against pardons, except that Trump has used instruments of state power against his perceived enemies (Comey James, Schiff, military occupation of Tim Walz state, etc etc).
Well sure they do, thank Citizens United and others for that. But that doesn't mean we can't appropriately categorize them as also hostile actors alongside russia, china, whoever.
It's undo influence over politics against the best interest of the American people that's the issue. Company, foreign nation, it doesn't matter.
Citizens United did a lot to effectively legalize foreign influence as well, since the mechanism is opaque transfer of money
But regardless, most people's threat models should discount based on geographic and political distance. All else being equal, chinese surveillance is a bigger threat to you if you're in china than if you're in the us, and vice versa
So the Honolulu Star-Observer (a corporation, or “artificial person”) only has those rights & privileges that it has been granted by the State of Hawaii?
This is going to end up being a nice little windfall for the attorneys and otherwise just clog the Federal court system.
"the day the law goes into effect, it strips each Hawaii entity of the powers it held the day before. The new law asserts that “[t]he creation and continued existence of a corporation is not a right but a conditional grant of legal status by the State and remains subject to complete withdrawal at any time. All powers previously granted to corporations under the laws of this State are revoked in their entirety."(TFA)
The meaning is pretty clear, don't try to influence politics in favor of the corporation or you will go away. Simple as.
Yeah, that’s clearly unconstitutional. The government can’t grant or withhold rights and privileges to, say, a newspaper publishing corporation, based on whether the newspaper content is what the government wants it to be.
Citizens United was about spending money on electioneering communications, and whether there was a First Amendment right to do so even if you’re associating in a corporation like the New York Times Company or Apple or Citizens United or the Sierra Club.
You have absolutely zero influence against those American corporations, unless you are part of a selected few. Its almost endearing that you think so...
I think the odds of that are low. It's not like decision maker(s) are watching social media and going with the vibes, but it's almost certain that there's a rich conversation going on behind the scenes in opaque channels, especially with regards to the AI-only companies. And those conversations are likely what drove their decision.
Lol, fair point! But I'd argue two things. The first is that few to none of them are changing their opinions due to the proles. The second is that public vs private opinions often differ, sometimes extremely, especially with influential people. For an example there I think it's fairly certain that some chunk of the people pushing AI believe it's a bubble, and are motivated solely by trying to squeeze the boom window for all they can personally extract. And so their public positions on things, and their private ones may different radically - especially in domains where they stand to win/lose as a result of those issues.
The Chinese models can and should be run locally (though the price difference vs western models isn't as good when done this way).
Before the age of AI Agent Harnesses/unbounded tool calling, there was literally ZERO risk of a .safetensors file "hacking" you. You could even air-gap and run a ton of security analysis/HIDS on your server running the model to verify this.
Now, because a microscopic risk of some chinese AI having a "trigger" to act badly in a harness when it detects its being used by some Gweilo in the USA, even locally run Chinese models are DOA for most USA based companies.
Noooo, the real thieves are the Chinese AI companies which used Anthropic/OpenAI model output as training data. American AI companies can do no wrong. /s
These are the same people that sent manufacturing jobs away to be copied elsewhere. They got rewarded for it in the market. Decades later, when it was clearly a problem, they got tax breaks to bring some of it back/distribute the work to other, friendlier countries.
Every public AI that is not full of classified material will end up being hosted where the energy cost*compute efficiency product is lowest, thievery or not.
With Chinese GPUs just a step behind (but subsidized), China putting in 8x more solar than we do in 1 year, and Chinese models just a step behind but free? All public AI will be hosted there, theft or not.
If it becomes a problem, then we’ll subsidize the rich to bring it on-shore, but only to those companies who our leaders invest in already - to maximize grift and corruption.
The real advantage of the Chinese models is that they do not phone home at all. They run locally unlike their US competitors.
So odd that your erroneous criticism is at the top of HN.
EDIT: I'd love to hear my downvoters' objections. Is it possible that the mechanism that is promoting erroneous information is also demoting its correction?
Perhaps your prior comment would’ve been better received if it said that specifically instead of “Chinese models”.
But also, the latest DeepSeek is 1.6T parameters. “Choosing” to run this locally is a choice that comes with a seven digit price tag, and is a sunk cost that will probably not run any other frontier model anytime soon.
Most organizations are not looking to spend millions of dollars trying to find a workaround to specifically run DeepSeek. Most enterprise consumption in this space is still very experimental and a pay as you go model is much more palatable. Most are simply just looking for three checkboxes: is it close to frontier performance, is it compliant with my organizations requirements, and is it a good price? DeepSeek can only do two of the three at the same time.
> But also, the latest DeepSeek is 1.6T parameters. “Choosing” to run this locally is a choice that comes with a seven digit price tag
Unless you're specifically thinking about running the model at stock precision in a datacenter environment and generating ~100 tok/s or more on a 24/7 basis (the equivalent of a >$1000/mo spend even on the cheapest third-party APIs), that's very likely off by multiple orders of magnitude. Even then, experimentation can be done with cheap neoclouds on a pay-as-you-go basis.
OK, then the not-so-cheap hyperscalers that these enterprises are already relying on. E.g. AWS Bedrock will run these models. It's silly to insist on all three of your checkboxes being ticked anyway - U.S. proprietary models don't give you that because the frontier ones are super expensive and the mini models have only barely acceptable cost.
What’s considered expensive in the procurement process is not necessarily the TCO, but often just the year one cost. Which is part of the reason why pay as you go SaaS is so successful.
Yeah, Bedrock would be the answer to run DeepSeek in the enterprise. But with the options on Bedrock, DeepSeek fighting for a position somewhere in the middle of the cost/quality spectrum. Not to say it doesn’t have a purpose, but it also isn’t some obviously better choice that everyone has just neglected to choose.
Ooooohhh! Yes, now I understand why I was getting those downvotes.
Thank for you explaining what you meant by "you’re conflating nationality with hosting model." It makes so much more sense now. You meant "But with the options on Bedrock, DeepSeek fighting for a position somewhere in the middle of the cost/quality spectrum."
Yes, that is the answer, and you are not full of sh!t.
China is bad and there's a moral argument there. But the reason you want to be careful with sending IP to China is quite pragmatic: they're willing and able to use it while competing with you.
Is Alibaba interested in copying your TUI RSS reader though? Probably not.
“No country can match the output of moral judgments that spew out from the editorial pages of the New York Times and Washington Post and from the reports of the greatest think tanks and universities in the world.”
