If I ask three models to write an intro to the cold war, they'll all try to pick words that sound like they should be related-ish. I'm not saying that's how they work at all, but the output is indistinguishable from just grabbing some words in the wikipedia page.
Humans make mistakes. They'll use words they recently learned. They'll use words that sound good. Entropy still applies, but these outliers are what keeps us from a synthetic piece of writing
Especially with how they pick (one of) the most likely word as the next one. And the most likely word is exactly the one with least entropy, the least surprising one and giving the least amount of information you can.
If I ask three models to write an intro to the cold war, they'll all try to pick words that sound like they should be related-ish. I'm not saying that's how they work at all, but the output is indistinguishable from just grabbing some words in the wikipedia page.
Humans make mistakes. They'll use words they recently learned. They'll use words that sound good. Entropy still applies, but these outliers are what keeps us from a synthetic piece of writing