The LLM produces a probability distribution over the likelihood of all possible next tokens. So whatever the tokens are, "ch", "ex", etc. the next one gets a probability.
During training, real life text is fed through the LLM, and rhe "correct" token is the one actually observed in the training text. Here's a recent video walkthrough in some detail, mostly aimed at providing a deeper understanding than "next token predictor function":
Thanks - that makes sense. On that basis the article’s thesis is totally wrong - it would be like a computer program rating its ability based on how well it predicts moves played by grandmasters in the past. It’s not inventing new moves.
I wouldn't necessarily say that. Anybody who's playing a chess game is predicting their next move, whether or not they're inventing new moves.
LLMs are not simple things like a Markov model, there's a lot going on in there, it's not deterministic, and it's completely capable of creating entire new styles of play based on complex interactions of internal states.
Argh, what I wrote was obviously wrong. What I meant was to refer to the simple n-state Markov models as used past decades.
Anything that's sequential, like language is, will exhibit Markovian properties, and be somewhat a "Markov" model.
Markov chains are a different concept than a Markov model, but I do agree that, technically, an LLM is a Markov model, just with an internal state space that is nothing like what is usually meant when ML people refer to Markov models.
During training, real life text is fed through the LLM, and rhe "correct" token is the one actually observed in the training text. Here's a recent video walkthrough in some detail, mostly aimed at providing a deeper understanding than "next token predictor function":
https://youtu.be/GlYgs6v2YfU?is=IxVMhoCCE4N4WRVK
(Start at 15:30 for the LLM specific parts)