Does anyone have any insight on the real inner working of chatgpt or its source code?
Because, so far it is advertised as a "magic" thing only working with a language model, but as a seasoned dev engineer, I'm quite sceptic.
When we look at all the replies, we can obviously see some patterns in the way question are replied. Something like:
Can you do operation x on y for me please?
Yes, opération X is reticulating that and this in a specific way like I'm reading Wikipedia.
For example bob become zob.
So, X(y) result in bar.
To me, I have the feeling that in addition of using gpt maybe for decoding, maybe for generating outputs, they might have a big base of predefined response "templates". Also, they can have specific "plugin" calculators or things like that, so that once tokenized, the operations would be performed by the plugin and not by some magic AI understanding.
It is easy as to pre record that + == plus == add. * == X == multiply == times.
Just to explain my scepticism to younger readers, in Emacs, for >40 years there was a fun and very light lisp plugin that was embedded: the cyberpsychoterapist.
It is based on this:
https://en.wikipedia.org/wiki/ELIZA
For anyone that tried that decade ago, you could have a 30 mins conversation without noticing that it is not a real person.
The fun trick in my youth was to feed the replies of it to my friends on Google Chat with them thinking that they were talking to me directly.
> they might have a big base of predefined response "templates".
Sort of. They fine-tuned the existing GPT-3 largest model on samples of dialog, which would be like your templates. The program doesn't "render templates", but the fine-tuning process has instructed it that responses following the template are statistically more likely to be the correct response to a given prompt. See their homepage [0], Methods section.
> Also, they can have specific "plugin" calculators or things like that, so that once tokenized, the operations would be performed by the plugin and not by some magic AI understanding.
This is likely the direction they are going to take it, but this tech demo doesn't seem to include it. Some of the "ChatGPT jailbreaks" suggest that they are experimenting with enabling web search, likely in a manner like you describe. [1]
Because, so far it is advertised as a "magic" thing only working with a language model, but as a seasoned dev engineer, I'm quite sceptic.
When we look at all the replies, we can obviously see some patterns in the way question are replied. Something like:
To me, I have the feeling that in addition of using gpt maybe for decoding, maybe for generating outputs, they might have a big base of predefined response "templates". Also, they can have specific "plugin" calculators or things like that, so that once tokenized, the operations would be performed by the plugin and not by some magic AI understanding.It is easy as to pre record that + == plus == add. * == X == multiply == times.
Just to explain my scepticism to younger readers, in Emacs, for >40 years there was a fun and very light lisp plugin that was embedded: the cyberpsychoterapist. It is based on this: https://en.wikipedia.org/wiki/ELIZA
For anyone that tried that decade ago, you could have a 30 mins conversation without noticing that it is not a real person.
The fun trick in my youth was to feed the replies of it to my friends on Google Chat with them thinking that they were talking to me directly.