My experience suggests that LLMs become not less accurate, but less helpful.
Two years ago they output a solution for my query [1] right away, now they try to engage user to implement that thing. This is across the board, as far as I can see.
These LLMs are not about helping anyone, their goals are engagement and mining data for that engagement.
[1] The query is "implement blocked clause decomposition in haskell." There are papers (circa 2010-2012), there are implementations, but not in Haskell. BCD, itself, is easy, and can be expressed in a dozen-two lines of Haskell code.
> These LLMs are not about helping anyone, their goals are engagement and mining data for that engagement.
Wow, this is a really interesting idea! A sneaky play for LLM providers is to be helpful enough to still be used, but also sufficiently unhelpful that your users give you additional training data.
This is obvious in retrospect - instead of making LLMs work better, LLM's handlers invented various techniques to make LLMs to look like they work better, one such example is summarization. Next gen LLMs then get trained on that data.
Now instead of having some answer right away, the user has to engage in discussion, which increases the cost that is sunk into the work with LLMs.
Two years ago they output a solution for my query [1] right away, now they try to engage user to implement that thing. This is across the board, as far as I can see.
These LLMs are not about helping anyone, their goals are engagement and mining data for that engagement.
[1] The query is "implement blocked clause decomposition in haskell." There are papers (circa 2010-2012), there are implementations, but not in Haskell. BCD, itself, is easy, and can be expressed in a dozen-two lines of Haskell code.