So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays.

I think you are missing a lot of details here, honestly.

> Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.

No, the hardware is a bottleneck, the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks (which is useful). Which is exactly the opposite of what singularity and AGI have been promising.

The systems we have now (unless the paradigm changes) will keep doing, essentially, fitting. No concept of truth and limited inference. That inference is based on already existing data, not on future data. In fact, there have been experiments about feeding output back to the input of LLMs and the degradation of the quality is very visible. If they are supposed to be so "intelligent", why it happens?

> It doesn't need to be perfect

I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.

> Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.

I will not deny there could be progress, but nothing similar to "autonomous", "reliable", "super intelligence" or "singularity" with this paradigm.

In fact, often in my experience, this is a waste of tokens for subpar results that shift the technical debt elsewhere. I mean if you try to develop full systems by "vibe-code like" techniques. If you use them judiciously, you can accelerate your workflow, maybe 2x, but not much beyond that if you want to have something worth to be used. Note that here I am talking about the full thing: with testing, quality, maintenance concerns and everything together.

If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Unless your job is convincing enough people that you can go 10x faster every time, deliver some sub-par thing, and find another customer, which, to me, would equal a scam.

I'm pretty sure we will solve this issue. Either already through World Models or another architecture.

It could also be, that we just need a 10 or 100 Trillion Parameter model to match so many generic ways of solving tasks and keeping the concept in the LLMs 'head' to solve it that it will just emerge with parameter size. Like with fable they said that it can chain together exploits which wouldn't work as standalone exploits.

What if the only real barrier is the depth of understanding of concepts and this is exactly what is getting solved with parameter count?

But look how young this field really is if you start counting it when it became relevant on mass. Its not 'just' an LLM which is changing the world, its machine learning overall. Robotics wouldn't be were it is today if its not for machine learning. Took humans time and energy to take the leap, to start learning what the status quo is and then actually doing more with it.

> the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks

While I do think its doable to achieve AGI in 5-15 years, even if it doesn't happen and it always means that people train an LLM or whatever, if you need 10 experts to teach this to an LLM OR every single senior has to teach this to their juniors every single time, the LLM will always win.

I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.

the craziest thing about this? If i wouldn't tell them what they are doing wrong, they wouldn't even know it.

Quality is already a very flexible term for a lot of people.

> I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.

Yet.

> If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way.

Now we come to the reality: I have seen so much garbage software its crazy. People using md5 as a password hash in 2024! No clue what coding best practices are, teams without code review, teams without a security expert not even knowing what crazy things they do day in day out.

Just a few month ago a team build an API for my team including a Swagger UI. Half of it didn't work. You pressed a button on the Swagger UI and a 500 returned.

And do'nt underestimate what it means that a lot of business people don't like software people. You know that fruit basket we get? and water and stuff? they don't do it because they like us they do it because thats what you have to do. if a Product Owner starts vibe coding with AI, he will have leadership convinved in no time, then it goes on production and it will run for waaaaaay longer than anyone would have guest.

Besides that there is plenty of software were complexity is less relevant or security is not that big of an issue.

Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm.

This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that.

> I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.

I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like.

But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat.

Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.

Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.

TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%.

Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.

Reinforcement learning can just solve things even if they are new. It doesn't understand how a tool works? Give it a vm with the tool, a thousand agents and let it discover it automatically.

Use the thumbs up/down emoji + chat analysis when a customer is unhappy, feed that to a RL Loop.

The AI Researchers though work on World Models, grounding the AI and letting it simulate. It can do the simulation in parallel (unlimited) and choose what is best.

> I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of.

But thats my problem. Soooo many do not have this even as senior developers.

> Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.

Yeah now i just ask the LLM to describe to me the bug. Works very well.

> Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.

This is the thing. It only needs to make the team 10-30% better to compensate token budget with one work collegue. We have reached this level in my opinion already. Choosing a head count vs. choosing tokens.

But it becomes cheaper and easier and better. So you will not just be able to do ith with 3 agents but with 20, 50 or 100.

It will be better if your team is an avg team. It will be worse if you have a high profile team, for now. But man our industry has such a weird broad quality spectrum.

> TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns

In worst case, my team always do code reviews, I do a code review on an ai instead of a human and adjust the harness or the infos the ai can access. I can actually work on making this workflow better and then i can clone it or spin it up for every single PR. For a human? I have to train them and they might leave.

But there are plenty of cases were code doesn't matter. Researchers write a lot of random shitty uggly code as long as it does what it does, it doesn't matter. I have scripts for small tasks, we have microservices which do one thing because it is a tech stack we only need for one use case.

I think you are confusing giving theories about what a bug might be with certainty. It does help bc it csn accelerste things, but many times I had AIs with challenging bugs throwing a lot of misleading theories to me. For the easier bugs, I was just as capable most of the time. Not every time, so there is some potential time saving there. But also time waste.

As for research and fast prototyping you are right: I find it a good tool to explore bc yiu do not need the quality of a final product and researxh is in big part throwaway work.

But I was talking about software that needs features, maintenance, etc. This is just not the same thing.