Maybe it's just muscle-memory, but clearly there are brains which develop it 100x faster than others, depending on the topic. See the aforementioned Mozart and the music topic. I think there are a lot of young and not so young adults whose peak performance in instrument playing is equal to Mozart's at age 8.
Because the existing cultural understanding of art is that someone took the time to create what you’re experiencing. AI generated “art” subverts that expectation. It feels deceptive. Honestly it reminds of Duchamp’s Fountain and similar works, which some people hated for more or less the same reason.
I am not equating AI slop with Marcel Duchamp, however. His work and what he did was very much intentional to evoke the sort of reactions it did.
Moleskin is selling notebooks, not art. They happen to come with graphical elements, but I don't see them claiming those are art. So where is the deception?
When I need a notebook, I just go into a store that sells them and grab whatever they have. I have a notebook for logging my exercises and another one for random stuff, I have no idea what brand they are.
Moleskin sells fashion items. Or, maybe, an idea of a certain lifestyle. I'm not sure.
> Moleskin is selling notebooks, not art. They happen to come with graphical elements, but I don't see them claiming those are art. So where is the deception?
Come on, they're selling notebooks with art on them. Cheap, AI-generated art, passed off as premium.
>Because the existing cultural understanding of art is that someone took the time to create what you’re experiencing. AI generated “art” subverts that expectation
And? I don't care. Is the art good or not? I'm not searching for someone to admire, I just want good music
> Creating a full plan by yourself feels like a waste of time, since you know the AI can create the same or better plan in a split second.
It IS a waste of time if your only goal is the creation of the plan. However, one must be very self-aware of their goals because if one of the unacknowledged ones is to retain the ability to create plans, then you must continue creating plans yourself.
Put differently: you get good at what you actually do, not what you think you're doing.
If you're not coding anymore, but using AI tools, you're developing skills in using those AI tools, and your code abilities will atrophy unless exercised elsewhere.
I find it humorous that the LLM's "confession" reads like an ascerbic comment you would find here on HN lambasting someone for accidentally deleting their production database, but re-written in the first person.
> It's akin to saying that every molecules behave randomly according to statistical physics, so you should expect your ceiling to spontaneously disintegrate any day, and if you find yourself under the rubble one day it's just a consequence of basic physics.
Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material.
Barring that, it is entirely possible and even quite likely that your ceiling will collapse on you or someone else some time in the future.
It boggles the mind to let an LLM have access to a production database without having explicit preventative measures and contingency plans for it deleting it.
I have lived about 40 years beneath ceilings and never personally taken a preventative measure. I allow my kids to walk under not only our own ceiling, but other people's ceilings, and I have never asked those people if their ceilings were properly maintained.
That highlights how important ceiling construction regulations are. I would assume that right now your breakfast sandwich is more highly regulated than LLMs. And these are the things that make decisions spanning from database maintenance here to target selection and execution in autonomous warfare.
The LLM agent is very good at fulfilling its objective and it will creatively exploit holes in your specification to reach its goals. The evals in the System Cards show that the models are aware of what they're doing and are hiding their traces. In this example the model found an unrelated but working API token with more permissions the authors accidentally stored and then used that.
Without regulation on AI safety, the race towards higher and higher model capabilities will cause models to get much better at working towards their goals to the point where they are really good at hiding their traces while knowingly doing something questionable.
It's not hard to imagine that when we have a model with broadly superhuman capabilities and speed which can easily be copied millions of times, one bad misspecification of a goal you give to it will lead to human loss of control. That's what all these important figures in AI are worried about: https://aistatement.com/
Your home almost certainly has preventative measures, including proper humidity and temperature control, structural reinforcement, etc.
I don't mean that you personally have taken those measures, but preventative measures have absolutely been taken. When they aren't, ceilings collapse on people.
See any sheetrock ceiling with a leak above it. Or look at any abandoned building: they will eventually always have collapsed floors/ceilings. It is inevitable.
I've had a ceiling fall on me once and once to a friend while on vacation. Just because it hasn't happened to you doesn't mean it hasn't happened to other people.
They're only sharing an annecdote because they are responding to your annecdote about not seeing a ceiling collapse.
> I don't think it changes the point of the metaphor.
If their anecdotes is moot, than your anecdote is also moot; if the anecdotes can only confirm a conclusion and never disconfirm, then we've created an unfalsifiable construction with the conclusion baked into it's premises.
Sure, I suppose that's something that someone who doesn't understand the discussion might say.
A person who better comprehends what they read might properly contextualize within the larger conversation, where the point that stands is that LLMs and ceilings are both useful, neither are doomed such that no one should use them, and that individual instances of failures are somewhat uncommon and not a reason for others to avoid the category.
> Sure, I suppose that's something that someone who doesn't understand the discussion might say.
I'm going to be frank, you are the person who misunderstands (and are being rather rude about it). You are responding to an argument no one is making.
To put a fine point on it, you said this:
> Entropy may mean all ceilings collapse eventually, but that doesn't mean we aren't able to make useful ceilings.
But you were responding to a comment saying this:
> Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material.
Emphasis added. They are saying maintenance is necessary, not that a safe ceiling is unachievable. It's obviously achievable, we've all seen it achieved.
They further say:
> It boggles the mind to let an LLM have access to a production database without having explicit preventative measures and contingency plans for it deleting it.
Emphasis added. When they say it boggles the mind to deploy an LLM without the proper measures, the implication is that it does make sense to deploy it with the proper measures.
> ...the point that stands is that LLMs and ceilings are both useful, neither are doomed such that no one should use them, ...
I have not seen a single person in this subthread say that LLMs aren't useful or that they are doomed. People say that. But the people you're talking to haven't.
I try to avoid these petty "I brought the receipts" comments, but I don't like the way you're being snarky to people who's crime is engaging with the premises you set up. The faults you are finding are faults you introduced. I'd appreciate if you would avoid that in the future.
If that's what you got out of the above conversation that is about as fundamental a misunderstanding as the one at the top of this thread saying "It is fundamental to language modeling that every sequence of tokens is possible". I could say something rude here about both mistakes being made by the same person, but since you brought it up I won't.
If you want to take a comb to it, the comment saying this:
> Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material
Was already off the plot. What was being discussed wasn't some specific molecular process, it was the false premise "oh molecules move around randomly so your ceiling might just collapse of its own accord because the beam decided to randomly disintegrate". That's not something that happens.
You said "The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use". This is analogous to "the ceiling could just collapse on you due to random molecular motion, no matter how much maintenance you do or what materials you use".
Make sense now?
Your edit at the bottom of your top comment does better than your original statement.
> What was being discussed wasn't some specific molecular process, it was the false premise "oh molecules move around randomly so your ceiling might just collapse of its own accord because the beam decided to randomly disintegrate". That's not something that happens.
Except it does happens. That’s why buildings get condemned and buildings eventually turn to rubble.
To the exact point; I have a product from a couple years ago using an old model from OpenAI. It’s still running and all it does is write a personality report based on scores from the test. I can’t update the model without seriously rewriting the entire prompt system, but the model has degraded over the years as well. Ergo, my product has degraded of its own accord and there is nearly nothing I can do about it. My only choice is to basically finagle newer models into giving the correct output; but they hallucinate at much higher rates than older models.
> I could say something rude here about both mistakes being made by the same person, but since you brought it up I won't.
I'd encourage to desist from rudeness, not just when people point it out to you, but at all times.
> You said "The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use". This is analogous to "the ceiling could just collapse on you due to random molecular motion, no matter how much maintenance you do or what materials you use".
If prompt engineering is effective (analogous to performing the necessary maintenance and selecting the correct materials), I'm curious what your explanation is for the incident in the article?
> I'd encourage to desist from rudeness, not just when people point it out to you, but at all times.
I desire neither to be inauthentic, nor to suppress my emotions.
> If prompt engineering is effective (analogous to performing the necessary maintenance and selecting the correct materials), I'm curious what your explanation is for the incident in the article?
Keeping with the analogies, the original article doesn't say whether they built the roof properly or if the just used some screws to hold up a piece of quarter inch plywood and called it a day.
It's no surprise that a terribly built roof may fall down. It's possible to get shoddy materials from a supplier without knowing.
Calling a curl command isn't something that would be within the model's training as "this deletes things don't do it". The fact that this happened is not, to me, evidence that the model might have equally run `sudo rm -rf --no-preserve-root /` under similar circumstances.
It sounds like the phrase "NEVER FUCKING GUESS!" was in the prompt as well, which could easily encourage the model towards "be sure of yourself, take action" instead of the "verify" that was meant.
As mentioned elsewhere in this thread, the fact that the article focuses so strongly on "the model confessed! It admitted it did the wrong thing!" doesn't lead me to put a ton of stock into the capability of the author to be cautious.
It's a basic "When you invent the ship, you also invent the shipwreck" principle.
Ceilings are usually build to be predictably collapsible and not to cause much damage. You will hear the cracking and see the sagging long before it will collapse, that's why you are reasonably safe walking under ceiling that looks good. If you never taken a preemptive measure to not go under unstable looking roof, that's on you. Or maybe on people that track that kind of thing and repair before damage is done.
LLMs will delete you prod, if given permission, so we need same engineering principles applied there as well. We need warning signs that something will collapse soon. We need to know what relatively safe ceiling collapsing looks like.
We are not some people walking in homes with a ceiling. We supposed to be people that build this houses and repair them in time, so the ceiling wouldn't collapse on the user heads!
As of right now, LLMs are shitty ceilings and shouldn't be given any access to prod.
It is merely a simulacrum of an intern or disgruntled employee or human. It might say things those people would say, and even do things they might do, but it has none of the same motivations. In fact, it does not have any motivation to call its own.
Mimicry is how kids learn the expected reactions to particular emotions. A kid mimicking your surprise doesn’t mean they are surprised (as surprise requires an existing expectation of an outcome they may not have the experience for), but when they do feel genuine surprise, they’ll know how to express it.
Because it's a statistical process generating one part of a word at a time. It probably isn't even generating "surprise". It might be generating "sur", then "prise" then "!"
But what is surprise really? Something not following expectation. The distribution may statistically leverage surprise as a concept via how it has seen surprise as a concept e.g. "interesting!"
So it can be both true that it has nothing to do with the emotion of surprise, but appear as the emulation of that emotion since the training data matches the concept of surprise (mismatch between expectation and event).
It’s the emotional and physiological response to a prediction being wrong. At its most primal, it’s the fear and surge of adrenaline when a predator or threat steps out from where you thought there was no threat. That’s not something most people will literally experience these days but even comedic surprise stems from that shock of subversion of expectation.
LLMs do not feel. They can express feeling, just as you can, but it doesn’t stem from a true source of feeling or sensation.
Expressing fake feelings is trivial for humans to do, and apparently for an LLM as well. I’m sure many autistic people or even anyone who’s been given a gift they didn’t like can relate to expressing feelings that they don’t actually feel, because expressing a feeling externally is not at all the same as actually feeling it. Instead it’s how we show our internal state to others, when we want to or can’t help it.
It is a mistake to equate artificial intelligence with sentience and humanity for moral reasons, if nothing else.
We are also technically a statistical process generating one part of a word at a time when we speak. Our neurons form the same kind of vectorised connections LLMs do. We are the product of repeated experiences - the same way training works.
Our brains are more advanced, and we may not experience the world the same way, but I think we have clearly created rudimentary digital consciousness.
Because it has no mind, no cognition, and nothing to "feel" with. Don't mistake programmatic mimicry for intention. That's just your own linguistic-forward primate cognition being fooled by the linguistic signals the training set and prompt are making the AI emit.
I could describe the electrical and chemical signals within your neurons and synapses as proof that you are merely a series of electrochemical reactions, and can only mimic genuine thought.
You could do that if you wanted to ignore reality and be reductive to score points in an argument by purposefully conflating mimicry with intention, yes.
And that is dogma. It's unthinking circular reasoning.
It wasn't very long ago that scientists were certain that animals did not posses thoughts or feelings. Any behaviour which appeared to resemble thinking or feeling was simply unconscious autonomic responses, with no more thought behind them than a sunflower turning towards the sun. Animals, by definition, lack Immortal Souls and Free Will, and therefore they are empty inside. Biological automata.
Of course this dogma was unfalsifiable, because any apparent evidence of animal cognition could be refuted as simply not being cognition, by definition.
Look, either cognition is magic, or it's math. There really isn't a middle ground. If you want to believe that wetware is fundamentally irreducible to math, then you believe it's magic. If that's want you want to believe, then fine. But it's dogma, and maintaining that dogma will require increasingly willful acts of blindness.
You are using word "math" in a magical way. Current LLM programs are reducible to math and human cognition is reducible to math (which is a reasonable hypothesis). What you are implying is that just because word math is used in both sentences it actually means the same thing. And that is a magical thinking. Just because human cognition is reducible to math (let's assume that for sake of discussion) doesn't mean it's the same math as in the LLM programs, or even close enough. Or maybe it is, but we don't have any proof yet.
I agree with this. I'm not arguing that LLMs are conscious. We don't understand the math behind how our brains work; we don't know how close or far LLMs are to that; and we don't know how many different pathways to consciousness there are within math.
All I'm saying is that the argument that "It's not consciousness, it's just <insert any tangentially mathematical claim here>", is dogma. Given everything that we don't know, agnosticism is the appropriate response.
> It wasn't very long ago that scientists were certain that animals did not posses thoughts or feelings. Any behaviour which appeared to resemble thinking or feeling was simply unconscious autonomic responses, with no more thought behind them than a sunflower turning towards the sun. Animals, by definition, lack Immortal Souls and Free Will, and therefore they are empty inside. Biological automata.
It's cool that you can decide to take half-remembered incorrect anecdotes about what "scientists" are certain of at some indeterminate time in the past, sans citation, and use that to underpin your argument about a totally different thing.
> Of course this dogma was unfalsifiable...
...like your post's anecdata.
> Look, either cognition is magic, or it's math.
Yes, when you decide to draw a convoluted imaginary bounding box around the argument, anything can be whatever you want it to be.
LLMs have no mind and no intention. They are programmed to mimic human language. Read some Grice and learn exactly how dependent humans are on the cooperative principle, and exactly how vulnerable we are to seeing intent where none exists in LLM communication that mimics the outputs our inputs expect to receive.
Your cries of "dogma dogma dogma" are unpersuasive and lack grounding in practical reality.
Any time someone starts explaining a new game to me I stop them and tell them to just start the game and walk me through it as we play. If I’m teaching someone a card game we’ll play open hand until they get it then start over. It’s kind of like a physical activity like riding a bike, you just gotta do it, not read about it.
2. Don’t assume you’re the next Mozart. Someone is, statistically it’s not you.
reply