I think the first goal of writing is to talk to yourself. Without an external, unchanging scratchpad, most people cannot keep their thoughts truly straight. It can feel otherwise, but trying to write something about any tricky topic even a few pages in length can take a long time to be self-consistent. Clarifying it takes even longer, especially when you intend to share it. Writing as thinking is definitely a thing. I believe it’s the trickiest part of understanding how such interesting things pop out of a language model trained to predict the next token and rewarded with pleasing sequences. Since it is not thinking, it proves that certain sentences (most?) can be generated thoughtlessly.
I say that LLM's are a pacifier. LLM output is to conscious output as a silicone nipple is to a breast.
Sure, at some level, there are some people who at least some times, cannot pecieve a difference. That does not prove that there is no difference or that the difference doesn't matter.
I think that LLM's are a broken mirror or maybe disco ball. What you get back is almost always a reflection of what you put in, just garbled through and with the reflections of so many other people. The best parts that I often get back are mostly just the stuff that I already put in it.
I think that's also where LLMs get the most dangerous. That ELIZA effect of telling you exactly what you told it but in a slightly rephrased way makes you think that you think that your ideas are genius. You've got someone "actively listening" and telling you whatever idea you just gave it was the best idea it ever heard.
That's not what you want from a serious writing partner. You want a writing partner to challenge you, to point out your mistakes and flaws, to make you better, to offer a truly different perspective to your own. You don't want a dark reflection from a "yes man" to spiral into.
I think it is a net positive for me, because I am super unconfident by nature and the agreeable, almost sycophantic output from major LLMs give me just enough of a boost to debias me. They are certainly better than a physical rubber duck.
I feel it is a net negative for me. One of my hobbies has always been fiction writing. The LLM reflections of my own writing become slot machines to me for cheap dopamine hits. It's not because LLMs are good at fiction writing, far from it, it's that LLMs are good at reflecting my own writing back to me in fun house mirror ways that amuse me and frustrate me and pleasantly surprise me.
That slot machine addiction gives me a lot of perspective on what LLMs are good at in coding versus what they appear to be good at. It's easy to characterize them as "over-eager junior developers" or "powerful rubber duck debuggers" but their biases are still your biases so long as you are the one writing the prompts. A lot of the "surprises" aren't intelligence but fun house mirror versions of the same biases you input into the system.
I keep referring to their coding capabilities as "legacy code as a service" and the addiction to the fictional equivalent only underscores that understanding of LLMs. You don't write the next great American novel getting a word salad of past great novels back full of the best and worst of your own writing. You don't write the next great American software getting a legacy code jumble of past great software back full of the best and worst of your own prompting.
At the end of the day I'm still a pragmatist and using the LLMs in my programming efforts, but I don't trust them as a partner, I use them as a tool. (A biased tool that I know needs a careful eye.)
> but their biases are still your biases so long as you are the one writing the prompts.
That is if you only work with your LLMs / Agents in a "straight ahead" manner, versus using knowledge of the bias baked into AIs and purposefully invoking a specific type of subject matter expert for problems that require specialized knowledge. It is similar to using people skilled at what you need, invoke through the vocabulary and language of your prompts specialized skills that are not tapped until using the terminology only those specialists use, and like magic their skills are suddenly in-context and the AI acts accordingly.
Don't just focus on the model, half the turns in a session belong to the human, it's an instrument that depends on how well the user wields it. It does not need to produce the novel insights, just to support the user to have them.
Thank you internet stranger, for introducing me to hard-maths drugs; am hooked!! \o/
I love the idea of this. So the BB problems are individual iterations of the halting problem right? To truly solve the problem one would have to come up with a program which would operate on all possible BB numbers?
Free labor enables capitalism, especially if you consider labor arbitrage as a mixture of free labor and properly compensated (according to the real value) labor. From literally being born, to family culture, education, and whatever level of broad social cohesion, it’s all free labor. Without that background, money itself loses its value, since an individual cannot have reasonable confidence in trading it for something of actual tangible value. It is abstract stored value, banked into society for free. Indeed, in many cases, the free labor is in the rational self interest of a group. But stability and love and peace aren’t monetized to their true value. Otherwise, markets should be much less stable. Bubbles are only notable for the large impact of a small group of bad actors. Overall, it’s pretty amazing what free labor does. Open source is just another instance of this long and critical tradition.
Free labor is derivative to incentivized labor. Your statement here doesn’t disprove or counter what I said. Again, follow the money trail. Everything you said if you follow the origin of the money it comes from paid, incentivized labor. Parents need money to raise kids… where do they get that money?? Our economy is called capitalism for a reason there is literally zero reference to charity or altruism in the vocabulary or even standard models that describe our economy and economic theory.
Put it another away: if we removed your ability to do incentivized labor and all you can do is charity work… you would run out of money and die from starvation. If we did the opposite and we removed your ability to do charity work… you’d be fine.
All of this re-emphasizes the point of this thread: In our objective reality, the world is driven by incentive based work while altruism is a side effect of surplus wealth generated by incentive based work. That is the fundamental reality.
I know this is such a late reply, but for clarity: my foundational point is the exact opposite hypothesis. Free labor enables incentivized labor. Economic theory has a neat concept of externalities, a necessary mechanism that only that which can be valued can be traded. If we removed all economic activity, then after most people die, the remnants would revert to much earlier models of free cooperation existing in small communities. Our world does require huge numbers of people to forcibly cooperate to create resources to then allocate.
My one other point is that incentivized labor is not the same as the value it creates. Indeed, it must be less. Otherwise, our economic system could support only a fixed number of people (subsistence), which would decay inevitably because there is no margin for error. But my point is that margin in reality isn’t fully realized, even by trillionaires, because then there would be no growth to support more people growing in their standard of living. There must be slack in this distributed system and the slack wasn’t valued: it’s free labor. It’s mixed in with incentivized labor, so I understand if you reject the premise entirely, but I do believe this is the essence of modern (specialized) capitalism. If skilled workers try to optimize or invent, more resources will be available for distribution for the same incentive (i.e. “worker productivity”). You can say “yeah that’s their job,” and I can say “that productivity wasn’t fully monetized because otherwise productivity would be lower overall.”
So, incentivized labor presupposes free labor, and economic productivity is a mix of free and monetized labor.
Interesting post, but the last bit of logic pointing to the Neural Engine for MLX doesn’t hold up. MLX supports running on CPU, Apple GPU via Metal, and NVIDIA GPU via CUDA: https://github.com/ml-explore/mlx/tree/main/mlx/backend
More specific info on the reference card is available in the paper's supplemental information. https://ieeexplore.ieee.org/ielx8/83/10795784/11125864/supp1.... Basically, they used special paper with a pro-Canon inkjet, along with a special ICC color profile.
Verification is indeed the majority of the time spent. Unlike programming, Verilog and VHDL and higher level things like Chisel aren’t executed serially by the hardware they describe like a von Neumann machine. Hello World for a chip isn’t designing the circuit, or simulating the circuit, or synthesizing the circuit to some set of physical primitives. No, it’s proving that the circuit will behave correctly under a bunch of different conditions. The less commoditized the product, the more important it is to know the real PDK, the real standard cell performance, what to really trust from the foundry, etc. Most of the algorithms to assist in this process are proprietary and locked behind NDAs. The open source tools are decades behind the commercial ones in both speed and correctness, despite heavy investment from companies like Google.
And so my point: the place where people best know how to make chips competitively in a cutthroat industry is NOT in schools, but in private companies that have signed all the NDAs. The information is literally locked away, unable to diffuse into the open where universities efficiently operate. Professors cannot teach what they don’t know or cannot legally share.
Chip design is a journeyman industry. Building fault-tolerant, fast, power-efficient, correct, debuggable, and manufacturable designs is table stakes. Because if not, there are already a ton of chip varieties available. Don’t reinvent the wheel because the intersection of logic, supply chain logistics, circuit design, large scale multi objective optimization, chemistry, physics, materials science, and mathematical verification is unforgiving.
There is some recent work [0] that explores this idea, scaling up n-gram models substantially while using word2vec vectors to understand similarity. Used to compute something the authors call the Creativity Index [1].
