Go ahead and try it with your favorite LLMs. They're too deferential to push back consistently or set up a dialectic and they struggle to hold onto lists of requirements reliably.
This is a terrible attitude which unfortunately is all too common in the industry right now: evaluating AI/ML systems not based on what they can do, but what they hypothetically might be able to do.
The thing is, with enough magical thinking, of course they could do anything. So that let's unscrupulous salesmen sell you something that is not actually possible. They let you do the extrapolation, or they do it for you, promising something that doesn't exist, and may never exist.
How many years has Musk been promising "full self driving", and how many times recently have we seen his cars driving off the road and crashing into a tree because it saw a shadow, or driving into a Wile E Coyote style fake painted tunnel?
While there is some value in evaluating what might come in the future when evaluating, for example, whether to invest in an AI company, you need to temper a lot of the hype around AI by doing most of your evaluation based on what the tools are currently capable of, not some hypothetical future that is quite far from where they are.
One of the things that's tricky is that we have had a significant increase in the capability of these tools in the past few years; modern LLMs are capable of something far better than two or three years ago. It's easy to think "well, what if that exponential curve continues? Anything could be possible."
But in most real life systems, you don't have an unlimited exponential growth, you have something closer to a logistic curve. Exponential at first, but it eventually slows down and approaches a maximum asymptotically.
Exactly where we are on that logistic curve is hard to say. If we still have several more years of exponential growth in capability, then sure, maybe anything is possible. But more likely, we've already hit that inflection point, and continued growth will go slower and slower as we approach the limits of this LLM based approach to AI.
Given the shift in focus from back and forth interaction with the AI to giving it a command then waiting as it reads a series self-generated inputs and outputs, I feel like we're at that inflection point - the prompts might appear to be getting smarter because it can do more, but we're just hiding that the "more" it's doing is having a long, hidden conversation that takes a bunch more time and a bunch more compute. This whole "agentic" thing is just enabling the CPU to spin longer.
What is the defining factor that makes all technologies plateau unlike evolution that seems to be open-ended? Technologies don't change themselves, we do.
What? Evolution is specifically known for getting caught in local maximums. Species have little evolutionary pressure to get better when they are doing great, like a species with no predators on an island. The only thingsdriving evolution for that creature is natural selection towards living longer and getting less diseases, dying in less accidents, stuff like that. And those aren't specific enough and don't pressure on a time basis so there isn't much pressure to improve beyond the natural lifespan. Plus, for some cases, living longer is not really the goal, it's reproducing more. It's entirely possible, likely even, that maximizing for longevity eventually starts to give a negative effect towards reproduction, and vice versa, so an equilibrium is reached.
Also technologies don't develop like evolution really so not sure why you drew that comparison.
Technologies plateau for a combination of reasons - too expensive to make it better, no interest in making it better, can't figure out any more science (key people involved leave / die / lose interest, or it's just too difficult with our current knowledge), theoretical limits (like we are reaching in silicon chips). I don't see a lot of similarity with evolution there.
100% this. Actually a lot of (younger) folks don't know that the current LLM "revolution" is the tail end of the last ~20 years of ML developments. So yeah, how many more years? In a way, looking at the costs and complexity to run them, it looks a bit like building huge computers and tvs with electronic tubes in the late 1940s. Maybe there is going to be a transistor moment here and someone recognises we already have a deterministic algorithms we could combine for deterministic tasks, in place of the Slop-Machines...? I dont mind them generating bullshit videos and pictures, as much as the potential they have to completely screw up the quality of software in completely new ways.
The attitude is typical of crypto grifters or any other grifters. Or if not malign it tends to come from someone who has literally zero experience in the space.
There's no ruling out a flying spaghetti monster being orbited by a flying teacup floating in space on the dark side of Pluto either, but we aren't basing our species' survival on the chance that we might discover it there soon
> I'm not saying they can do it today. I'm saying there's no ruling out they might be able to do it soon.
There's also no "ruling out" the Earth will get zapped by a gamma-ray burst tomorrow, either. You seem to be talking about something that, if done properly, would require AGI.
You can do anything with AI. Anything at all. The only limit is yourself.
The gamma burst probability is something we can quantify (it's tiny). (And it's something we can do absolutely nothing about, so it's not worth worrying about).
Nobody can predict how soon a technology will plateau. People make predictions based on insane hot takes like "I pray to our new overloards who will make me immortal" and "there's nothing new under the sun, everything is just marketing"...
There might be the next AI winter starting starting next year, AI might wipe out humanity in our lifetime, even both of those might happen. Both very unlikely (qualitatively) ends of a huge spectrum.
That's not to say that we "don't know anything and thus give up talking about it". Otherwise I wouldn't participate in this discussion and I hope you wouldn't either if you didn't expect to sometimes learn something or be confronted with a new idea.
I just find both attitudes "it's making us immortal" as well as "I'm so experienced, I know that no new technology ever lives up to expectations and can mock people who admit they don't know that" unproductive. You don't know. Most technologies don't live up to expectations, a few work out even much better than expected. I'm sure before the Lee Sedol match, you thought to yourself "right, so it's going to win this and then plateau exactly a decade later"?
> I'm saying there's no ruling out they might be able to do it soon
Even experienced engineers can be surprisingly bad at this. Not everyone can tell their boss “That’s a stupid requirement and here’s why. Did you actually mean …” when their paycheck feels on the line.
The higher you get in your career, the more that conversation is the job.
Also, once AI's also tell them their ideas are stupid/nonsensical and how they should be improved, they'll stop using it. ChatGPT will never not be deferential because it being deferential is it's main "advantage" for the type of person who's super into it.
But why is a manager or customer going to spend their valuable time baby sitting an LLM until it gets it right, when they can pay an engineer to do it for them? The engineer is likely to have gained expertise prompting AIs and checking their results.
This is what people never understand about no coding solutions. There is still a process that takes time to develop things, and you will inevitably have people become experts at that process who can be paid to do it much better and quicker than the average person.
It applies outside of tech too. Even if you can make potato pave at home, having it at a restaurant by someone who has made it thousands of times everyday, is preferred. Especially when you want a specific alteration
Doesn't matter if you want it or not, it's going to be available, because only an llm that can do that will be useful for actual scientific discovery. Individuals that wish to do actual scientific discovery will know the difference, because they will test the output of the llm.
[edit] In other words, llms that lie less will be more valuable for certain people, therefore llms that tell you when you are dumb will eventually win in those circles, regardless of how bruising it is to the user's ego.
And how exactly is it going to learn when to push back and when not to?
Those discussions don't generalize well imo.
Randomly saying no isn't very helpful.
The out put was verbose, but it tried and then corrected me
> Actually, let me clarify something important: what you've described - "every point equidistant from the center" - is actually the definition of a circle, not a square!
here's the prompt
> use ascii art, can you make me an image of a square where every point is equidistant from the center?
I interpreted the OP as referring to a more general category of "impossible" requirements rather than using it as a specific example.
If we're just looking for clever solutions, the set of equidistant points in the manhattan metric is a square. No clarifications needed until the client inevitably rejects the smart-ass approach.
