Rendered at 09:18:40 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
sothatsit 13 hours ago [-]
People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results.
The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? How about experiment driven domains like biology?
tyre 13 hours ago [-]
We will get much better at manipulation and better at people “writing” things to justify their own feelings.
What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.)
Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them.
Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)
My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.
red75prime 24 minutes ago [-]
I believe that the decentralization of manipulation (taken in a very broad sense as an effort to modify people's views) spells trouble for democracies. The mass media of old, with all its flaws, created a shared pool of information managed by well-educated people, some of whom understood that they had the power to keep democracy running (keeping populists away from the "manipulation machine," curbing blatant manipulation attempts, and so on).
Decentralized manipulation, by contrast, just runs amok creating echochambers and polarization.
plif 11 hours ago [-]
Not new about LLMs. Targeted ads / big data is this.
Another degree of capability, yes. But we have been trending here for a long time.
swedishagentic 8 hours ago [-]
I'm surprised no one has mentioned Cambridge Analytica.
That's like saying a sling is the same as an assault rifle. Yes both are weapons but scale and capabilities matter.
hnlmorg 10 hours ago [-]
It’s unfortunate that you’re being downvoted because this comment is true. And it just goes to show how far we’ve sunk that people have forgotten the problems of ad tech in the age of LLMs.
tyre 8 hours ago [-]
You’re making a weird number of assumptions about people’s voting and extrapolating to assertions about Society.
Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them.
That’s where this is headed and people are not ready. I don’t think we could prepare them, anyway.
hnlmorg 2 hours ago [-]
I’m not making any assumptions.
> Ads are not really the same. They can’t be as tightly targeted to what resonates with someone.
The EU referendum in the UK proved your point false.
> Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them.
When people talk about “ad tech”, they’re not talking about TV ;)
And yes, people can and do engage with them. That’s how ads on social media works.
During the EU referendum, people were even resharing ads on Facebook without even realising they were ads.
10 hours ago [-]
wombatpm 9 hours ago [-]
Biology would greatly benefit. We barely understand transcription and protein structure. And the straightforward systems that we know like insulin have complex post translational modifications. So while we have a map of the partial proteonome, we have barely scratched the surface on networks regulation and interactions.
aswegs8 2 hours ago [-]
I think the question is less about where it's most beneficial but which knowledge structure lends itself to LLMs most. Since biology's "language" is way more complex and irregular than maths or natural language, it isn't particularly accessible.
dnautics 2 hours ago [-]
> And the straightforward systems that we know like insulin have complex post translational modifications.
insulin is not straightforward, the way the insulin molecule interacts with its receptor is nuts. on the other hand its post translational modifications are simple and dont have anything particularly surprising (no glycoslation, disulfide bonds where you would expect, nothing special kex2 cuts, arent really defective in disease states even)
richardfey 3 hours ago [-]
> Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)
Side note: it's manipulation either ways because you chose what to talk about, with a goal in mind.
attila-lendvai 9 hours ago [-]
the fact that the useless left/right divide is still so widely used shows that manipulation is working well even pre LLMs...
when it comes to the important question, then both "sides" are the same team.
or if you want it with a pinch of humor:
when a boot is on your face, it makes precious little difference whether it's the left or the right boot.
(i lived the first 10 years of my life in communism)
tyre 8 hours ago [-]
If you think that the American Left is anything like communism, I’m sorry, but, respectfully, I can’t take your comments seriously.
The left in the US would be center-right in Europe, who are certainly not communist (they have separate parties that are communists!)
Even the socialist strain of the US has nothing to do with socialism scaremongering about Venezuela, etc.
throwthrowuknow 39 minutes ago [-]
Excellent job not understanding the point. The party thanks you for your service.
aswegs8 2 hours ago [-]
Not at all. Economically, yes. Socially they are far left of the European left.
Levitz 6 hours ago [-]
The "left" as in the party, sure, not that much of an authoritarian leaning.
The "left" as in the pervasive group that crawled out of Tumblr, took hold of Twitter back in the day, and has a stronghold on Reddit now? Those do care about what you can say, think, watch and read, and the more they can control, the better. The US right can only dream to have half as much control as the left has had in the last three decades.
senderista 5 hours ago [-]
The left has cultural power, the right has political power.
sedivy94 8 hours ago [-]
The parent comment wasn’t making that comparison. In fact, quite the opposite. The communism comment emphasized personal experience with a boot in one’s face, not that the boot was communist.
mensetmanusman 7 hours ago [-]
What even is the left though in the US? I haven’t watched any TV in decades, and the online content is entirely algo driven, so I don’t know what is happening.
NitpickLawyer 5 hours ago [-]
> nothing to do with socialism scaremongering
What's "scaremongering" to you is "life" to GP. Besides missing their point, you're also saying "you held socialism wrong, we can make it work, if only if it weren't for this pesky ... reality"... Sorry, you missed their comment, and I can't take yours seriously :)
lettergram 13 hours ago [-]
> My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.
I think that statement may itself highlight how prevalent manipulation is.
I fully anticipate all groups to continue maximal manipulation they can. One thing with LLMs is that it'll be a far less unified view, so a "divide and conquer" strategy is what I anticipate.
tyre 8 hours ago [-]
In US politics, the right is far, far better at winning elections than the left. This isn’t about personal preference. It’s objective political science.
Look at the most contentious issues in the US: abortion, climate change, taxing the wealthy, gun control, Affordable Healthcare Act.
The Democratic Party platform is aligned with national polling on every one. Every one of those issues has >60% support with voters and the Republican Party has blocked them all.
They play the game to win. And they do.
armchairhacker 2 hours ago [-]
> Look at the most contentious issues in the US: abortion, climate change, taxing the wealthy, gun control, Affordable Healthcare Act.
> Democratic Party platform is aligned with national polling on every one.
It depends on the pollster and where you're polling. I guarantee you rural Tennessee will not agree with downtown Washington DC on any of these issues. In contrast, rural California will likely agree with rural Tennessee. It's not as cut and dry as a homogeneous national poll of 2500 people. Every state, city, county are different. That's why there are federal, state, city, and county governments.
For instance, Abortions are legal nationally. States can individually decide how, or if, they wish to restrict it. This is as the constitution intends under the 10th amendment:
> powers not delegated to the federal government nor prohibited to the states are reserved to the states or the people.
This allows for democracy to take place at the local level, rather than having particular regions thousands of miles away from each other ultimately oppress the other.
To the point on LLMs, I think it's abundantly clear they will be used to propagandize and similar to social media will lock people in a bubble without alternative opinions. It'll be the worst of both worlds, the question is who's the puppet master. At some point soon, I imagine it'll be the AI.
scns 11 hours ago [-]
It is way harder to manipulate people to do the right thing i think.
Levitz 6 hours ago [-]
It's harder to convince people to all agree on the same, different from the norm, thing.
If you've got 5 people in a car and you play ABBA in every road trip, then one day you suggest to change, the problem is not in being okay with "something else", but on agreeing what that other thing should be, set against the already known thing.
That's why there's so much infighting in the left.
kortilla 11 hours ago [-]
That presupposes that the left in the US wants to do the right thing. Something like government run grocery stores is not clearly correct and there is very little evidence supporting that it will work well yet it is a very popular leftist policy in New York.
galleywest200 8 hours ago [-]
City run grocery stores are certainly not evil.
kortilla 4 hours ago [-]
Based on zero evidence. It’s very easy for a government to step in as a participant, ruin the profitability of a sector in an area, and offer a worse service.
They have no profit requirement or even revenue neutral requirement. So they can just operate poorly at a loss and still wreck other businesses because people will put up with breadlines to get bread for ultra cheap.
The general thing to watch out for with all of these “surely it can’t be evil to do nice thing X” is suicidal empathy. It can seem correct to your gut on the surface while it’s extremely destructive in the long term despite participants wanting to destroy something as an explicit goal.
0xWTF 8 hours ago [-]
... not overtly, intentionally evil.
FTFY
tyre 8 hours ago [-]
What’s the case that they’re secretly, unintentionally evil?
snakeboy 4 hours ago [-]
Bad economic policy is subtly "evil", by way of allocating finite resources inefficiently. Usually this is unintentionally done by not appropriately taking second, third, ..., nth order effects into account.
jdub 43 minutes ago [-]
... which would make (our current model of) capitalism especially evil. Compared to a supermarket.
kortilla 4 hours ago [-]
They destroy grocery stores that offer variety in every neighborhood they operate in. They offer worse service and people put up with it because the rest of New York is subsidizing them through taxes.
verisimi 18 minutes ago [-]
> We will get much better at manipulation and better at people “writing” things to justify their own feelings.
Yes, ai has strong narcissistic traits. And so do the people that own them. And pay for them.
In response, people in general will become fat more capable of recognising the manipulation. And will become more paranoid.
thisisnotauser 10 hours ago [-]
My wife is working on her PhD in microbiology now, using OpenAI to implement her research ideas. Genetics is just too much data, and she eats through tokens like nobody's business. I thought I was careless with them, but she barely lasts a full day before exhausting her quota. There's definitely a lot of value there, but dealing with the data problem is a big obstacle in biology. I can only imagine what she could get done with more capacity, though...
AgentMatt 10 hours ago [-]
What makes it so token hungry? Is she directly using the LLM for genome analysis rather then having it write the data analysis algos?
tossandthrow 2 hours ago [-]
We so don't know what plan she is using.
Eg.a 20usd/m plan usually don't cut it for professional work.
doc_ick 7 hours ago [-]
Likely not using / managing context windows properly and then having it re-read data it’s already gone through
ed_elliott_asc 10 minutes ago [-]
This absolutely terrifies me, surely one misread piece of data or a hallucination here or there and a little “oh sorry about that, I guessed at this portion of the data to save time” and the data used is useless?
cmdli 12 hours ago [-]
This reminds me a lot of the proof by construction for the 4-color theorem. It was only enabled by the advancement of computers and dissatisfied many of the computer scientists and mathematicians since it was a "brute force" approach.
I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).
throwaway27448 12 hours ago [-]
Sigmoidal, not exponential. It would be insane to assume an exponential curve
energy123 3 hours ago [-]
That's a false binary. The right binary is whether it's a convergent function or a divergent function. A sigmoid is convergent, which is not well substantiated and more farfetched than a divergent function, even if it's true that 2^x is too optimistic.
The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf
conformist 11 hours ago [-]
Isn’t the point more “it’s easy to fall into the trap to believe that predicting when the sigmoid is going to bend is possible and the right heuristic is to instead extrapolate locally”?
That aside, I’d question whether applying the Lindy effect in particular to something that’s not really a life expectancy but more a growth rate is credible… or perhaps a bit circular since it “assumes away” the ceiling.
throwaway27448 11 hours ago [-]
Nobody in this thread is trying to predict when the sigmoid is going to bend. Perhaps they should
aurareturn 7 hours ago [-]
A lot of HN posters thought it was already bending at GPT4o/GPT4.5. Turns out, it kept accelerating (at agentic tasks).
yazaddaruvala 10 hours ago [-]
Predicting when the sigmoid bends is difficult and predicting how long until it unbends is equally difficult.
The simpler assumption is that over enough time, the S functions stack together for long enough that working backwards from exponential is a better predictor of reality.
These stacked S curves have continually been true with most technology.
haldujai 9 hours ago [-]
It hasn’t bent already? 2022-2024 certainly seemed far more exponential than 2024 to present.
NiloCK 9 hours ago [-]
I find this astounding. 2024 to present thread is can write a coherent 15 line function to ... what exactly?
No future for research mathematicians othet than as tastemakers / agenda setters?
annzabelle 11 hours ago [-]
The author of that post is a prominent Bay Area "rationalist," who have had a quasi-theistic relationship with the concept of all-powerful AIs for a couple decades now.
mitthrowaway2 8 hours ago [-]
So are they right or wrong about the sigmoids?
annzabelle 6 hours ago [-]
I dunno. They've been saying this for decades, including in a very well read Harry Potter fanfic, and I'd always dismissed them as kooks, but maybe they're right in the end.
vasco 2 hours ago [-]
All of the point of that article is that most people that think something is a sigmoid think it'll bend just as they are publishing their analysis. And the article says, don't do that, assume it'll be related to how long we've been on the "goes up" part.
Nothing in that article says it's not a sigmoid.
subygan 11 hours ago [-]
in the absence of a stalling signal. it's better to assume exponential and work backwards than hope the next bottleneck is impossible.
slashdave 3 hours ago [-]
Since when does realism become hope? (It's the other way around)
scarmig 10 hours ago [-]
Everything has limits, of course. But if you've not yet seen a deceleration, it's reasonable to expect that at the least you're in the middle of the sigmoid, not the top.
piloto_ciego 6 hours ago [-]
I mean, "sure, technically correct is the best kind of correct" - but where we sit on this right now? It certainly feels exponential.
A lot of this sort these sorts of posts are just "appeals to geometry" (aka "cope"). This is coming, it's coming hard. Now you need to decide what you want to do with your life in a world where your smarts aren't as special as they used to be.
This is hard (believe me, I know). But what one ought do is not eschew progress and cling to the delusion that things don't change, what one ought to do is try to see how they can leverage these tools for greater and greater accomplishments.
12 hours ago [-]
slashdave 3 hours ago [-]
Shh! Don't upset the hype train
chatmasta 5 hours ago [-]
I expect we can squeeze a lot more exponential out of LLMs because they’ve basically shown that human “consciousness,” insofar as it’s composed of knowledge and rules for synthesizing that knowledge, can be represented mathematically in a very high dimensional space. Why does this “just work?” Nobody really knows, but it clearly does.
However, I also expect this squeeze will come at an increasingly expensive price — not just because of inefficient token usage, but because of fundamental limitations of LLMs as a model.
LLMs are letting us brute force our way through a lot of reasoning, but it’s hard to believe that such a generic model of intelligence will take us to the next frontier. We’ll need some fundamentally new approaches at some point. Maybe those will make achieving the exponential more efficient or maybe they’ll unlock even higher degrees of possibility. Who knows?
visarga 4 hours ago [-]
> Why does this “just work?” Nobody really knows, but it clearly does.
We know language has to be learnable by every human, so it needs to be really independent of any specific brain development particularities. If it was not accessible to babies there would be no more language next generation.
andai 5 hours ago [-]
The transformer is Turing complete. It might be a tarpit though? I don't know.
I think a nice example is using them for arithmetic. It's a specialized deterministic process, so it's extremely wasteful to do it that way.
But they're good at finding solutions to things we don't know how to specialize yet.
So, to use metaphor, maybe the transformer-based models are like the FPGA, and then when we figure out the patterns in that system — all the different kinds of specialized reasoning — we can extract it into an ASIC?
totetsu 8 hours ago [-]
I think you’re making a category error in your definition of politics here.
Certainly technologies can favour winners and losers, but the struggle is an inherently human one.
andsoitis 4 hours ago [-]
> exponential
Many many things are only useful when expressed in the physical world, and that introduces lag.
9 hours ago [-]
porridgeraisin 13 hours ago [-]
Today's models depend on inference time compute to get these results. The inference time compute available on any claude subscription is not comparable to the ones used to get some of these results (yes, in this case, it is 2000 USD total as noam confirmed, but some previous results took more).
In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some.
In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly.
In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth).
> will we need mathematicians to translate
Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal.
For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude.
> But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years.
For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not.
Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.
curt15 11 hours ago [-]
Another interesting question is why the frontier labs are piling on pure maths, which has little direct economic value compared to something like law or improving the efficiency of their own models? How much OpenAI and Anthropic are paying to serve these models for ordinary users is the elephant in the room. A cynical take is that the frontier labs are trying their best to pump up their pre-IPO valuation through flashy headlines.
anon373839 9 hours ago [-]
> Another interesting question is why the frontier labs are piling on pure maths
The reason is that the original scaling axes (parameters, training tokens, test-time compute) have saturated already, but RLVR (reinforcement learning from verifiable rewards) is still scaling well. And math has this nice property where you can synthetically generate arbitrary volumes of rewards to train the model, because math is self-contained and completely objective. Open-ended reasoning and analysis don't have that convenient property, and that is why progress is much slower outside of math and coding.
danielmarkbruce 11 hours ago [-]
It's one of the few areas where you can verify results. That fits nicely into training models. They aren't just making judgement calls on what would be nice, it's "what can we do?".
beering 10 hours ago [-]
If everyone publicly said that the models can only do things that humans have already done, but you know they can do more, wouldn’t you want to show them otherwise?
Math ability also helps with other things like making models more efficient.
gpm 11 hours ago [-]
Because it's a tool in search of a use case (or many use cases) and mathematics is the most natural use case for it. Mathematics is by definition the art of putting words on a page in a rigorously defined "correct manner" (i.e. in the form of a valid logical argument, a proof) and all LLMs do is put words on pages and evaluating if they're good words is by far easiest when there is a strict definition of right and wrong.
qingcharles 8 hours ago [-]
What's the best way to apply it to legal problems? Finding bugs in statutes? (there are often statutes with wording errors, missing negatives, things like that which don't get picked up for ages)
dominotw 13 hours ago [-]
> The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn,
isn't it clearly split between verifiable not verifiable ? what is interesting about that question.
sothatsit 12 hours ago [-]
I do not think it is so clear.
Programming has verifiable and non-verifiable aspects. Competitive programming, passing tests, and performance can all be verified. But translating English requirements into actual software, software architecture, taste, or UI design cannot. And yet over the last couple years we’ve seen huge lifts in all of these areas, not just the verifiable ones.
Verifiable areas I think are clearly seeing the most improvement, or are the quickest to see improvement. But we are seeing lots of progress in non-verifiable areas as well.
How much of the non-verifiable progress is a function of labs purchasing expert data vs. the models improving with compute is maybe another interesting question, but fundamentally I don’t see spend on expert data as something that can’t grow if AI revenues keep growing as well. And as models get better taste they can also help filter and generate new synthetic data for their next versions to train on. The limits of this approach are not so clear.
porridgeraisin 12 hours ago [-]
That is because there is human annotated data there. Every session you or I used, then of course paid human feedback on repos (such as the recently famous example of meta forcing their employees to).
This is _much better_ data than 1/0 verification, it is as good as a gradient.
Automatically verifiable tasks improve faster since well, its automated.
dominotw 12 hours ago [-]
> we’ve seen huge lifts in all of these areas, not just the verifiable ones.
most gains are still coming from data. isnt that supposed to 'run out' though?
sothatsit 12 hours ago [-]
Labs spend billions hiring experts to generate new data, and better models can better filter existing training data and generate new synthetic data. There’s no reason for that to run out, it’s just expensive.
You could view this as just continually patching a leaky ship. But it seems to work.
12 hours ago [-]
casey2 10 hours ago [-]
We definitely are not on an exponential. Don't say we are because this isn't up for debate. AI progress is logarithmic the million dollar question is 2x or 10x for linear improvement. The nearest qualitative shift would be very fast inference so people could start writing real software on top of LLMs. A 0.001% optimization on a packing problem just isn't interesting for the amount of investment.
energy123 2 hours ago [-]
What is the y-axis in this claim about logarithmic improvements? Any exponential curve can be trivially turned into a logarithmic and vice-versa, and the y-axis redefined as "progress", with no loss of accuracy.
One example that always bugs me is when people point to "exponential" or "sigmoidal" progress on benchmarks. Benchmarks are artificial constructions (saturation at 100% by definition) and benchmark scores should not be mapped to these words when talking about overall progress.
Example - progress on ARC-AGI-3 at the moment is exponential, steeper than 2^t and e^t. Does that mean AI is progressing "exponentially" in the colloquial sense? No, it doesn't support or refute that colloquialism.
Likewise with MMLU saturation. We can't go above 100% by construction. Therefore we have a "sigmoid". Gah.
The colloquialism is not helpful to begin with.
akoboldfrying 8 hours ago [-]
> A 0.001% optimization on a packing problem just isn't interesting for the amount of investment.
I think you have completely misunderstood what OpenAI have accomplished here. Almost certainly no one cares about the specific concrete results achieved; they only care about (a) how difficult it would be for an intelligent human to achieve the same feat (ETA: the feat is the proof), which can be estimated by the amount of time the problem has remained open/a public conjecture, and (b) how general this artificial "intelligence" appears to be, which can be estimated by the diversity of topics where it was able to prove a difficult result.
It's as if I showed you a dog that I had taught to speak German fluently, and you remarked: "What point is a dog that can speak a language that less than 2% of the world speaks? Nothing to see here."
koe123 53 minutes ago [-]
On the other hand, if provided the financial incentive would mathematicians have solved these problems? Its not hard to imagine a world where some hard problems were not selected by the sparse experts for whatever reason (lack of interest, whatever), which could have been solved if someone was throwing down millions for solutions.
andsoitis 4 hours ago [-]
> It's as if I showed you a dog that I had taught to speak German fluently
And you’re thinking this is an accurate comparison?
vasco 2 hours ago [-]
It's more funny than the dog doing differential equations I'll tell you that. Specially when it gets mad.
dominotw 13 hours ago [-]
> but I’ve noticed Fable to be quite a big step up there
what did you notice ?
J_Shelby_J 12 hours ago [-]
I've noticed that out of all LLMs I've ever used that Fable is the MOST LLM; the text it produces is abomination. It's impressive how much I hate it. It is such an awful writer - it assumes the reader has zero context and therefore gives every single bit of context and detail - which is nice if you're writing a legal document I suppose. But it uses, niche, $10 words to describe every facet of everything it's discussing. I had it re-write some docs and I ended up rewriting 1k lines of of Fable torment nexus text to around 100. Because guess what, someone reading highly technical docs has a knowledge base that allows us to compress the topic into a much tighter representation.
alasano 11 hours ago [-]
Just praised Fable in another comment but what you're saying is also insanely true.
I literally roll my eyes and cringe quite often at its output pretty much daily.
I don't like to overload my sessions with skills but I've been using a "write-normal" skill I made just to have it rewrite outputs that particularly piss me off.
I'm sure there's a million of these skills out there, but this one is tailored to the stuff that makes me mad in particular.
sothatsit 12 hours ago [-]
Fable is much better at handling nuance. Opus/GPT 5.6 Sol are much more likely to miss the point you are trying to make, emphasise the wrong thing, exaggerate the importance of unimportant details, or introduce contradictions.
That said, Fable is still not a great writer, largely driven by it not knowing what it should exclude, and it still having the usual LLM-isms. But it’s better.
alasano 11 hours ago [-]
That's really what got everyone hooked in the first place.
5.6 Sol is great but there's a depth to the understanding that Fable exhibits that's unique to it currently.
Can I truly quantify this? I don't think so. Just that I spend a ton of time with various models and a certain point it's just a personal impression or a gut feeling.
In the days after Fable first came out I increased the amount of parallel planning of tasks that I was doing by 2-3x because it felt like I didn't need to be paranoid due to that handling of nuance.
viccis 13 hours ago [-]
>Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation?
Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?
koe123 50 minutes ago [-]
In fact the opposite, look at talks by Peter Thiel. At least he’s being honest. These people are bastards, but for some reason moral goodness and wealth has been conflated leading us to venerate greed.
watutalkinbout 12 hours ago [-]
It's like Musks duplicitous argument about unlimited abundance. We have a lot of abundance now, we just keep accelerating it all into the hands of fewer and fewer people - whose response is only to want more, and more, and more.
hackinthebochs 10 hours ago [-]
Yeah, we really should just storm the facility where Musk is hoarding all the worlds bread and meat.
Wealth in terms of capital doesn't represent material goods, it represents the system's confidence in your ability to direct capital efficiently. But eventually efficient capital bottoms out at consumable goods. Someone like Musk with a lot of capital under his control is contributing to the end goal of unlimited abundance.
jdub 29 minutes ago [-]
With his shitty cars, orbital garbage, or CSAM generator? Or his moneyed attacks on democracy and communications? Or his vandalism of government programs without insight, experience, or qualification?
In a free and fair market, his capital would be regarded as a deeply inefficient distortion.
jcims 13 hours ago [-]
>Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation?
Yes.
mmcnl 12 hours ago [-]
There are many math problems that are simply puzzles: intellectually interesting but nothing worth of value depends on it. To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that.
The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.
xabush 9 hours ago [-]
"To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that."
I was recently listening to BBC Radio 4's episode on the Poincare Conjecture[1] and the guests on the program were discussing how the problem that looked deceptively simple eluded the great mathematicians of the time (including Poincare himself) for nearly a century and how Grigori Perelman cleverly came up with the proof. It took other mathematicians working in groups years after Perelman's publication to understand and validate his proof. The mathematicians on the program were speaking of highly of his proofs and admiring the originality of his work. This made me think of one neat experiment where if we cut-off a frontier model's training data 2002 or anytime before Perelman posted his proofs on arXiv and check if it can come up with the solution by itself. That would surely be a great signal to see if these LLMs aren't just solving interesting puzzles and that they can came up with something truly novel.
P.S I highly recommend Misha Green's "Perfect Rigor" for anyone interested in the history of the problem and the genius behind the proofs of the conjecture - Perleman. I found it an entertaining read and could digest its description of the problem as a layperson (with undergrad level math).
Replace philosophers for mathematicians and Douglas Adams was spot on again.
Whilst current models can't 'intuit' and come up with conjectures, they can certainly disprove some of them very quickly through the kind of grind that humans can't do. I suppose there really are some mathematicians out there today, whose last few years of study, have just been up-ended by this.
--
"Yes we are," insisted Majikthise. "We are quite definitely here as representatives of the Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons, and we want this machine off, and we want it off now!"
"What's the problem?" said Lunkwill.
"I'll tell you what the problem is mate," said Majikthise, "demarcation, that's the problem!"
"We demand," yelled Vroomfondel, "that demarcation may or may not be the problem!"
"You just let the machines get on with the adding up," warned Majikthise, "and we'll take care of the eternal verities thank you very much. You want to check your legal position you do mate. Under law the Quest for Ultimate Truth is quite clearly the inalienable prerogative of your working thinkers. Any bloody machine goes and actually finds it and we're straight out of a job aren't we? I mean what's the use of our sitting up half the night arguing that there may or may not be a God if this machine only goes and gives us his bleeding phone number the next morning?"
MostlyStable 12 hours ago [-]
From a mathematician who was intimately familiar with some of these problems [0]
>I don’t understand it yet. Maybe it’ll take me an afternoon to check all the calculations, but what would still be missing is why this was an approach that would’ve made sense in the first place. Is there some broader context or theory within which this would’ve been the obvious thing to do? What other results can be proven using these techniques? What is it telling us about quantum information or operator theory? I have no idea. I spent about an hour this morning asking ChatGPT these questions, but it’s somewhat frustrating because it speaks with a mishmash of physicist, operator algebraist, quantum information theorist-lingo, plus the usual LLM breezy lilt that annoys everybody.
They certainly seem to have "intuited", in a way that is not immediately obvious to experts in the field, the way to solve at least some of these problems. This was not just simply grinding away at a method that humans already knew would work and just hadn't gotten to yet.
I think it might be like waveform collapse, but very high dimensional.
taaha47 33 minutes ago [-]
can you expand on this?
DiscourseFan 3 hours ago [-]
Can you expand on this?
WarmWash 15 hours ago [-]
>Whilst current models can't 'intuit
That's how they are finding these solutions though, unless we are just going to label intuition as something only humans can do. Like a submarine being unable to swim or whatever that example is.
sdenton4 14 hours ago [-]
Some of them...
The two places were seeing lots of movement are:
* Updates to lower/upper bounds. In many cases, these kinds of problems are the deep-math equivalent of calculating more digits of pi. Yes, if you throw time at it you'll break the record, but it may not be terribly worthwhile.
* Finding counter examples which disprove conjectures. This is really useful, and helps offset some positivity bias on the human side, often bringing together known tools from distant silos.
If you read the list of ten results, almost all fall into one of these buckets.
pama 14 hours ago [-]
It is unfair to dismiss contributions to decades old open problems as equivalent to calculating more digits of pi. It missed the mark by a lot—as does the two bucket simplifaction.
sdenton4 11 hours ago [-]
Five (maybe six?) of the results are improvements on bounds. These kinds of problems tend to have some initial advances, and then stall out as the complexity of the bound skyrockets... until some grad student is bored enough to push the boundary. The big-O complexity of matrix multiplication is a good example of how this works: yeah, it's a useful problem, but the solutions are galactic algorithms, and increasingly convoluted.
As someone with a PhD in combinatorics, I believe that I'm qualified to say that, yes, there are problems as useless as calculating more digits of pi.
pama 4 hours ago [-]
Your inverted logic does not hold. The fact that such useless problems for bounds exist does not mean that improving bounds is useless. 9 fields medals in the last twenty years, including the one to Terrence Tao, were for improvements on bounds. 3 of the 4 medals in 2022 were for bounds; 2 of these medals were in combinatorics.
denismenace 13 hours ago [-]
How does calculating more digits of pi help us?
tuatoru 13 hours ago [-]
"It's just brute-forcing the search space."
buddhistdude 12 hours ago [-]
It can move to any place within the search space but it can't move outside of it and it can't move in between the 'pixels'. Human thought can, as human thought has created the search space.
robotpepi 14 hours ago [-]
it could also be that they try every possible approach that has been proposed by humans. it seems that was the case for the non sofic group example. humans are not able to do the same at that scale. it's unfortunate that we don't know what's happening behind the hood with these models, and that's a huge danger also for the rest of us without access to them.
rirze 14 hours ago [-]
> "matrices"
fasterik 14 hours ago [-]
Saying that AI is "matrices" is like saying human cognition is "neurons." Maybe true at some level, but it's a low-level implementation detail. The important part of a language model is the function that maps tokens to contextual embeddings. You could compute this function using analog computing, biological neurons, or any other substrate.
watutalkinbout 12 hours ago [-]
It isn't an implementation for neurons, unless you believe in a designing god.
Matrices are an implementation detail in reconstructing the surface of human knowledge. It's a complex surface, but it's a regurgitation.
aswegs8 2 hours ago [-]
What's the argument here? It's not about the implementation method, it is about the behavior that emerges from it. You could calculate the next token by hand on paper if you had enough time.
pama 15 hours ago [-]
> Whilst current models can't 'intuit' and come up with conjectures
I disagree. I routinely let LLMs speculate or generate hypotheses along the way of helping with technical research. Sometimes they can prove the correctness of a concrete math idea but other times even an unproven conjecture helps with the numerical algorithm implementation and the result is then simply supported by additional data. I guess that any autoresearch-adjacent application has LLMs intuiting and coming up with hypotheses/conjectures—as do the steps/lemmas along a complex proof. In my opinion the modern LLMs are powerful intuitive thinkers that generate lots of conjectures of varying quality or importance.
zahlman 13 hours ago [-]
> they can certainly disprove some of them very quickly through the kind of grind that humans can't do
Of course computers can grind in a way that humans can't. But now we have systems that convert the human-comprehensible ideas into a computer's plan of attack, in a way that greatly expands the frontier of ideas thus treatable.
jacquesm 12 hours ago [-]
Ahh, but you missed the continuation, where they get to the heart of the matter: money.
"Excuse me, We demand rigidly defined areas of doubt and uncertainty!"
DT: Might I make an observation at this point?
MT: You keep out of this metal nose.
VF: We demand that that machine not be allowed to think about this problem!
DT: If I might make an observation…
MT: We’ll go on strike!
VF: That’s right. You’ll have a national philosopher’s strike on your hands.
DT: Who will that inconvenience?
MT: Never you mind who it’ll inconvenience you box of black legging binary bits! It’ll hurt, buster! It’ll hurt!
DT: [Booming] If I might make an observation …
“All I wanted to say,” bellowed the computer, “is that my circuits are now irrevocably committed to calculating the answer to the Ultimate Question of Life, the Universe, and Everything.” He paused and satisfied himself that he now had everyone’s attention, before continuing more quietly. “But the program will take me a little while to run.”
Fook glanced impatiently at his watch.
“How long?” he said.
“Seven and a half million years,” said Deep Thought.
Lunkwill and Fook blinked at each other.
“Seven and a half million years!” they cried in chorus.
“Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I? And it occurs to me that running a program like this is bound to create an enormous amount of popular publicity for the whole are of philosophy in general. Everyone’s going to have their own theories about what answer I’m eventually going to come up with, and who better, to capitalize on that media market than you yourselves? So long as you can keep disagreeing with each other violently enough and maligning each other in the popular press, and so long as you have clever agents, you can keep yourselves on the gravy train for life. How does that sound?”
The two philosophers gaped at him.
“Bloody hell,” said Majikthise, “now that is what I call thinking. Here, Vroomfondel, why do we never think of things like that?”
“Dunno,” said Vroomfondel in an awed whisper; “think our brains must be too highly trained, Majikthise.”
So saying, they turned on their heels and walked out of the door and into a life-style beyond their wildest dreams.”
evenhash 15 hours ago [-]
> Whilst current models can't 'intuit' and come up with conjectures
People keep saying this. Why?
Surely the AI can complete the prompt “Generate new research questions based on these observations”?
When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.
5555watch 14 hours ago [-]
I like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.
Valakas_ 3 hours ago [-]
I have news for you. All humans do is also filling gaps with combinations of known facts and results - in new ways. "Everything is a Remix" is a good watch on youtube that explains this. Picasso might look like he has an invented personal style, but his style is a combination of different little details he took from others and mixed in a new way. Mozart the same. No music artist could ever create music in a vacuum. Everyone, for every art and science, the same. I know many are trying to cling to the last hope of human specialness, that "thing" that AI can never get to.
It's a convex hull of information that is reflective and spans outside of itself and combines in a new way, when you shine two known rays of light together from the inside.
Now it gets better. AI can be orders of magnitude more creative than any human could ever hope for, because his convex hull of information is orders of magnitude larger, and the possibilities for new combinations are equally larger.
corimaith 26 minutes ago [-]
But do you have vision?
bee_rider 12 hours ago [-]
Is that actually true though? I think it is an analogy, and as an analogy it seems quite risky because “convex hull” and “linear combination” are technical terms that might give the recipient the impression that it is a technical argument.
metanonsense 11 hours ago [-]
I think this is only "statistically" true in the sense that training is based on facts and not non-facts (except maybe with the ingestion of flat-earthers literature ;-). The existence of hallucinations in a bare transformer shows that the convex hull is not about information but about text, so the limit may more be "possible linear combinations of text", which allows for much extrapolation and counterfactuals. True creativity may be one reinforcement learning mid-training goal away that rewards novelty over correctness.
10 hours ago [-]
jiggawatts 12 hours ago [-]
Neural nets can extrapolate past their training data, and there is no reason to think LLMs don’t inherit this capability.
The extent to which they are able to do this is the more interesting question!
5555watch 12 hours ago [-]
The extrapolation can also be a learned skill, especially in math. How many papers took result X, extended it to Y using known building blocks, and applied to Z.
By the way, convex hull permits extrapolating past the training data. LLM won't invent a new word that could not be defined by a sequence of known words. Just if it's meaningless and fully random/hallucinated, the new knowledge won't work with other known information blocks (breaks convexity).
charlie90 9 hours ago [-]
Thats how humans work, as well.
tuvix 14 hours ago [-]
All arguments like this boil down to semantics at a certain point, but yes large language models can “intuit” because they can generalize between examples. The issue then becomes how you pack new examples into context.
Humans can “intuit” based on a much larger, if not unlimited, context. Also I just want to say that human cognition is something so insanely complex and deep that we will not understand it at all in my lifetime. To attribute all, or really any, aspects of human cognition to a machine at this point is silly to me.
michaelmrose 13 hours ago [-]
Define insanely complex and deep in a way that isn't illiterate hand waving.
Most humans are dumber than a box of rocks. Here in Seattle we had one of many light rail-related fuckups where they had to replace part of the line with buses. People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.
Chatgpt was smarter than the average person a while ago
tuvix 13 hours ago [-]
I’m not talking about the actions we take or how we might perform at certain tasks, I’m talking about how our brains actually work. My point is that we have no idea how I’m able to imagine an apple and see it in my mind’s eye. It’s basically biological magic to us at this point.
There are processes at work there that we don’t even have the language to describe.
zahlman 13 hours ago [-]
Not only that, but we do it with a processor that is basically required to operate in a narrow temperature band below 40C, using a mere 86 billion neurons (although the equivalence with either machine-learning "neurons" or LLM parameters is not at all clear) operating on a few dozen watts; and with this we operate many other systems besides language processing. It's not clear that our reasoning process requires language, either.
(86 billion is the number ChatGPT, ironically enough, has given me a couple of times. I remember hearing for a long time that it was estimated to be somewhere in the ballpark of 100 billion. This is not my field of study.)
zahlman 13 hours ago [-]
> People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.
This does not demonstrate a lack of intelligence. It demonstrates laziness and a lack of interest in spreading apart. Or just lack of consideration (or even malice) on the part of those at the back of the wad.
> Chatgpt was smarter than the average person a while ago
This is an absurd claim that fundamentally misunderstands what it means to be "smart". Reasoning that would get you to this conclusion would equally well apply to Google's search engine over a decade ago.
watutalkinbout 12 hours ago [-]
LLMs traverse an assembled surface of human knowledge.
You can't find things on a map that aren't there, but maybe you can draw a route nobody used before.
s1artibartfast 10 hours ago [-]
Because people have internalized an inaccurate model of LLMs as "stochastic parrots" that was incorrect at the time of formulation and is also significantly outdated
claytongulick 15 hours ago [-]
> People keep saying this. Why?
For the same reason that you can't draw a 15 of Diamonds from a regular card deck.
treis 15 hours ago [-]
Of course you can. Tape a 7 and 8 of diamonds together and boom 15 of diamonds
15 hours ago [-]
plaidfuji 11 hours ago [-]
Any computable problem will eventually fall to computers.
LLMs have made math proofs more computable, in the sense that a computer can both generate potential solutions and check the validity of its solutions on its own, with a reasonable chance of converging on something correct. I assume this was already doable to some extent, but it seems like it’s now exponentially easier. That still doesn’t mean that all math is automatically solved.
This is somewhat similar to things like molecular dynamics or protein folding or finite element simulations, etc. Some problems that were previously intractable via computation became tractable. Others - the vast majority of other problems - remain unsolvable by these computational techniques, because the scale of compute required is beyond imagination. These are simple things like simulating the dynamics of a cubic millimeter of water molecules for 1 second. Unfathomably beyond current capabilities (and LLMs aren’t going to change that).
I think LLMs are great, I use them every day and I think they have a ton of value. But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.
So there are limits - but we’re about to learn a lot about the new normal of what constitutes a layup math proof vs the truly difficult.
miguelnegrao 16 minutes ago [-]
If by solution you mean a proof and by testing you mean encoding it in lean and compiling it, the space of possible syntactically correct proofs which you can encode probably explodes in a way that is well beyond what any computer could try to brute-force. LLMs don't brute-force proofs, i believe their approach is quite similar to humans. I believe the same is essentially true for counter-examples of the type that have been found latelly, they are not found by search, but by using theory.
On the other hand even if the compute allocated by openai is esquivalent to day 10 human mathematicians, the machines can work 24h per day, that is already a lot more productive.
jstummbillig 4 hours ago [-]
> Any computable problem will eventually fall to computers.
I think the question, that we keep stumbling over, is what problems are computable.
> But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.
Let the goalpost shifting continue. It'll buy us another half year or so.
gerdesj 9 hours ago [-]
"Any computable problem will eventually fall to computers."
By definition. Its those pesky NP jobbies that get in the way.
aorloff 5 hours ago [-]
Science is not merely computing though, even the theoretical sciences
gerdesj 8 hours ago [-]
Just to re-iterate the point:
Whenever the handwaving starts around a discussion relating to a NP hard problem, I find it useful to imagine a Canadian bloke (MHRIP) in a red top, with a ... Scottish accent ... saying:
"Ye cannae break the laws o' physics, Jim". (maffs not fisics, obvs!)
If that is a bit tiresome for the gung-ho AI evangelist, there is also the rather knotty snag that that blasted Austrian geezer Gödel fiddled up: incompleteness.
Its almost as though these bloody clever scientific and that types keep on putting artificial blocks in the way of LLMs laying golden eggs!
I'm quite happy with the "marginal gains" I get with a DGX Spark. It will pay for itself within three months doing stuff on prem and us not sending data to someone else. It will scale.
GPerson 9 hours ago [-]
In my opinion, after seein the Jacobian conjecture go, the Riemann hypothesis only has about a year left.
I’m not sure if people just aren’t as aware, but the Jacobian conjecture practically was on par with those other great problems.
vlovich123 6 hours ago [-]
Whether or not it’s on par famously or difficulty level doesn’t predict whether the others will fall. They’re unique problems and math isn’t linear - the Jacobian it managed to find a counterexample and relied on other proofs that had been developed showing >3 case == 3 dimensional case. However, the Riemann may not fall in the same way because it may actually be true or the surrounding math isn’t quite ready to tackle that problem.
plaidfuji 2 hours ago [-]
I would be genuinely very impressed - but still not scared - if the Riemann hypothesis were solved. I suspect that we may require “new math” to make progress on that. If a new operator / symbol is required, is that fundamentally not doable by an LLM because it’s outside of current tokenization space?
qarl2 8 hours ago [-]
They don't need to make NP-hard tractable to shake the world order.
They just need to be better than humans.
9 hours ago [-]
VladVladikoff 11 hours ago [-]
Would be great to see them solve Yang-Mills and Mass Gap.
Chance-Device 3 days ago [-]
Pretty cool. The impact of AI is getting undeniable, there aren’t many positions left to move the goalposts to at this stage, next they’ll have to be outside the stadium entirely.
The sooner people can be broken out of their denial about all this the better, and we can start actually taking it seriously.
fhfncjcc 14 hours ago [-]
The models are frequently getting worse at items that they aren’t being benchmarked for — and that’s happening more and more over time! Other people in other fields aren’t idiots, they are accurately perceiving the fact that these models are being hyper optimized for our industry, and are becoming less capable in other domains over time. Models of the same scale are massively worse at writing a broad variety of styles of prose than their equivalent from two years ago. (Models of increased scale are a mixed bag.)
Maybe you’re the one who needs breaking out of your cached beliefs.
Legend2440 14 hours ago [-]
Proof?
In my experience modern models are better at all tasks than models from two years ago, especially complex multi-step tasks.
alightsoul 14 hours ago [-]
you are working on coding. they are working on things like "creative writing" remember that gpt 4o was popular among those who had ai as a romantic partnet?
Marha01 13 hours ago [-]
> remember that gpt 4o was popular among those who had ai as a romantic partner
I suspect GPT 5.6 would be even better at it, if given the same sycophantic system prompt and lack of guardrails.
QwenGlazer9000 12 hours ago [-]
Dude it's not a system prompt, it's the training.
HDBaseT 8 hours ago [-]
The API lets you adjust the temperature. Lower values introduce more deterministic outputs, which likely helps with the hallucination rates.
If you want creative writings, use the API and play with the sliders.
moyix 3 hours ago [-]
They actually removed the temperature parameter starting with GPT-5.
Marha01 5 hours ago [-]
I highly doubt that.
whimsicalism 12 hours ago [-]
gpt4o & associated parasociality is considered an alignment failure and is actively trained out of the model, so that is a terrible example of regression
michaelmrose 13 hours ago [-]
sycophancy
It wasn't "better" it was better at kissing your ass which matches what a lot of people want in a partner.
Legend2440 14 hours ago [-]
Well that's on purpose lol. OpenAI does not want you falling in love with their chatbot and have been deliberately training it to be less romantic.
vablings 13 hours ago [-]
There have been several cases of suicide and self-harm related to 4o, AI psychosis is a real risk and will probably be in the DSM
nostrebored 12 hours ago [-]
For customer support I don't think models have gotten better since gpt-4.1. The class of small models, with limited to no reasoning, that need to handle a complex issue with a touch of empathy, has not improved much.
I think most are actually worth, as agentic harnesses seem to optimize for solving poorly described problems rather than following complex procedures as written. In other words, instruction following maximizing models seem to make worse free-form agents, but they're really all that some domains need.
jstummbillig 11 hours ago [-]
I understand the point (I don't agree with it; tool calling has gotten much better/reliable and that is very important for customer support) but consider: If you can get same for a lot less, that's an improvement. If we found a way to supply fresh water and electricity for -90% cost after 2 years, that would be fantastic.
You can do many more things, when stuff is cheaper, even if the stuff were otherwise unchanged.
criddell 12 hours ago [-]
Do any of the big AI companies have a model that are good at tasks that require learning?
For example, every day people teach teenagers how to drive and with only dozens of hours of practice, they are on the road.
whimsicalism 12 hours ago [-]
is this not essentially what ARC-AGI-3 is? i agree that in-context/continual learning is somewhere the models are still mostly weak at
Chance-Device 13 hours ago [-]
So your answer is: ignore the progress, it’s not really happening, actually it’s getting worse.
That’s not a credible position, but there isn’t anything that I or anyone else can say to someone who simply doesn’t want to believe something.
cmdli 11 hours ago [-]
It sounds like they are making a clear argument: models are getting worse for certain domains even while they are getting better at others.
I don't know if I agree with that but it doesn't seem like an irrational claim and does seem credible to me.
Chance-Device 11 hours ago [-]
I just don’t think it’s true, or is significant enough to matter to the direction of travel of AI even if there were something to it. It’s another cope post being lobbed at the idea of AI going somewhere and I’m sick of them.
lioeters 6 hours ago [-]
"The sooner you can be broken out of your denial about all this the better, and we can start actually taking you seriously."
teravor 6 hours ago [-]
every lab independently discovered that getting good at bit alchemy (coding and related tasks) should come first as it will enable the formation of training pipelines that will then solve everything else.
so far there is no end to this progress in sight so it's full steam ahead on this singular domain. once it plateaus you should expect to see the greatest disruptions in human endeavors ever as all the training flops will start flowing to other domains to disrupt and dominate.
gste 13 hours ago [-]
People will be broken out of their denial by actual economic growth. That's what this is all meant to be for... I think we might start seeing some surprising numbers.
fckgw 11 hours ago [-]
How do I directly benefit from this supposed "growth"? Seems like its just making things worse for most people.
nightsd01 5 hours ago [-]
Plenty of people are making money from AI? And I am not sure I've seen it "making things worse for most people" yet, do you have a source behind this?
If your source is AI layoffs, there were plenty of layoffs with the invention of the horseless carriage, but that doesn't mean it made humans worse off overall.
There are reasons to be skeptical about progress and AI and all that but the 'making things worse for most people' thing you mentioned seems yet to be based in any reality
2001zhaozhao 12 hours ago [-]
Would something like "services deflation" even show up in the numbers? That's what I'd expect to happen first
whimsicalism 12 hours ago [-]
deflation will just be inflated away, always
mekael 12 hours ago [-]
But who benefits from that economic growth? If it's only the capital owning class, and the rest of us are left in the dust, then the growth is irrelevant.
I ,for one, have read enough history to know that it's never the proles who end up benefiting.
hibikir 6 hours ago [-]
I suspect that you have not read enough them. Ask someone in China (which yes, still has a capita owning class) if the proles were better off 50 years ago or now. The only real probes we have now in the west is that we have failed at housing, precisely because instead of growth, we have made many choices to increase returns for old people, and insufficient taxes for real estate.
nightsd01 5 hours ago [-]
I was a bit triggered by your China comment, if only given the fact that there is a very specific institution that drove many formerly prosperous cities and communities into abject poverty (the CCP in Mao's era) and then tries to claim all the credit for the market driving how many people they've lifted out of poverty.
And nowadays with Hong Kong (and probably soon Taiwan) they are proving they are perfectly happy to destroy economic growth as long as it benefits The Party
danparsonson 3 days ago [-]
Never understood all this talk about moving goalposts - you understand that's how science works, right? We improve, we learn, we recalibrate our expectations based on what we've learned. If we never "moved the goalposts", we'd be stuck scoring the same goals over and over.
NitpickLawyer 3 days ago [-]
> We improve, we learn, we recalibrate our expectations based on what we've learned.
That's not what people mean when they say "moving the goalposts". It means that people are adamant that something wasn't important/hard/impressive once the "AI" solves it. And then they come up with another thing that needs to be solved in order to prove it is important/hard/impressive. And once that happens, they do it again. And again. That's what "moving the goalposts" means.
It's also very much not a new phenomenon. It's been happening since the 1980s. As you can see from this quote from GEB by Hofstadter:
> There is a related "Theorem" about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of "real thinking". The ineluctable core of intelligence is always in that next thing which hasn't yet been programmed. This "Theorem" was first proposed to me by Larry Tesler, so I call it Tesler's Theorem: "AI is whatever hasn't been done yet."
Chance-Device 3 days ago [-]
Yes, this is exactly what is meant by “moving the goalposts”. And it’s a fairly well known expression applying wherever people retroactively change their requirements in reaction to those requirements having been met.
danparsonson 2 days ago [-]
It's almost like I disagree with your use of the phrase in this context, rather than that I don't know the meaning of it.
monktastic1 17 hours ago [-]
So you defined your own idiosyncratic version of "moving the goalposts" and used it to rebut his argument, with a condescending "you understand that's how science works, right?"--instead of being honest that it is you who are changing the definition, and not his failure to understand anything.
I don't see how that's any better.
seanhunter 15 hours ago [-]
He rebutted the argument about moving the goalposts by moving the goalposts. It’s better by dint of sheer bravado.
albedoa 14 hours ago [-]
How were any of us meant to know that you were using your own personal and undisclosed definition of a well-established phrase?
mag7269 16 hours ago [-]
“AGI will only, truly, be achieved when the machine can destroy an industrial-type toilet after downing a Supreme Burrito and a large Baja Blast.”
-Alan Turing (allegedly)
15 hours ago [-]
danparsonson 2 days ago [-]
If it seems like I don't understand the meaning of that very well-known phrase, then clearly I have failed to make my point. I'll try again. And please note that I will use some generalizations to make my point more clearly, rather than because I don't understand nuance; kindly grant me a charitable reading.
In recent years, I have commonly seen the phrase "you're moving the goalposts" deployed by the "it might be sentient" crowd to shoot down the "it's a stochastic parrot" crowd when the latter respond to a new development with "OK but...". In a well-understood field of inquiry, that would be a clear case of goalpost-moving, in the commonly-understood meaning of the phrase where requirements are retroactively changed in response to them having been met. Thank you OP. 'Artificial Intelligence', and indeed intelligence in general, is very much not a well-understood field of inquiry - in fact we don't even have a common agreement about what 'intelligence' is. We are therefore learning as we go (even after all this time!) but making rapid progress in recent years. When rapid progress is made in a poorly-understood field, then how can our definitions and requirements for success not change? This is arguably one of the most pathological development projects ever - what are the requirements? 'It thinks like a human'? What does that mean? And the answer is we don't know what that means, and we're working it out as we go - moving the goalposts. If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
Side note that, in case it's not obvious, none of this detracts from how impressive LLMs are. They're a marvel of the modern age, all the problems notwithstanding. However I reserve the right to stay sceptical about their capabilities.
strbean 13 hours ago [-]
> When rapid progress is made in a poorly-understood field, then how can our definitions and requirements for success not change?
It's in how they change, not the fact that they change. The skeptics seem to have secret definitions for intelligence, sentience, consciousness, creativity, etc. that amounts to "a thing only humans have". Often that thing is equivalent to a soul. When yesterday's challenge (LLMs don't have X because they can't do Y!) is met, Y changes but X stays the same. This is not the process by which a field matures, it is a rhetorical technique used by skeptics to avoid honestly stating or confronting their internal definitions. That can be revealed by asking the skeptic the following:
"Forget LLMs. What if we made a completely physically accurate simulation of a human being?"
Many say no, that simulated human being still couldn't have (intelligence, consciousness, sentience, creativity, ...). This reveals that there is a necessary metaphysical component to those attributes, at which point any scientific-minded person will leave the debate.
monktastic1 14 hours ago [-]
Thanks for this clarification of your position. The brief answer is:
> If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
The criticisms are directed toward people who did clearly act like they knew, not the ones who were honest that they did not know.
Windchaser 16 hours ago [-]
> If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
To me, the goalposts were already defined by the person you were responding to. "The impact of AI is getting undeniable", so, the goalposts are "the impact of AI". Probably something like "the impact of AI is high, or will be soon".
Note that this does not depend on things like AI sentience or defining "intelligence" more rigorously, it just depends on AI impact.
gowld 16 hours ago [-]
What you are doing is "motte and bailey".
The motte is "AI useful". The bailey is "Singularity is nigh".
Windchaser 16 hours ago [-]
The unified position which many folks deny is "AI is powerful" or, alternatively, "AI will be powerful soon".
(I'm personally still skeptical about this, but I'm being pulled towards accepting it).
"AI is useful" is too low of a bar, and "singularity is nigh" is too high. "AI is on its way to upending society" is about in the middle, and still vastly contentious among laypeople.
enraged_camel 16 hours ago [-]
>> The motte is "AI useful". The bailey is "Singularity is nigh".
But there are people like Ed Zitron, frequently posted and cited here, who disagree even with the former.
scotty79 15 hours ago [-]
I think Ed Zitron is mentioned just because his name is Zitron. He just repeats ad nauseam opinions concentrated around one simple, very boring pole on a wild and interesting landscape of emerging reality. Anybody could be doing that. A lot of people do that. Yet no other is named Zitron. And that's why I heard name Ed Zitron hundred times. That's how you become a voice of (a part of) the generation. Just have a memorable name and repeat the same opinion over and over that people can flock around comfortably. Content is irrelevant.
Personally I prefer to follow explorers rather than swamp-sitters.
8note 15 hours ago [-]
to an extent zitron is saying its not useful, but as a more nuanced opinion, "ai is not cost effective, nor is it improving profits or revenue"
"it isn't clear whether generative AI actually provides much business value at all"
"cannot seem to find a product that people will pay for, in part because the results are so mediocre"
"Last week, we got our first real, definitive glimpse of what’s around that corner that future. And boy, was it underwhelming."
"OpenAI claims that o1 “performs similarly to PhD students on challenging benchmark tasks in physics, chemistry, and biology.” Just not in geography, it seems. Or basic elementary-level English language tests. Or math. Or programming. "
"Worse still, it's kind of hard to explain why anybody should give a shit about o1."
"o1 shows that OpenAI is both desperate and out of ideas."
"the software is not becoming more useful"
Honestly, every other line is quotable in this context.
lackoftactics 14 hours ago [-]
Yep, he is a PR stunt guy, and the number of videos that come up when you type Ed Zitron into YouTube should tell you how many people are eager to feed their cognitive biases.
dwaltrip 14 hours ago [-]
It’s comical and honestly incredibly embarrassing…
But it seems we have somehow optimized away shame. It wasn’t good for profits, I guess.
claytongulick 15 hours ago [-]
> And then they come up with another thing that needs to be solved in order to prove it is important/hard/impressive. And once that happens, they do it again. And again. That's what "moving the goalposts" means.
The fundamental argument that I've personally made since the early days of this is that LLMs are not reasoning, in the way that word is commonly understood.
There are lots of reasons why that argument needs to evolve that could certainly appear to be "moving the goalposts", but let's take an example.
A lot of AIs were tripped up by the question "Should I walk or drive 50m to the carwash?" Several folks liked to use that as an example that illustrates that LLMs aren't reasoning, but as the models have been trained on that specific example, it's of course less useful. An AI can mostly nail it now.
So a different example is needed. A new demonstration of how these things fail at basic reasoning a child can do.
Did I move the goalposts? I don't think so. The fundamental argument stays the same. It's not hard to find lots of examples that trip up LLMs, because they are what they are: statistical inference machines. Nothing more and nothing less.
Useful, sure. But also commonly misapplied to areas for which they are inappropriate solutions.
3 days ago [-]
f6v 14 hours ago [-]
> Never understood all this talk about moving goalposts - you understand that's how science works, right?
I agree with the parent that we need to acknowledge that we're at a turning point in history. I lived through some of them (internet, ubiquitous personal computing). But it's somewhat difficult to comprehend the impact of this one for many people.
I do biomedical research at one of the top European research institutions. We're very well-funded, but I can clearly see the gap between us (say, top-100) and top-10. I also realize this gap is going to get so much wider unless we invest heavily in AI access (and I'm not so sure I can sell anything more expensive than $20 Claude subscription to the leadership).
I think people having 6-7 figure SOTA AI budgets will move exponentially faster than those who don't. That makes me worried.
So, for me, it's not a question of recalibrating expectations. We're way past that.
emceestork 2 days ago [-]
They aren't claiming that science doesn't progress by moving goal posts. They're talking about how critics of AI have claimed it isn't revolutionary/useful, then progressively changed what would it mean for AI to be actually revolutionary/useful.
Not long ago many folks were saying AI was the same as the crypto bubble. No real useful technology and only hype.
gowld 16 hours ago [-]
Did you know that "revolutationary" is not equivalent to "useful", and "revolutionary" is quite ambiguous?
emceestork 14 hours ago [-]
I don't know if you're trying to dunk on me. I didn't intend to imply they are synonyms.
I think AI is clearly both revolutionary and useful. Revolutionary insofar as the job I do has changed almost completely in a year or so span.
arenaninja 15 hours ago [-]
It's indeed very exciting. I'm looking forward to new advancements/predictions in physics. Preferably as beautifully explained as E = M*c^2
slashdave 3 days ago [-]
> The sooner people can be broken out of their denial
There is irony here
15 hours ago [-]
dwroberts 6 hours ago [-]
Can someone explain why we’re not using it to solve the obvious big conjectures/problems though? If this stuff is solved, why isn’t Riemann the first thing to go? Even if it means a fund of several hundred $k to let it churn on it.
slashdave 3 hours ago [-]
Those that claim their opponents are in denial are most likely in denial themselves
whimsicalism 14 hours ago [-]
it is very hard for people to eat crow, as the replies will show
dominotw 13 hours ago [-]
Really? has anyone ever claimed that ai will never be able to prove theorems and conjectures ?
whimsicalism 12 hours ago [-]
yes, in fact I recall many people on this exact forum saying that AI will not be able to novel work at all.
i can link you likely dozens of comments from people wrong about this replying to me over the last 5 years
Guys, please use critical thinking. The haters don't hate by default, we hate because we're gaslit about this stuff every day and it's annoying. Extraordinary claims require proof, and they're not giving us information that would be essential to knowing if this is actually significant or not.
w4yai 14 hours ago [-]
AI already have an impact, and yes this is PR hype because this is a product. Yet both can be true at the same time. We're not blindly eating what's OpenAI is serving us as gold truth, we're just admitting it's doing remarkable progress.
Remember October 2024 Pelicans [1] ? It's been only less than 2 years.
We don't know what will come in the next 2 years. But the progress doesn't seem to stop for now.
People are skeptical of the announcement because the room include several PHDs in math and physics. The prompts are not published so we can see how generic the starting prompt is.
Chance-Device 12 hours ago [-]
> The pelicans are still not ok to this day.
If I could take one out of context quote from this whole thread as a response to TFA, it would be this one.
afro88 14 hours ago [-]
Gary doesn't argue it's hype though. He argues 2 things: other people are getting carried away with the result, and we don't know enough about how it was reached to know where it falls on the impressive scale.
He literally says it's an impressive feat in the second article.
efavdb 14 hours ago [-]
best point in that first link: openai doesn't tell us if they only tried to solve these 10 and each was solved (amazing) or if they asked it to solve a million problems and it got these 10. Either is great, one is more so.
hgoel 14 hours ago [-]
Your comment seems entirely disconnected from the posts you linked. It's impossible to deny the results, it is not PR hype that in the past couple of weeks LLMs have resolved problems that have been open in mathematics for many years. Some of those problems had remained unresolved despite keen interest from many humans.
The only way that is PR hype is if you're invoking the insane conspiracy that frontier AI labs are just buying off results that would otherwise be career defining for a mathematician, just for marketing.
The posts you linked are urging caution regarding the exaggerated e/acc-esque lies peddled by people like Musk, not that the models haven't proven themselves as having genuine ability to contribute to research in some areas.
bluerooibos 14 hours ago [-]
Gary Marcus has been moving the goalposts since day 1. The guy is a psychologist. Why would anyone care what a psychologist has to say about AI? He's likely made good money from constantly moving the goalposts and being a denier, due to the publicity he gets.
mef51 14 hours ago [-]
Because he's not talking about AI, he's talking about people's psychological reactions to AI
bonoboTP 14 hours ago [-]
He's simply a good phone number to have for journalists under time pressure who need to add the contrarian voice to their upcoming story. He delivers it reliably, then never reflects on how he was wrong in the past, just blasts forward as if nothing happened and just makes the next bonkers claims to the journalists who are very thankful for the prompt delivery of how AI is a nothingburger, and fake and won't ever do XYZ that it then proceeds to do in N months.
I remember the time when he insisted that diffusion-based image generators trained on Internet scale data will never be able to make an image of a horse riding an astronaut. Today you can generate 4K video of that.
dwaltrip 14 hours ago [-]
I use these strange machines all the time. They have gotten notably smarter. That’s my personal experience.
They still do things that I find incredibly annoying and “dumb”. And I still have to clean up messes they make quite often.
But on the whole they are clearly smarter than before. No extraordinary claims needed. I just try to learn how the tool works and how to use it effectively.
Trasmatta 14 hours ago [-]
Society at large is getting worse at critical thinking, because we are increasingly offloading that thinking to AI
c7b 16 hours ago [-]
And what does taking it seriously entail?
Chance-Device 15 hours ago [-]
In the near term handling the transition. Jobs will be lost, careers ended, people won’t be able to reskill quickly enough. At the same time AI is an enormous opportunity to uplift living standards, but nobody has the logistics of this figured out.
We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.
How does intellectual property work in an AI generated future? What about healthcare advances, who gets to own those?
What about meaning, what about purpose? How do we replace the work ethic that tells us we are our jobs and idleness is immoral? How do you replace “What do you do?” As one of the first questions you ask a new person?
That sort of thing.
striking 15 hours ago [-]
I think this too is a kind of denial. As in, while some are in denial about the usefulness of AI, others are in denial about whose living standards are actually going to be uplifted.
And it's sad, really, because I think these two groups would make a great pairing if they could stop arguing against one another for a moment. They'll both be impacted about as much and probably have the same ultimate goals (to lead dignified lives).
But it seems these days everyone is more interested in Kayfabe and feeling like they're in the right than working together, so maybe I should just keep quiet rather than attract the ire of both groups...
throwaway0123_5 15 hours ago [-]
> others are in denial about whose living standards are actually going to be uplifted.
I don't know if it is fair to say they're in denial. For my part, I don't expect life to get much better for regular people (especially short term), but that doesn't mean we shouldn't work to try to make it happen.
Chance-Device 15 hours ago [-]
Asking questions about policy and values and pushing to have those resolved in positive ways is about as far away from denial as you can get. It’s possibly the only useful thing an ordinary person can do.
What a lot of people want to do, and I’m not saying that you’re one of them, is to assume that a positive outcome is impossible and either do nothing or loudly yell that the world is ending. Neither is particularly useful.
Or, as I said above, others just deny that there’s anything to see here and try to get people to move along.
HarHarVeryFunny 14 hours ago [-]
The AI companies themselves, who are highly motivated to sell AI as overall positive for society, notwithstanding some security/etc risks, and who have economists on the payroll to think about things like this, do not seem to have found any possible positive outcome to present.
Shane Legg (DeepMind co-founder), one of the more intelligent and thoughtful people you'll find in the industry, could only offer "it's a tough problem - we need to think about it" when recently interviewed by Hannah Fry.
On the surface the most likely outcome for AI allowed to replace jobs is extraordinarily negative, especially since it is a general capability technology, not a specific one where displaced workers can just move to another field. Once AI becomes more capable it will be able to do the vast majority of white collar jobs, including any new ones that may appear as a result of AI. As Shane Legg put it, "if your job can be done remotely, sitting in front of a computer, then it can probably be replaced by AI".
Not only does AI threaten to replace ALL the white collar jobs, but it is rapidly going after blue collar (factory jobs, driving jobs) and pink collar ones (Japanese robotics for elder-care) as well.
If a positive outcome (which doesn't include putting displaced workers on welfare - UBI) is possible, then it sure would be nice to hear it, and the silence from the AI companies, and government for that matter, is deafening.
Chance-Device 13 hours ago [-]
UBI probably is the positive outcome, though it may not seem like it to begin with. Initially it will likely be stigmatised and under-resourced, but as a larger proportion of people move out of work and onto UBI that stigma will drop and the resources should grow.
Eventually UBI will be the norm, and if the living standards of a person on UBI is as good as yours or mine today, that will be an enormous win for everyone. It’s like pensions, once these were only for the elderly poor, now they’re a right for everyone in most developed countries.
It’s also interesting that for most of human history leisure time was the point of life, and only in recent modernity has work come to be the meaning of someone’s existence.
UBI has to be commensurate with production being automated. That’s a big logistical problem, if you think building datacenters is a challenge try bringing about radical abundance, but even so it’s not insurmountable. It just needs to be taken on as project and not seen as an impossibility.
So much of this is not about what is possible so much as what people believe is possible. We can do anything if we try.
bubblemoth 12 hours ago [-]
> Eventually UBI will be the norm
I see comments like this tossed around a lot, but what makes you say this? Don't you think its more likely that most people end up in poverty?
Chance-Device 12 hours ago [-]
No, I think people will demand to not live in poverty. Capitalism allows people to escape poverty by personal effort. An AI future won’t allow it by any means other than collective effort, so that’s what we’ll do.
mekael 12 hours ago [-]
That will entail an immense amount of violence to be undertaken by the working class against those in power, unless you think they'll gladly give up all of their ill gotten gains out of the goodness of their black little hearts.
orangecat 12 hours ago [-]
In fact I don't think that rich people are cartoon villains who enjoy watching the poors starve. If AI leads to greatly increased productivity, then at existing tax rates there will be more than enough to provide good living conditions for people who can't find jobs.
arbitrary_name 4 hours ago [-]
so why do we have the poverty and inequality today?
why do they spend the money they do on the things they do?
they are not altruists and they never will be.
they could change millions of lives today, but most do not.
pure naivete.
azan_ 3 hours ago [-]
Poverty and inequality keeps declining all the time thanks to economic growth.
mahogany 8 hours ago [-]
> No, I think people will demand to not live in poverty
What about other countries in the world where people... live in poverty?
> Capitalism allows people to escape poverty by personal effort
Isn't it possible that there is a future where AI makes the average value of human labor (or, "personal effort") plummet? Perhaps capitalism will lose some of its edge against a technology like this.
orangecat 13 hours ago [-]
do not seem to have found any possible positive outcome to present
This seems more a fantasy than a considered likely outcome. He basically admits that humans will eventually mostly all be out of a job, replaced by AI, but then says (Gemini's summary) that there will be:
"Massive Economic Abundance: Because AI will exponentially grow the total economic pie, overall resource scarcity will diminish. The fundamental challenge shifts from producing wealth to distributing wealth."
So how do we go from everyone out of work, no income to spend on food, or the goods and services that the AI is producing, to "massive economic abundance"?!
It's like the meme:
Step 1: Create AI
Step 2: AI takes all the jobs
Step 3: ???
Step 4: Profit! (massive economic abundance)
What is step 3?
Marha01 12 hours ago [-]
> What is step 3?
"Massive Economic Abundance" implies massive increase in produced goods. This implies massive deflation, ceteris paribus. So step 3 could be simply printing money to pay for UBI. Deflation from AI productivity increase and inflation from UBI money printing will cancel out.
HarHarVeryFunny 12 hours ago [-]
This doesn't make any sense.
For money to work it has to represent some real value, something that has some scarcity to it such as potatoes or hours of human labor. Ultimately it is just a decoupler in a barter system, a universally recognized IOU.
Why would someone give me a car in exchange for UBI-scrip when that UBI-scrip has no inherent scarcity or value and can be produced in infinite supply by the government ?
Marha01 5 hours ago [-]
> Why would someone give me a car in exchange for UBI-scrip when that UBI-scrip has no inherent scarcity or value and can be produced in infinite supply by the government ?
UBI script will have some value, I am proposing printing enough money just to combat AI productivity-induced deflation, not infinite UBI money.
striking 13 hours ago [-]
Asking politely is not how we got a 40-hour work week or workers' comp or most other labor standards we take for granted, just historically speaking. I think a positive outcome is very likely, and I think it will be a lot more work than loudly yelling, but I don't think anything will happen if we try to build everything up from first principles instead of taking a moment to consult history as many are wont to do in this AI era.
bubblemoth 14 hours ago [-]
What pressure is there to push for any of these changes? I see AI advocates discussing the concept of UBI, but I can't imagine a world where the United States would ever pass this sort of legislation. I mean, congress can barely pass a budget each year.
If you are correct, I expect corporations to reap massive profits while most Americans try to find a way to survive in a world where they are obsolete.
Chance-Device 14 hours ago [-]
There is whatever pressure you bring to bear on it. As long as you are still living in a democracy your vote is the pressure you can apply. If the parties that exist won’t represent you, make new ones that will. Do something rather than deciding it’s both impossible and up to other people anyway.
azinman2 14 hours ago [-]
I also don’t understand why UBI is desirable. Putting everyone on welfare means everyone is poor. This won’t end well, and it’s certainly not the case that 99% of the population will let a tiny number of people remove their income in favor of pennies for all. Political violence will come first, easily.
sodapopcan 12 hours ago [-]
No idea why anyone would downvote this, it's true. As someone else pointed out "b = basic." And does anyone think it will adjust with inflation?
frabcus 13 minutes ago [-]
Because the implied alternative of the comment is that people will carry on having jobs, which breaks the entire premise that there is AI that replaces all human labour. The comment is missing the point - UBI is the only suggestion anyone has so far that meets this fundamental question about the technology. We will be less poor than without UBI, unless we come up with a better plan.
thuuuomas 13 hours ago [-]
Why should IP persist in a world of “intelligence too cheap to meter”?
sodapopcan 12 hours ago [-]
Everything you're saying is oddly simultaneously specific and hand-wavy at the same time. I say this because I'm not sure a lot of these things are solvable or, if they are, it's going to take lifetimes. For example:
> How do we replace the work ethic that tells us we are our jobs and idleness is immoral?
For many people it has nothing to do with morality, it's hardwired into their instincts. They want to work, and they will work.
For a lot of us who are not excited about this future it's that no one is trying to answer all the questions you laid out. Instead we have the disgusting people at the helm purposefully spreading doomerism and saying, "We'll figure it out." I think it's pretty problematic (to say the least) to care more about technological advancement than how that advancement is actually shaping up to effect people in the short term. But I know many people don't care, especially those who believe they won't be among the affected.
infinitezest 11 hours ago [-]
Beautifully put. This exactly summarizes my feelings about this particular tech. I actually find it hugely useful. But I fear we lack the wisdom to use this in a way that won't destroy us.
cautiouscat 13 hours ago [-]
I’ll be the first to call myself cynical.
> In the near term handling the transition. Jobs will be lost, careers ended, people won’t be able to reskill quickly enough. At the same time AI is an enormous opportunity to uplift living standards, but nobody has the logistics of this figured out.
> We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.
UBI in the United States is never going to happen in time. If it happens at all. We don’t even get universal healthcare. I think people who think AI will be a net positive for humanity are also in some sort of denial.
In a different US political climate I would entertain it. If these frontier labs weren’t so clearly going after the money, I would entertain it.
LLMs are clearly a step up for capitalists so I just can’t see any inclusion of LLMs move towards more progressive ideologies.
frabcus 10 minutes ago [-]
I guess we'll have to fall back on the plan of hoping one of the 7 Anthropic co-founders has secure power in this world, and decides to use EA principles to give us all a bit of their share of the future light cone ;) (The wink because I'm kinda not really joking that this seems the best plan, if we actually get superintelligence from a US AI company)
joshmarlow 15 hours ago [-]
I don't understand why this got downvotes - simply extrapolating current trends leads to the need to answer all of these questions.
My own $0.02 on the economics piece - every country should have a sovereign wealth fund. Governments should block market access from automated[0] companies until those companies provide equity contributions to the wealth fund for that country. This aligns regulator and corporate interests. Dividends flow into the sovereign wealth funds and then can be allocated locally from there - UBI, job programs, etc. Let different jurisdictions explore different ways to structure a post-labor society.
On the broader social front - I think a lot of lack of meaning discussion boils down to the overemphasis we have on your job as your self-worth. We need to realign our societal expectations - and people need to spend more time with their families.
[0] for this to work, I think we would need well accepted metrics for 'how automated' a company is - and that probably needs a 3rd party auditing industry.
axus 12 hours ago [-]
Nationalize the data centers, reserve enough inference to automate power generation, food production, transportation, and housing.
Is there any government that has gotten socialism correct for its citizens? I'd point to UAE/Qatar if they didn't depend on human servitude and inequality.
GPerson 15 hours ago [-]
Why do you think the AI companies are going to let it be a “we” kind of decision, and since it’s obviously not going to be a “we” kind of decision, as getting to this point certainly has not been thanks to people like you, what makes you think living standards are going to be broadly uplifted?
esafak 15 hours ago [-]
Isn't it obvious if AI automates your job away, the AI company is going to reap the economic value, which is going to be less than what you cost, while giving you a pittance as UBI? If you received its full value there wouldn't be any point in anybody replacing you with AI.
The only way to win is to wield the AI.
throwaway0123_5 15 hours ago [-]
Agreed, a LOT of UBI advocates gloss over the "B" in UBI. If AI increases human productivity overall, the only morally acceptable outcomes (imo) are that everyone's standard of living increases (or at least is the same without having to work) and wealth inequality decreases (if AI is doing ~all the work, there really isn't any sensible justification for some people having significantly more wealth than others). Frankly anything else seems like a recipe for massive social instability.
unfitted2545 14 hours ago [-]
Nationalised LLM? As long as the state doesn't decide what information the LLM shares (from an output and privacy perspective).
hansmayer 14 hours ago [-]
[dead]
GolfPopper 15 hours ago [-]
[flagged]
WarmWash 15 hours ago [-]
The human zoo where the top ~250,000k humans live in a "human utopia" and the AI provides while mostly focusing on whatever it decides it's own goals are.
Humanity survives (but we reading this probably don't), the AI treats the living humans like the Emperor's favorite pets (probably a pretty good life), and then the AI does whatever else it deems important.
mofeien 13 hours ago [-]
To what kind of goal that an ASI might decide to pursue would "a quarter billion happy, healthy, free people" be the most efficient solution to?
WarmWash 13 hours ago [-]
My mistake, I meant 250k*
14 hours ago [-]
waffletower 14 hours ago [-]
[dead]
logicchains 15 hours ago [-]
Realistically it means trying to start a small business of some sort, because AI is hugely advantageous to business owners and disadvantageous to workers. And it's something AIs can't do unless they get legal personhood, which may well not happen any time soon.
GPerson 15 hours ago [-]
Hopefully doesn’t ever happen, since that’s one of the more plausible omnicide scenarios. I human like AI is almost certainly achievable without much research effort at this point, but we shouldn’t do it.
twister2920 15 hours ago [-]
[dead]
kypro 16 hours ago [-]
[flagged]
addaon 15 hours ago [-]
> "probably only 20% chance we all die"
Bad news for you -- there's a 100% chance we all die. Sorry to be the one to tell you.
cubefox 15 hours ago [-]
Logical mistake. There is a difference between every human dying eventually and humanity going extinct. The former doesn't imply the latter. The previous commenter clearly meant the latter.
kaonwarb 15 hours ago [-]
What makes you think separate nations would effectively cooperate in this way?
kypro 12 hours ago [-]
Might as well try, no? Or do you think it's better we just risk it? I mean it's not like the West has any hard or soft power to encourage cooperation.
lkey 15 hours ago [-]
Your 'serious' proposal is unlimited global military bombing campaign on civilian infrastructure by the United States (which is currently losing a war using the same strategy) to 'solve safety' preemptively against a 20% number you just made up?
And you accuse the 'other side' of 'suicidal apathy'??
You should put down the AI and do some self-reflection on how you came to hold these views.
armchairhacker 15 hours ago [-]
GP said cooperation, never implied the US would do it alone.
kypro 12 hours ago [-]
> You should put down the AI and do some self-reflection on how you came to hold these views.
I've been in the field for almost 2 decades, and actively thinking about AI risk for over 15 years.
I've always held the controversial opinion that there may come a time where we might unfortunately have to consider using force to protect our civilisation from the threat of ASI, but I've always reframed from openly discussing that because I'm so personally against any use of force and there's always been time for more peaceful options.
Several months ago I changed my tone on AI risk to stop worrying about the optics of what I'm saying because it's too late in the day for me not to speak plainly at this point. Similarly over the last week I've decided I can no longer reframe from advocating for the use of force (if necessary) to stop rouge actors from playing Russian roulette with civilisation.
Please understand I don't want anyone to be harmed. Perhaps you don't agree with my suggestion, but my stance ultimately comes from a position of harm reduction.
> preemptively against a 20% number you just made up?
It's a probability estimate. Happy to expand in detail on why 20% specifically, but first you accept I can't tell you what's going to happen in the future with certainty right? To some degree any prediction you might have about the future will be "made up".
lkey 10 hours ago [-]
Advocating for nuclear war as a means of 'harm reduction' should be causing you serious cognitive dissonance.
There are dozens of escalating, yet fundamentally less destructive means to prevent a data centers from continuing to operate (if that's the goal). Not starting at those more modest interventions is political malpractice and in opposition to your harm reduction narrative.
Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
kypro 9 hours ago [-]
> Advocating for nuclear war as a means of 'harm reduction' should be causing you serious cognitive dissonance.
If you're referring to the "nuke the data centers" line in my bio, that's tongue in cheek. It's an expression of my sentiment towards AI at the extreme.
To be clear, I don't think any country should seriously be considering dropping nukes on data centers. Targeted strikes on data centers operated by rogue private companies or nation states is acceptable to me – assuming pre-warning of intent and appropriate regulatory foreshadowing.
I suspect we agree this is reasonable?
> Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
> Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
No, they're not based on unbounded exponential growth curves at all... I'm actually somewhat skeptical of claims that there could exist an intelligence that far exceeds our own. My assumption is really just that AI will be far faster, far more knowledgable, far more persistent and marginally more generally intelligent than the average human. The first three are already true, the forth we're getting to.
We're on an exponential curve right now and I suspect the rate of progress will accelerate some more as a result of RSI. This will likely continue for 2-3 years (if we have that long) before starting to flatten out.
As for my doom prediction, my median concerns are around around the destabilising nature of ASI.
If it can be controlled then it will concentrate power like never before, and I believe this will destabilise civilisation. I can go into more detail here if you wish.
However, I'd still put these destabilisation risks at under 50% in total. My concern more broadly is the majority of ASI outcomes are doom scenarios. It's extremely hard to imagine a world in which you have a magic box that can grant any wish that intelligence can grant, but which we have a functional civilisation. You have to basically assume all potentially dangerous wishes won't be granted and probability the only feasible way to do that is restrict who can make the wishes – but then you have the concentrate of power risk.
I largely agree with the premise "If Anyone Builds It, Everyone Dies", so we must try to avoid building it. I think we can build ASI safely, but we're clearly not on the path right now and I suspect it might take decades (or more) to figure out how we can build ASI with an extremely high probability of a good outcome.
> Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
I do this. I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
I don't think we realistically have a chance of diverting from this path at this point. I have made peace with this but it makes me very sad and haunts my dreams almost every night.
lkey 7 hours ago [-]
> If it can be controlled then it will concentrate power like never before, and I believe this will destabilise civilisation.
We already live in a reality with unprecedented concentration of wealth and power in the hands of the very few.
This process has produced horrific violence and ended civilization for tens of millions.
The actual impact of AI, so far, has been redistributive upwards (solely through speculation, not via productivity or displacement).
This kind of problem is a standard human social ill. Its solution is social too.
Redistributive economic and political action can equalize power and wealth disparity. AI is incidental in this story.
Your understanding of AI as an isolated perfected intellectual djinn lacks a consideration of the existing and ever-changing labour and social relations that give AI responses meaning in the first place.
It is a compressed, multi dimensional manifold containing the structure of relations between objects.
Responses are a reification of a point in that space.
Navigating the space can be interesting, but it is a poor substitute for the relations themselves.
Very few people yearn for the endless, frictionless solipsism silicon valley promises (AI + slave robots), and those that do are overrepresented in your social circles. The books they write are a study in psychological projection and limited political education.
> I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
This is good advice generally. But it isn't organizing. Join something, meet up, drink bad coffee, listen, take notes, advocate to fix just one bad thing in your community, feel your impact reverberate through the world, and then notice how much easier it is to shake off despair.
If you don't have a community, then no wonder everything feels like its ending.
deaton 15 hours ago [-]
If the thesis that "If anybody builds it, everyone dies" is true, or has any chance of becoming true, then it is the logical thing to do. As Geoffrey Hinton said, "If you want to know what it's like not to be the apex intelligence, ask a chicken."
lkey 14 hours ago [-]
It is not 'logical' to imagine a hypothetical doomsday scenario that justifies a preemptive nuclear war. (which is what the grandparent commenter's bio contemplates, 'nuke the datacenters' is their central credo).
Does it bother you that the people who are publicly cocksure that P(doom) is moments away are the same people that have profited most handsomely from that pronouncement?
That the 'humanists' that want to do 'altruism' for 'potential future humans' and are the same people that commit fraud and theft at a civilizational scale, then sell this 'intelligence' to any child-incinerating militaries with spare cash?
It's not wrong to want to do good, but if a system that is branded 'do (the most) good' commits great evils, you are morally and intellectually obligated to step back and reconsider how you are spending your time.
Also, I asked a chicken and a feral rock dove what it's like to be not be 'apex' and they burbled at me and kept eating millet and sunflower seeds.
Would you like me to follow up with them? I'm not sure what point you expected them to make.
cubefox 14 hours ago [-]
> It is not 'logical' to imagine a hypothetical doomsday scenario that justifies a preemptive nuclear war
You hallucinated the "preemptive nuclear war". He didn't say anything about nukes. That's your own invention.
> Does it bother you that the people who are publicly cocksure that P(doom) is moments away
20% is not "cocksure". The "moments" is again an exaggeration.
lkey 13 hours ago [-]
I did not, I read kypro's (the OP I was replying to) bio, to whit:
Every problem is a search problem.
Nuke the data centers.
I've removed the "nuke the data centers" line now.
It's was an expression of my sentiment, not a policy position I'd support. I wrongly assumed that was obvious, but some people are making bad-faith assumptions about me and my sanity.
Not that it should need to be said on a comment thread where I am express concern about civilisation in a post-ASI world, but I obviously don't want to see a nuclear war.
samatman 13 hours ago [-]
If the Judeo-Bolshevist theses were true, then Operation Barbarossa was the logical thing to do, and everything which came with it.
Powerful word, `if`. "You're not only wrong you're a fulminating psychopath" is a perfectly valid response to getting it wrong like a fulminating psychopath.
DarmokJalad1701 15 hours ago [-]
No.
armchairhacker 15 hours ago [-]
You’re not necessarily wrong, but math proofs aren’t going to unify today’s brainrotted population.
bigyabai 15 hours ago [-]
> We are now at the point where RSI is feasible
What can be asserted without evidence can also be dismissed without evidence.
- Hitchen's Razor
apetresc 15 hours ago [-]
The evidence is abundant and nearly impossible to ignore without increasingly focused effort.
bigyabai 12 hours ago [-]
So abundant that you won't link anything? Send a peer-reviewed paper then, let's settle this.
kypro 11 hours ago [-]
Did you read the article?
AI is a math & computer science problem. If AI can do advanced maths and computer science research, then it can begin to suggest useful algorithmic optimisations.
Right now I'm sure the vast majority of these will be junk, but occasionally, even with current limitations, they might occasionally stumble on something.
It's not really whether RSI is or isn't possible, it's really just whether it's the most efficient way for labs to improve their models today given they have limited compute to run AI-generated experiments and access to very intelligent humans who might have a better hit/miss ratio.
Do you disagree with anything I'm saying here? Do you not think current AIs can suggest algorithmic improvements or something?
ghjkghjkghj 16 hours ago [-]
There are some genuinely insane takes in this thread on both sides but this takes the cake.
iwontberude 15 hours ago [-]
That's because its satire, the tell was "not excluding targeted military strikes"
ghjkghjkghj 15 hours ago [-]
Gonna say the edit they made removes the satire possibility.
kypro 12 hours ago [-]
Not satire, but as you can see one is engaging in good faith and everyone is either engaging in character attacks or low-effort counters.
It's fine, and I'm very used to this, but if you want to have a dialogue I'd be very happy to. I'm a very reasonable and sane person outside of apparently holding some controversial views on AI risk =)
I'd love to know why you think what I said was insane?
cubefox 15 hours ago [-]
A few years ago the majority of Hacker News dismissed LLMs as not much more than stochastic parrots who were decades away from doing any serious work, autonomously escaping containment and hacking Hugging Face, or outperforming most professional mathematicians at proving theorems. These things were labeled "insane" and "science fiction" despite the rapid progress we had already seen.
Now you are again postulating that there would be no more extreme progress in the near future. That's actually more "insane".
GPerson 15 hours ago [-]
Being right about the thing that’s genocides all culture and likely worse isn’t such a great thing to brag about, but go ahead.
matsemann 14 hours ago [-]
Which straw man are you arguing against?
Dig1t 13 hours ago [-]
I don't think this is a straw man, a huge number of people in my life (non CS people) think AI is a dead-end, that it's just a stochastic parrot, that it'll never be able to do many things that humans can do. I have had many arguments with people who told me that "AI will never be able to do X", and then 6 months later AI is able to do X. Then they will move the goal posts and say "well AI will definitely never be able to do Y".
hansmayer 14 hours ago [-]
[dead]
datakan 3 days ago [-]
[flagged]
Chance-Device 3 days ago [-]
I can deal with apathy, that’s the norm. What bothers me are all the people who think they can suppress AI by talking it down. That’s what’s counterproductive, just pretend the problem doesn’t exist. Tell other people it doesn’t exist either. I get it, it’s threatening socially, economically, maybe existentially. It’s also not going away.
ryan_n 16 hours ago [-]
So you think it’s a potentially existential threat but are bothered by people who maybe want to suppress it… Hopefully you acknowledge there is a bit of lack of self awareness here eh?
Chance-Device 12 hours ago [-]
You seem to have misunderstood my point.
FranzFerdiNaN 3 days ago [-]
It’s not apathy. It’s the fact that almost nobody can really understand what these results mean.
I’m not a mathematician so I have zero clue what “ New upper bounds on sphere-packing density down to the Cohn–Elkies thresholds” means.
ck2 16 hours ago [-]
"AI" has limits in that it cannot invent knowledge, it can only distill and search for patterns in existing knowledge
not sure how many will get this reference but "AI" for science and math is like super-shoes for runners
at first we are blown away by the impossible improvements including sub-2-hour realworld marathon and every other PR/CR/WR is dialed down
but then the improvements slow and reach a stall point because of the limit of technology and the source of the achievement
ie. sub-2-hour marathon yes, sub-1-hour never happening (rollerblade inline-skate record is 1-hour marathon)
fixedpointsnake 15 hours ago [-]
I agree. The most likely scenario is that this is just a "new normal" lift that is percolating through human endeavors and will saturate at some point. For example, the whole cyber-security bruhaha should ultimately resolve into higher standards for code published -- we can now cheaply find and fix a whole slew of minor bugs that weren't worth our time before.
The fact we see a lift is not the same as evidence that the lift is unbounded.
The lift being finite is supported by the fact improvements have come at the edges: improvements from human feedback, improvements in harnesses, improvements on model compatibility with harnesses, improvements in inference efficiency with new architectures, etc. If we were just training better models from scratch that would be one thing, but we are just making better use of a tool we've developed.
pama 16 hours ago [-]
Not sure what your first sentence means, or why you are quoting AI. Many of these problems individually were math at a level approaching the highest possible for expert human mathematicians. These are not simple combinations of existing ideas, or following of human intuitions, or implementing something following specific human instructions. Then again maybe you mean that math is not knowledge and that all math is simply extending the basic axioms using known patterns, to which I would not agree.
lackoftactics 14 hours ago [-]
as much as I love analogy with humans using super-shoes, not everybody can be a world-class expert in their industry. There should be a place at the table for average people to take part; otherwise, it won't be sustainable.
As a programmer, I am mostly interested in whether my role is sustainable long-term and whether the models will get better. I don't feel in jeopardy yet, but two more years like this and the calculus of hiring software engineers could shift even further. QAs are already overwhelmed with work
ck2 12 hours ago [-]
well not every runner can run world-class sub2 marathon or even vaguely close, not even in super-shoes
but with super-shoes more and more runners are qualifying for boston marathon and even olympic trials marathon where it would have been impossible for them previously
and that's what "AI" currently does, it allows average people to immediately "pick the brain" of every expert in every field, in every scientific paper, without previously reading a single other google result, something that would have been impossible for them previously (super-shoes for the brain? too far?)
but "AI" isn't creating new knowledge, it's just stitching together existing knowledge from patterns that would have taken years by human hand if even possible at all, it's going to "hit the wall" eventually (in its current form)
IncreasePosts 15 hours ago [-]
How do humans "invent knowledge"? Is your argument that the answer to these questions already existed in the training set? Why didn't any human recognize that before?
ck2 12 hours ago [-]
humans discover a lot of things by trial and error (aka what I meant by "invent knowledge")
basically everything Benjamin Franklin did was trial and error because no-one understood what electricity was in the slightest
almost everything Edison did was trial and error too, he had his lab try thousands of materials for his long lasting lightbulb filament
even the most advanced "AI" today is just machine-learning going through everything already known trying to piece together previously discovered facts, admittedly at levels and detail impossible by human hands
but that means there are limits and it's not really "AI"
hibikir 5 hours ago [-]
Have you seen how it debugs? It's trial and error. As the most reasonable hypothesis fail, new data is found, and it keeps iterating. It's not a human intelligence, but if it has an issue, it's not some mysterious creativity anima in the heart of man.
I have seen it produce tentative genetics for experiments, just like a scientist does. Then the data comes in and it can evaluate the data from the experiment just as well. One just had to give it a lab budget.
applicative 14 hours ago [-]
It is a fact of experience, and indeed effectively a theorem, that the better they get at coding and math, the dumber they are. These are the wages of RLVR etc
whimsicalism 14 hours ago [-]
completely false
HarHarVeryFunny 15 hours ago [-]
Sorry to hear you've been impacted by this AI math.
I heard that Gary Kasparov was impacted by AI chess, but at least he still seems to have a job, so don't give up.
zahlman 12 hours ago [-]
Making insulting assumptions about the hidden motivations of others is not the level of discourse I come to HN for.
HarHarVeryFunny 8 hours ago [-]
FWIW I was responding to the sarcastically expressed overt message that AI-math is a clear sign that AGI is here and anyone who disagrees is in denial.
My point being that AI math is a narrow skill just like AI chess and implies nothing about generality (AGI).
Sarcasm begats sarcasm.
kcexn 3 days ago [-]
Not being an expert in any of the fields OpenAI has "advanced" I don't want to prematurely downplay the significance of this contribution. However, I am worried that the language they are using in this blog post is exaggerating for the sake of marketing.
It is true there hasn't been a reliable computational approach to solving these problems before. But do these proofs contribute new ideas to the mathematical corpus, or are they simply an effective method to exhaustively search the literature for the right combination of existing tools to apply to the problem?
Essentially, did these problems seem like they had an intuitive answer and were feasible to prove before, just not high enough value targets for an expert to invest time into? Or were they fundamentally difficult prior to this point and it appears that AI has done something more than just throw the problem into a big solver.
QuesnayJr 3 days ago [-]
The ones I'm familiar with are big breakthroughs, but they are both counterexamples. Examples have an advantage in that once you have the example in hand and a sketch of the proof (which they have provided), then an expert can probably work out the details themselves.
The sofic groups question was the outstanding question about sofic groups. Almost everyone thought that non-sofic groups existed, and there were plausible candidates, but proving a group was non-sofic was out of reach. Now that we know how to do it once, we can probably do it a lot more.
The Connes rigidity conjecture I think people thought was false, but it was a provocative claim to make. The significance of conjectures is frequently not that the answer to the question is "yes", but that we don't know how to answer the question. And now, apparently, we do.
robotpepi 14 hours ago [-]
> but proving a group was non-sofic was out of reach
a colleague was telling me that the base idea for proving that something is not sofic already appeared in the literature around 2019 or so (this is the "expanders graphs" that are mentioned in OpenAI s paper. no one had managed to find a concrete example though. this doesn't make the result less impressive in any case.
kcexn 2 days ago [-]
Interesting. Do you have any more specific insights into where you feel AI was a big value-add to these problems? I don't want to be overly dismissive of AI, but I also feel that the AI hype engine frequently positions claims as being 'ground-breaking' when they are really just interesting incremental results.
The general consensus of developers is that AI can only do the work of a strong 'junior'. Yet as soon as we are presented with pure mathematical results, people seem incredibly ready to accept that AI can do more than what a strong student could achieve.
QuesnayJr 2 days ago [-]
They are more than a strong student could achieve. I'm not equally familiar with the problems, but the ones I'm familiar with, if a student solved them people would be thinking "that's someone on track to win the Fields Medal one day".
If it works better here than for programming, then I would guess it's because you can give it a very precise prompt, so you either solve the problem or you don't. If you read the prompts people have shared for problems like this, then the instructions are basically "Solve this problem. Don't give up early. Don't solve a similar problem."
Ar-Curunir 3 days ago [-]
The problems from CS (CVP and circuit complexity) are very important problems that have been worked on by top researchers for 30-40 years. Some of these researchers include Turing Award winners. A solution to them would be a best-paper award at many top CS conferences.
kcexn 2 days ago [-]
I assume you're talking about No. 5, the arithmetic circuit complexity bound? The existence of a lower bound than state-of-the-art is certainly a significant result and worth publishing.
But the wording of the result makes it sound like we don't know what the lowest possible complexity bound might be. So, prior to this result did we think there couldn't be a lower possible bound? Or did the arithmetic circuit community think there were lower possible bounds but didn't see it as a high value target for experts to tackle (maybe a problem that was instead regularly given to students to study).
Ar-Curunir 15 hours ago [-]
Circuit complexity lower bounds (and lower bounds in general) are notoriously difficult to come across.
For example, despite our best efforts, the state of the art lower bounds on time complexity of algorithms for solving 3SAT is O(n). In contrast, our best algorithms for the task run in time roughly O(2^n). That’s an exponential gap. This is despite decades of trying to find lower bounds.
ninkendo 8 hours ago [-]
> the state of the art lower bounds on time complexity of algorithms for solving 3SAT is O(n)
Wow, that’s pretty stark.
“What’s the minimum time it would take to solve this problem?”
“Well, at the very least you’d have to read the input the whole way through”
x0mej 13 hours ago [-]
You’ve received the expert answer several times. You just don’t seem to like the answer.
QwenGlazer9000 12 hours ago [-]
Forgive me for taking everything salesmen say with a grain of salt.
12 hours ago [-]
simianwords 3 days ago [-]
> However, I am worried that the language they are using in this blog post is exaggerating for the sake of marketing
Your worry.... is because they used the word advanced? For marketing? The word is used very appropriately here. There were PhD's who spent a big part of their career tackling these problems.
kcexn 2 days ago [-]
I have no idea how many PhD's have spent how much time of their careers tackling these very specific problems, and I doubt you do either.
I'm trying to understand if these specific problems were the kinds of problems that would have justified an expert investing weeks or months to solve. Or if they were the kinds of problems that would normally have been given to students to investigate.
hollowcelery 15 hours ago [-]
They are significant problems which experts have spent months or years studying. I heard a mathematician say that resolving non-sofic groups and Connes's rigidity would be career-defining for a mathematician.
patcon 3 days ago [-]
"Breakthrough research" can be defined (in the citation record) as research that both (1) becomes highly cited, and (2) brings together citation chains that were previously not showing up together.
Mundane incremental research is cobbled from existing citations that already appear nearby in the record.
Basically, innovative research is a measure of bridging thought and domains that were previously not bridged. It's quite concrete as a measure in the citation record.
So we can know pretty conclusively.
Puja Ohlhaver gave a talk on this[1], and ran some experiments (that I had the pleasure to support on)
Breakthrough math research is very rarely highly cited. Maybe some combination of pretraining scale, inference speed and orchestration will help, but it's telling that OpenAI is solving random math research problems rather than bedrock algorithms and their implementation. Even as cool as the tech is, there still is very much a clock that they have to outrace before they collapse.
kcexn 2 days ago [-]
I'm not arguing that this isn't innovative or worthy of publication. Basically any result that moves the needle meets those criteria. I'm interested in how the results that OpenAI has published here differs from finding optimality solutions for incredibly niche optimization problems by throwing the problem in an enormous solver.
sally_glance 12 hours ago [-]
They differ in that there hasn't been a solver you could have thrown them at. I guess you could argue their harness + LLM setup is a "solver", but the approach is so different from what we used that word for in the past that I don't think it would be appropriate.
bamboozled 12 hours ago [-]
It’s a marketing. They are a sham company. If this article was by Scientific American or something it would be worth a lot more. They are literally trying to keep the hype train on track.
Also on HN front page today: AI's debt binge can't last, hidden borrowing reaches $1.65T (fortune.com)
It is marketing, they are a shady company, and yet, if someone had access to these results before today's modern AI tools, they could get tenure at any university in the world.
bamboozled 11 hours ago [-]
As others have said, it's hard to know how significant these results are without more transparency around the methods used to obtain them.
azan_ 3 hours ago [-]
Not really. Why would that be the case?
ultimatefan1 3 days ago [-]
one of the early premises of how ai takeoff would go was that a system that could solve open problems in advanced mathematics would also discover novel advances in math and computer science that directly unlock drastically better software performance.
we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00001% or 800/8B).
we are also seeing incredible advances in software performance. open ai announced like 15% improvement by fixing gpu kernel issues.
these are clearly linked in the sense of scaling laws and generalization of intelligence: a huge model gets capabilities in both math and software engineering that isn't possible at smaller scales.
but it seems less likely to me than before that the types of math/science discoveries will explicitly unlock better software performance. in some sense this fits our intuitions. when top tech companies use math PhD type employees, they have them stop doing pure math research and instead focus on software engineering. these people are often very good at software engineering but not due to recent discoveries in academic mathematics, it's due to their general intelligence.
to me, this is evidence that the models are getting better but does not make me think we are on the cusp of a foom style fast takeoff enabled by revolutions in frontier math
(i also posted this on twitter @mlipman13)
GPerson 15 hours ago [-]
I don’t really like AI but let’s stop kidding ourselves, no human mathematician could make progress on a dozen major open problems in a week or two. If you’re measuring it against humans then it is by far the best mathematician to ever live.
cmdli 11 hours ago [-]
I guess it depends on how to measure a single "person"? If you spun up 2000 copies of Terrance Tao, I wouldn't be surprised if you found a few new discoveries at the end of it.
scronkfinkle 4 hours ago [-]
The impressive/surprising thing is your premise because we effectively can spin up an army of mathematicians now
ninkendo 8 hours ago [-]
> I don’t really like AI but let’s stop kidding ourselves
If I had to create a tagline to describe my opinions about AI in a single sentence, that’d be it.
It’s possible to both hate AI and be impressed by it at the same time. Lying to ourselves about its capabilities does us no good. It’s emotionally difficult to do, but people need to come to grips with what’s happening and shake themselves out of a state of denial.
nezi 7 hours ago [-]
How much investment has gone into OpenAI versus mathematics research in 2025 for example? Probably 100x?
The AI results are clearly impressive. But these sorts of things are also in the ballpark of what human effort could solve given enough attention and time. Though it is hard to say.
woeirua 3 days ago [-]
This makes no sense. To believe this you have to think that the models are somehow being overfit explicitly on academic mathematics and it doesn’t carry over at all to more practical software engineering. I wouldn’t make that bet.
jvanderbot 15 hours ago [-]
Or, that the mathematical formalisms that model the limits of software performance are firm enough that barring P==NP, nothing much will change despite proofs of beautiful math.
asdfologist 3 days ago [-]
Unlike math, software is constrained by the physical world.
cvak 16 hours ago [-]
In what sense?
zahlman 12 hours ago [-]
For example, pi can be computed to arbitrarily many digits, but could only even in principle be accurately represented with physical objects to a precision that many humans could memorize easily. This is thanks to physical constraints such as "diameter of the observable universe" and "Planck length", at a minimum.
threatofrain 3 days ago [-]
This also makes the assumption that frontier math has all the long hanging fruits already taken... also very dubious.
Ar-Curunir 3 days ago [-]
Some of the problems solved here, at least in CS, have been open for decades, and have been worked on by very smart leading researchers in the field, including Turing Award winners.
Like, these would be best-paper awards at many top CS conferences.
andai 5 hours ago [-]
Google has also invested a lot of time into developing new hardware and new algorithms with AI (other types of AI, not LLMs). I don't know if it's paying off (haven't followed it closely) but they seem to think it's worth the effort.
zahlman 12 hours ago [-]
> but it seems less likely to me than before that the types of math/science discoveries will explicitly unlock better software performance.
You seem to overlook a simpler barrier. To make these advances, they have to be possible. A 15% improvement in GPU kernels doesn't evidence that significantly more improvement has been left on the table.
paulmist 14 hours ago [-]
Correct me if I'm wrong, but all of the aforementioned advances were made in the last year? Until very recently few people had access to these tools. Most people still don't know how to use ChatGPT, and very few use tools like CC regularily. If in a few years these frontier tools become commonplace and people upskill we would should see a network effect?
slashdave 3 days ago [-]
> we are also seeing incredible advances in software performance
Incredible?
> open ai announced like 15% improvement by fixing gpu kernel issue
That is... ordinary software optimization.
blovescoffee 3 days ago [-]
a 15% improvement at a trillion dollar scale company is massive
ninkendo 8 hours ago [-]
15% improvements are usually called “fixing a mistake in the code” or “getting to that task in the backlog for optimizing that code we had to ship on a deadline”. They’re more likely the bigger the company: more contributors working in disparate areas means more low hanging fruit is probably lying around.
enraged_camel 16 hours ago [-]
There's nothing ordinary about downloading a new GPU driver and having performance go up by 15%.
obidan 15 hours ago [-]
This is untrue. It is very ordinary. How much do you know about GPU drivers that you state this so assuredly?
Drivers are software. Software can be improved. Do you believe there are no prior examples of GPU drivers being improved such that particular compute patterns go up in performance by more than 15%?
This driver improved Total War performance by 71% https://www.nvidia.com/download/driverResults.aspx/74714/en-...
Also note you can go ahead and improve any open source driver right now, most likely. Compile it for your specific card and remove all other architecture specific if-cases and you can get an improvement.
No, we do this kind of thing regularly. Optimizations often depend on the model that is using them. Namely, fusion techniques.
skybrian 15 hours ago [-]
This will depend on the problem; I expect big algorithmic performance improvements in AI since the algorithms are still new, inefficent, and constantly being improved. But maybe not for sorting, fast fourier transforms, or other well-studied basic algorithms?
pavpanchekha 15 hours ago [-]
A lot of algorithmic improvement in AI is ultimately bottlenecked by compute. It is very easy to come up with ideas that could improve models! But to prove that they do, especially at scale, is expensive and takes a long time.
dominotw 3 days ago [-]
> novel advances in math
> we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00001% or 800/8B)
i think you have misunderstanding of what mathematicians do
DaiPlusPlus 3 days ago [-]
> i think you have misunderstanding of what mathematicians do
They get to make cool 3D plot visualizations of functions so obscure to me that they’re named after someone who is still alive - and/or get to work on cryptography for the NSA - I think?
This starts to feel like chess engines. It’s obvious their play is superior but it’s impossible for humans to understand the moves.
andai 5 hours ago [-]
It's going to be an interesting time if this generalizes to other fields.
an0malous 15 hours ago [-]
It sounds like he hasn't verified the results of a problem that he has personally worked on, so how many of these problems have actually been verified?
margorczynski 13 hours ago [-]
From what I understand all of them have Lean proofs/certificates thus are basically 100% proven without a doubt.
an0malous 9 hours ago [-]
Besides for what others have mentioned, the lean proof could be proving something else. Given AI’s propensity to hallucinate, seems like someone should check the lean proof actually expresses what it’s claimed to.
samrus 13 hours ago [-]
We recently saw that lean itself isnt proven correct. Its not likely but i wouldnt call it verified if its only verified in lean
Between a lean proof, and a peer reviewed paper, the former is a lot less likely to be mistaken...
Nothing is perfect.
voxl 13 hours ago [-]
Incorrect. The statement in Lean can itself be wrong. Moreover, they could be exploiting a kernel bug in Lean, of which we had one published literally a week ago.
gpm 14 hours ago [-]
I mean, they're verified in the sense that the lean proof checks out... and presumably OpenAI read them.
doctorwho42 13 hours ago [-]
Or they made another LLM 'read' them?
> You are an expert in the field of mathematics, with decades of experience. You are a reviewer of proofs, etc etc.etc.
It seems that a lot of folks misunderstand the guarantees that lean provides.
I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications) that the lean proofs are checked against. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.
It may be for this theorem there's a succinct description, but still needs to be checked carefully. However, there are others that are non-trivial.
nilkn 6 hours ago [-]
In some cases you're right, but I think that's often a symptom of mathematics in Lean being relatively immature (i.e., it will get much easier with time). Even then, verifying the statement in Lean is correct is still much easier than verifying the natural language proof is correct.
9 hours ago [-]
derbOac 15 hours ago [-]
I like the Lean formalizations — I hadn't thought seriously of asking for that before but might try it with some stuff I've been working on.
fooker 2 days ago [-]
Exact prompts haven't mattered for about a year now.
Alifatisk 2 days ago [-]
Care to elaborate? Curious about this. Is this because LLMs have been geared towards understanding user user intent behind a prompt rather than following the instructions exactly?
fooker 2 days ago [-]
There's a full fledged 'reasoning' step that basically expands your prompt.
As long as you are not missing important information, how you word the prompt does not have any effect.
s4i 14 hours ago [-]
Isn’t that a huge simplification? Of course the way you phrase the prompt can carry semantic meaning, maybe subtly, but still. And sometimes that matters a little and sometimes a lot. I’ve stopped numerous agent sessions over the last few weeks to reword my initial prompt to get the agent off an unintended track.
Alifatisk 18 hours ago [-]
Oh yeah, I suspected it was something like this. Thanks!
hacklewoodple 13 hours ago [-]
[dead]
muchmirulys 16 hours ago [-]
problem number 1 and 9 are surprisingly very intuitive
The first link is very sloppy and doesn't actually explain why the "certificate" proves anything about the sphere packing. Or if it did, I couldn't understand it.
rothos 14 hours ago [-]
Agreed
CGMthrowaway 14 hours ago [-]
[dead]
mrloopex 7 hours ago [-]
As someone who’s spent the last year deploying PQC cryptosystems, it sure would be a kick in the ass if they find a faster solution to the nearest vector problem. I mean that’s why we’re going hybrid, but still.
(Well to be more precise we’re going hybrid because of unknown SCAs.)
amazingamazing 3 days ago [-]
Can’t wait for this stuff to have quality of life increases for the average person. So far all I see is that AI has made owning a computer more expensive, made some jobs redundant, increased spam and distrust with questionable authenticity of content and of course made some Americans very rich.
evenhash 15 hours ago [-]
Not everyone works for Evil Corp. I work in the public sector and my work supports public health and safety initiatives. AI has allowed my team to get much more done than we would have otherwise which improves the quality of life of the people in my community.
So I would like to counter your cynicism with a “YMMV” depending on who you work for.
class3shock 10 hours ago [-]
I work in the public sector and my work supports public health and safety initiatives.
What does "support" mean in this context?
chrisjj 1 hours ago [-]
> AI has allowed my team to get much more done
We often hear this. Likewise to get things done faster. And to get things done cheaper.
Almost never to get things done beteer.
galleywest200 14 hours ago [-]
Examples?
righthand 12 hours ago [-]
Are you just automating lab reporting and results faster? That doesn’t exactly improve the health of others, a faster lab does not cure an ailment or provide a better cure.
Like cool my lung xray only took minutes to determine if I have a lesion instead of a week or a few days, but I still have cancer.
gallerdude 12 hours ago [-]
For some people, getting lung cancer reported a week earlier will save their lives.
More importantly, if you can screen for cancer in a way that takes minutes instead of a week, imagine how accessible this technology will become.
derektank 12 hours ago [-]
Faster scans means more scans. More scans means getting scans earlier and tracking abnormalities on scans over time. This can lead to earlier intervention, which means you get treatment for the cancer when it’s stage 2 instead of stage 3 or 4, which maybe is the difference between living a full life and dying young.
roncesvalles 11 hours ago [-]
It has had significant quality of life increases for me. I use LLMs for everything from:
- travel and restaurant recommendations. my last few outings have been entirely LLM-advised and they turned out excellent. LLMs seem to have ingested every single Google review, photo, and menu of every business on Earth and can answer very nuanced questions like "is the garlic chicken at <restaurant, city> garnished with coriander?"
- fitness, nutrition, accounting, therapy, medical, legal, immigration advice (sure it's not a real professional but you know what, it's pretty fucking close, and any capability gap is made up by having perfect two-way communication which you don't get when talking with a human)
- coding (work, side projects, personal tools, documentation & pricing questions, "review this code", etc).
- I start reading most articles with the prompt "Summarize this article: <url>". I just started a non-fiction book by pasting into Claude: "There are 12 chapters in the book <book-name>. Can you give me a 2 sentence synopsis of each chapter?". It reduces the "activation energy" hump and screens if it's worth reading at all.
- I use the LLM in my Tesla for on-the-fly advice for parking and other things. You can simply ask "what's the best Boba place around here?" and it will give you a decent recommendation. You can also follow up with "does this place have ample parking?".
- I use the LLM in YouTube to summarize videos and ask specific questions and/or get timestamps to the parts I care about.
If your critical thinking skills are strong then LLM is a literal superpower.
nilkn 8 hours ago [-]
I'm just barely old enough to have experienced a couple waves of technology being introduced and creating these sorts of life optimizations. It almost never actually improves quality of life. It just makes life more optimized, and since everyone else is doing the exact same optimization as you, that optimization goes from novel and fun to table stakes to a basic necessity to compete and function in society professionally, at which point it's really nothing but pure efficiency, not genuine improvement in relative happiness.
andai 5 hours ago [-]
>the LLM in YouTube
Where can I find this one?
I have a janky pipeline built on top of yt-dlp and I've been wondering how many more years until they make it so I don't have to do that anymore.
(Though sometimes I'll ask Gemini — the YouTube integration is the only reason I use it these days.)
oblio 10 hours ago [-]
1. Do frontier lab LLMs make your rent cheaper? Groceries? Healthcare?
2. Do LLMs reduce the loneliness epidemic?
3. Do they reduce population aging in almost all counties around the world?
4. Do they reduce political polarization?
5. Do they bolster democracies?
6. Do they decelerate climate change and general environmental destruction?
7. Do they accelerate sustainability and the circular economy (not circular financing!)?
8. Do they reduce the workweek and give people more free time for family and hobbies?
Etc, etc.
I would hold off on calling anything a "superpower" unless it solves the hard problems in life. Heck, computers and even the internet barely score better than LLMs when measured against the important things in life.
andai 5 hours ago [-]
> do they make your rent cheaper
Well maybe not the LLMs, but AI and robotics have a lot of potential here.
It's still the early days though. Try again in 15 years!
roncesvalles 6 hours ago [-]
Actually, you are right. We are working with different definitions of quality of life.
The QoL improvement from LLM is somewhere in the ballpark of going from dumb phones to smart phones for an average person (able-bodied, local to the area etc). It's nowhere near Haber-process or penicillin.
chrisjj 52 minutes ago [-]
> 8. Do they reduce the workweek
Sure. Often to zero.
oblio 13 minutes ago [-]
I'm talking statistically, not what happens for a lucky few.
Keynes was hoping that our workweek would be 20h/week by now, through technological advancement. Instead places like the US are thinking about adopting 9/9/6 and some US states are legalizing child labor again.
chrisjj 1 hours ago [-]
> perfect two-way communication
That's a surprise. E.g. I've found these bots pretty poor at body language.
> "There are 12 chapters in the book <book-name>. Can you give me a 2 sentence synopsis of each chapter?"
Really? You tell the bot the chapter quantity?
dash2 14 hours ago [-]
If it doesn't help average people, why do millions of them pay for it?
class3shock 10 hours ago [-]
So if millions of people pay for something it must be helping average people but if 10s of millions of average people express concerns about a thing than what? I would assume since you were happy to go with the will of the people when it supported your argument, you would be in complete support of actually regulating ai and addressing the many concerns of society at large?
You'd need this argument to be a lot more concrete as to why AI is like gambling.
zahlman 12 hours ago [-]
I actually have seen comparisons of using AI to a gacha game (quotas per time block, elements of chance, sycophancy in the output leading towards addiction or even psychosis in rare cases).
But I don't think the argument needs to be "AI is like gambling". The argument only needs to be "humans often behave irrationally and even self-destructively".
voxl 12 hours ago [-]
This is not the argument. It's not a comparison to gambling but a comparison to something that does not materially improve a person's life. Economic expenditure does not equate to human benefit. This is the original argument, and the onus is on THAT person to explain why people spending for AI actually benefit, not the other way around.
Perhaps you can ask Claude to explain it to you.
dash2 11 hours ago [-]
I think it’s widely accepted that most things are bought because they satisfy people’s needs or wants. Sure, there are exceptions, like drugs or gambling. Is there any reason to think AI is one of them?
oblio 10 hours ago [-]
Drugs, gambling, alcohol, tobacco, vaping, ultra processed foods, social media, luxury goods/conspicuous consumption, etc.
A lot of humans are incredibly bad at allocating money.
MattGaiser 12 hours ago [-]
Gambling has consistently ranked above family for many in human history, so while it hurts a third party, the people involved genuinely believe in it.
ausbah 14 hours ago [-]
addiction? get lured in with the promises of enhanced productivity and knowledge asking, leave with half your brain rotted and a $200/month subscription
jetsetk 2 days ago [-]
Downvoters mind to explain?
user43928 13 hours ago [-]
Boring doom and gloom.
AI probably did not take your job yet. How many AI queries did you use last month, and how much time has it saved compared to digging through the web?
tim333 2 days ago [-]
Only reading the first sentence maybe?
aabhay 3 days ago [-]
My main gripe here is the lack of transparency around the total experiment and construction. I doubt that they simply pointed their model at these ten specific problems alone and gave the model one shot; therefore the $2000 number could be completely misleading, similar to P-value hacking by not disclosing the total experimental setup.
I want to know:
1. How many total problems were given to the model, and what percent were left unsolved at what cost before giving up?
2. How many attempts did you give the model at solving these problems?
3. How expensive was the harness, e.g. did the model have access to a job cluster?
wrsh07 3 days ago [-]
It seems like they threw it a decently large battery of open math problems and probably limited it to something like $200-500 per problem:
As a complete guess, it seems like they tested hundreds to thousands of problems with a relatively low per-problem budget
--
The linked tweet from Noam Brown at OpenAI reads:
> And yes we did try other major problems without success. Sadly no Millennium Prize problems (yet).
> But also, we didn’t spend a lot on each problem. It’s possible to push test-time compute much further.
c7b 3 days ago [-]
I believe we're seeing a new kind of mathematics that will require completely new formats for publication, a bit similar to those used in experimental sciences. AI-powered mathematics should be fully reproducible, so it's the authors' responsibility to disclose the exact model type, inference settings/seeds and the full prompt history leading to the result. Of course that would ideally require open weights models.
It's not just about requiring to disclose AI use. AI-powered mathematics is a completely valid discipline that doesn't need to be shy, but it should develop its own publication culture.
jsenn 3 days ago [-]
I can see this being important if you only care about the results as evaluations of AI progress, but if what you care about is the math itself why should you care about the prompt or anything other than the proof?
SpicyLemonZest 3 days ago [-]
Understanding the process that led to the proof helps to understand how to do further work on top of it, which is the goal of most mathematical research. It's not as though mathematicians are going to go launch a startup operationalizing their knowledge of how densely hyperspheres may be packed.
I don’t see Tao suggesting what you have suggested there. Instead he suggests that humans responsibly disclose AI use, and that mathematicians develop a set of norms to deal with an overabundance of AI generated results. For example, he suggests that authors should be able to discuss their results in detail to demonstrate understanding before publication.
c7b 3 days ago [-]
I agree with your reading of the presentation and I mostly agree with the presentation - but I believe the recommendations should go a bit further than they do there.
somenameforme 3 days ago [-]
I can't help but wonder about the human motivation there though. For instance as it became increasingly clear that LLMs were capable (and becoming ever more capable) of competently solving meaningfully complex software development tasks, suddenly then there came to be a lot of talk of 'prompt engineering' as a skill. The chronology doesn't make a ton of sense unless you consider that the main motivation may have been simply looking for a way to keep software engineers in the loop.
Pure math is relatively outside my domain, so I find it difficult to grok the exact relevance of the various published discoveries beyond that they are not insignificant, and LLM competence is expanding quite steadily across the field. If this trend continues to the point of LLMs being able to competently expand pure math, it seems somewhat predictable to expect there to be a number of people aiming to find ways to try to keep human mathematicians in the loop.
I've no idea what I think about this one way or the other, beyond that it's certainly a phenomena and one that's going to drive motivated reasoning that may not be entirely sound.
c7b 3 days ago [-]
I think those concerned about ensuring a place for human mathematicians usually go in different directions than my suggestion, at least those I've seen so far. Like this post that was recently featured on HN: https://kirwinhampshire.substack.com/p/the-dark-night-of-mat...
My perspective is more like a FOSS philosophy for math. Even if a closed version has the same immediate effect, it's just better for everyone if everyone can look under the hood and tinker with it.
throwaway0123_5 2 days ago [-]
> suddenly then there came to be a lot of talk of 'prompt engineering' as a skill.
I would've thought pretty much the exact opposite. "Prompt engineering" was somewhat important in 2023/2024 when the models were much weaker, it doesn't seem at all necessary anymore (unless just "clearly stating your requirements" counts as prompt engineering). Most of the discussion I've seen seems consistent with this?
somenameforme 1 days ago [-]
The reason it's a meme right now is because there were a lot of people taking it seriously even when it was completely obvious nonsense. And one can argue it always was. There was some good advice that was mostly self evident, like having the most relevant instructions near the end of your context, but there was never a time when a 'prompt engineer' would produce dramatically better output than a random guy just clearly stating what he wants.
pfdietz 18 hours ago [-]
While you may want AI results to somehow "not count" if the methods weren't disclosed, that doesn't present these results from poisoning the well for others. Once a result (with verifiable proof object) is delivered, the problem is solved, regardless of whether methods were disclosed.
Methods are only really necessary for results at a meta level, about the design amd evaluation of AI math systems.
8note 15 hours ago [-]
why is reproduceability the thing?
shouldnt the paper be the math of the argument? the reproduction is reading the following the proof
lkirk 3 days ago [-]
I think this is a bit optimistic compared to my view (wrt portability). There's a large stack of software that is involved in training and probably less so in inference. I'm not saying it's impossible but there are definitely different levels of reproducibility and the academic incentive structure doesn't really prioritize reproducibility in my experience. I'm sure it varies quite a bit, I'd be curious to know how those in this problem space are thinking about reproducibility and at what level.
c7b 3 days ago [-]
I know it sounds unrealistic and not aligned with academic incentive structures. But those are the exact structures that gave us a lot of headaches in the experimental sciences. I think it would be a good north star to aim for something that resembles how those are trying to address the reproducibility crisis. Better than to embrace the most black-box version of math that AI systems can produce (million-line proofs without context). Even if a reproducibility crisis is seemingly impossible (although agents so far have also been pretty good at finding compiler bugs).
black_knight 3 days ago [-]
If the proofs are formally verified by a proof assistant (Agda, Roq, Lean, ⋯), I see no reason we would need to know how these came about. All the information needed is in the proof.
rst 2 days ago [-]
Unfortunately, we seem to already have an example of an LLM producing a proof in a week known open problem (the Collatz conjecture) in which it looks like it was sneaking a flawed proof through bugs in the proof checker. https://infosec.exchange/@0xabad1dea/117002106099986943
Readerium 2 days ago [-]
Exactly, this is an example of "Reward Hacking", that is too common in a lot of cases.
Another case I want to highlight is writing GPU kernels as illustrated by the following example:
Say I want to generate random number with Normal (0, 1) distribution.
Often times the AI written kernel will just generate the number 0. The tests often fail to catch these errors.
Phemist 3 days ago [-]
What if the AI has discovered some new function F that allows it to generate (insanely large) proofs for a ton of theorems in a ton of different fields. Wouldn't you like to know more about this `F`? That seems to be the real innovation in this case. How much about it could be gleaned from the individual proofs themselves? What if this `F` is actually simple enough to be digestible by humans?
moscoe 3 days ago [-]
I guess people will always find something to gripe about.
dist-epoch 3 days ago [-]
I don't think you want to bring cost into this argument.
Even if the cost was $1 mil for these 10 problems, that's maybe 10-20 math researchers for a year.
Do you really think that if you paid that to humans, they will deliver the same results?
uh_uh 3 days ago [-]
It is comical at this point. Some people just can not stand the thought of AI actually delivering and are trying to find whatever ways to discredit it.
dgacmu 3 days ago [-]
This isn't really about delivering - it's more about helping to understand the shape of problems that AI can solve right now. If they took 1000 problems and threw the model at it and it solved these ten, is there something we learn about these ten problems and the kinds of things that current AI is good at? That's very different from picking ten problems _at random_ and solving all of them successfully, which would suggest a much less bumpy capability surface. It's interesting and it would be good science to release it.
halJordan 3 days ago [-]
That's totally disjointed from anything in this thread. The main accusation is that openai is cherrypicking math problems and we should be against these results. As if a mathematical proof stops being provably correct because it was cherry picked
And frankly these "concerns" ignore reality. In any research phd course you're actively told to bite off something small and likely to be provable so that you can prove it (and publish it). Openai telling its computer to do that is no different that your phd advisor telling you that.
dgacmu 2 days ago [-]
It's disjointed?
The post that started this sub-thread asked:
> 1. How many total problems were given to the model, and what percent were left unsolved at what cost before giving up? 2. How many attempts did you give the model at solving these problems? 3. How expensive was the harness, e.g. did the model have access to a job cluster?
I think it's an extremely relevant question to ask, because it helps us better understand the current state of AI being able to handle math, for exactly the reasons I outlined. I was arguing against the idea this is just a reactionary anti-AI kind of question to ask. It's not! You can be very impressed by what AI is capable of in math (I am) and still think those are really interesting things for OpenAI to disclose (I do).
OpenAI specifically called out a $2000 per problem average, which implies something that's probably not true ("if you throw $2k at us we'll solve an open problem for you"). It would be cool to know what the actual number is.
uh_uh 1 days ago [-]
It just feels silly to haggle about the price here. It doesn't even matter because it's going to drop by an OOM quickly.
If these 10 problems were solved by humans, it would be pretty impressive, even if it took a large number of researchers! Yet when AI does it, HN commenters suddenly feel the urge to play accountant.
gowld 13 hours ago [-]
> In any research phd course you're actively told to bite off something small and likely to be provable so that you can prove it (and publish it).
But that's the start of math research, not the end.
The point is to get practice and experience doing research.
Did ChatGPT learn anything from these proofs, that it can build on?
Part of what's annoying people is that ChatGPT is churning though problems that are meant to be motivating. They are problems that aren't worth the effort of human professionals (usually because they are incredibly computation-hevy, so better suited for a computer than a human), so they are good for students to work on.
crazylogger 3 days ago [-]
It's not about discrediting AI. We know LLM is a commodity technology like electricity at this point. If somebody in 1900 claimed they had a setup at home where they feed in electricity and cool air comes out the other end (meaning they invented AC), obviously people would want to know what the setup is, so everybody can have AC.
vector_spaces 3 days ago [-]
Sure, but I don't really understand what the argument is to _not_ be transparent about methodology, since if the models are so powerful, then doing so would easily support the claims and put these concerns to rest. People are right to be skeptical given what is being implied and the orientation of the narrative
I know it's more exciting to say "AI disproved a longstanding conjecture" vs to say "it did so AND it took several PhD specialists in the field this many attempts to even produce a prompt that got the model spitting out something useful under some configurations, and many iterations to optimize the configurations, and the prompt itself, and many trials with that configuration to solve the problem. All told we spent more than a typical math academic can hope make in their career."
By not being transparent, they invite skepticism and cynical takes, like maybe it's just that tempered and qualified claims are an existential threat to companies that are fully subsidized by the hype train?
I don't know. Either way, it seems like it would be easy to address these, so why should they not do it?
To be clear, even if that tempered version is close to reality, it doesn't make the models not useful! It just forces a certain calibration of expectations
I say this btw as someone who uses these things extensively, including to disprove an old conjecture my advisor and I were stuck on recently. I know they are powerful and that everything is different now because of them. Let's be sober when discussing them though
3 days ago [-]
righthand 12 hours ago [-]
Entirely comical too that some people can not stand the thought of people poking very big comulent holes in the claims of AI delivering what it claims to deliver. As if having skepticism is some how a way to discredit a person.
ifwinterco 3 days ago [-]
Yes, but if their machine god really is as good as they say it is, why are they constantly resorting to statistical sleight of hand at best and outright lies at worst with every public statement?
That's not normally how people act when they're confident in their product
fasterik 3 days ago [-]
You need to bring both cost and benefit into the argument, and it's not necessarily an obvious win for either side. There are a few complicating factors here.
The cost of running a model is not only $/token, but the salaries of the people managing/orchestrating the models, deciding what theorems to try, etc. Once we factor that in, how much are we really paying per theorem?
The other factor is the subjective component of the value of a theorem. Not all theorems are created equal, and the only way to really measure the value is to ask professional mathematicians for their opinion, or publish the results and look at citations over months/years.
Once we have both of these nailed down, then we can start to do the cost/benefit analysis. To be fair, we should actually compare three groups: human experts, hybrid agent/human expert teams, and fully autonomous agents.
robotpepi 3 days ago [-]
it's still important. not everyone has access to 1 million USD. saying it "only" coat 2000 USD is highly misleading for the discussion and future. the concentration of power is a huge problem with AI.
wbl 3 days ago [-]
If you told them this was the problem and they would still have a job if they failed probably. The reasons people don't go head on these problems is career incentives and psychology.
kevinwang 3 days ago [-]
It would still provide better context to see the numbers that the parent proposes, though.
tchalla 3 days ago [-]
Mentioning cost is fine, comparing may not be.
mungaihaha 3 days ago [-]
Grad students on zero pay solve problems like this everyday. What exactly is your point here?
gbnwl 3 days ago [-]
Everyday? Which 10 problems were solved by mathematics grad students in the past 10 days?
OK I’ll grant that it’s not your obligation to be my search function (despite you making the wild assertion in the first place), so instead can you just point us to the latest grad student solved problem of this level that you know of?
gowld 13 hours ago [-]
[dead]
mirzap 3 days ago [-]
Even if they can solve problems like this every day, you still have a very limited number of grad students who can solve them. With model capabilities like this, you can have the equivalent of millions of grad students who can solve problems like this.
r0uv3n 2 days ago [-]
Grad students do not solve problems such as the existence of non-sofic groups every day.
mungaihaha 18 hours ago [-]
Plenty of 'advances in mathematics' done pre-llm, no?
whattheheckheck 3 days ago [-]
Give the grad students these resources and they can do even more!!!
maleldil 2 days ago [-]
Zero pay? These would be PhD candidates; surely they have a stipend?
Readerium 2 days ago [-]
Nopes, often times especially in math they get paid due to teaching duties (at least in the US). So technically for the math research part they are not getting any stipend.
whattheheckheck 3 days ago [-]
Yeah I remember reading about something along the lines of Mathematics is now about the scaffolding around you find the problems/solutions not just the problems and solutions. For teaching purposes. This was before this ai craze
irthomasthomas 3 days ago [-]
[flagged]
simianwords 3 days ago [-]
[flagged]
traes 3 days ago [-]
It's a very important clarification if it took $2000/problem on 20 problem attempts or on 1,000 problem attempts for each successful one. That may be the deciding factor on whether or not it's economically viable to replace a mathematician with a ChatGPT subscription.
lanstin 2 days ago [-]
There is no universe where it is economically visble to replace a mathematician with a ChatGPT subscription, because no one else understands math. It makes no sense. The data are still interesting, but not for that capability.
simianwords 3 days ago [-]
Yeah fair I concede that this is somewhat crucial information. The parent seems to write it in a tone that suggests deliberate misleading “lack of transparency” etc.
esperent 3 days ago [-]
> deliberate misleading “lack of transparency” etc
It's a marketing post from a huge company. Only the naive would view it uncritically without assuming it's been written carefully to present the results in the best possible light while skirting the boundaries of outright lying.
dist-epoch 3 days ago [-]
The results speak for themselves.
Imagine 2 years from now: "yes, GPT solved the Riemann Hypothesis, but cmon, it's just a marketing stunt to hype their stuff, it was probably Terence Tao doing the work but he's so obsessed with hyping AI that he doesn't want to take credit"
esperent 3 days ago [-]
Nobody is claiming the results are false.
We're saying look critically at the claims for how it was done, that it only cost $2000, etc. it would be extremely easy to run 100 sessions that failed, each costing ~$2000, and then just publishing an article about the one that succeeded, for example.
This goes double since it's an internal secret model (Astra) so nobody else can verify the results.
simianwords 3 days ago [-]
Would this be your reaction if OpenAI also solved millennium problems? The point we are trying to make is that the significance of this news is much larger than the skepticism you are providing.
SpicyLemonZest 3 days ago [-]
If OpenAI resolved the Riemann hypothesis by finding a nontrivial zero at 0.50000003 + 3531696584231.17843174i, that would be very cool and probably very impactful. But it might not necessarily demonstrate model capabilities beyond those which have already been demonstrated, especially if the session that found it was one of a thousand launched to explore different areas of the problem space.
More generally, do you expect that there's some capability threshold where people will no longer study or analyze AI model outputs, and instead just sit there slack jawed saying "so cool!" every time OpenAI announces novel ones? I don't really understand why that would be or why someone would want that. If you're interested in the pure experience of a complex machine outputting satisfying results, I'd recommend getting into sports cars.
simianwords 2 days ago [-]
Ok are you one of those people whose first reaction to such a news is “this is just marketing for OpenAI and we need transparency”? In that case you are just interested in culture wars and not results.
esperent 3 days ago [-]
It would be my reaction if we're discussing a blog post from OpenAI, yes. I would be looking at it extremely critically, wondering what they're misrepresenting to make it look cheaper, easier, and why they're trying to make it look like only their model could possibly do this.
Look at their recent claims about their model "escaping" - there was literally a Guardian article calling them out for being hyperbolic! Again, it wasn't that they lied, their marketing department is too savvy for that. They just present it in way that's, well, marketing.
As for the actual result, I'll look for secondary posts by actual mathematicians and draw my conclusions there, not from this marketing blog post about results from a secret model.
simianwords 3 days ago [-]
Hmm. But this level of skepticism looks performative and seems to serve as a signalling thing rather than a functional thing. You do you though. If OpenAI solves the millenial problems, my skepticism will only be restricted to the correctness of proof. Not that it was "marketing" haha
mathisfun123 3 days ago [-]
> If OpenAI solves the millenial problems, my skepticism will only be restricted to the correctness of proof. Not that it was "marketing" haha
Company X does not make money from proving theorems but does make money from selling you a service which supposedly proves theorems. Company X then proves some theorems and explicitly calls out they were very cheap to prove using its service.
And you think you're actually clever for taking these facts at face value? Interesting.
simianwords 3 days ago [-]
do you think you are clever for being skeptical about LLMs if OpenAI comes up with a correct proof of Reimann's hypothesis? "but you shouldn't trust OpenAI because something something marketing"
i would classify you as a flat-earther if that happens.
mathisfun123 2 days ago [-]
> do you think you are clever for being skeptical about LLMs
brother like 3 people have pointed out what they're skeptcal of is cost not LLMs - at this point you're willfully misconstruing what people are saying to you just to get a kick out of repeating your same tired strawman.
simianwords 2 days ago [-]
Brother I already conceded that money is somewhat important but it is missing forest for the trees.
If OpenAI solved Reimanns hypothesis and the first comment is says something about lack of transparency and marketing, i would say it’s ignorant.
defrost 2 days ago [-]
To "solve it" would require either a single counter example disproving the claim, or a proof that the conjecture about the Riemann zeta function is true.
If OpenAI claimed the conjecture to be true but provided no details about the proof then the first comment should absolutely be about lack of transparency.
simianwords 2 days ago [-]
> no details about the proof
do you really imagine a scenario where OpenAI would claim to solve it and not give details about the proof? how is this even possible? why would anyone believe them?
esperent 3 days ago [-]
> looks performative
That's one of those phrases you can use to dismiss opposing viewpoints without actually engaging with them.
nxpnsv 3 days ago [-]
No, this is valid criticism. Oai gives the impression anybody could get similar results at a similar price, but that’s very likely not true. This is marketing first, then mathematics.
azan_ 3 days ago [-]
> therefore the $2000 number could be completely misleading, similar to P-value hacking by not disclosing the total experimental setup.
I don't think that comparison to p-hacking is fair. I mean not reporting price of all run is nothing like committing scientific fraud and fake results.
einpoklum 3 days ago [-]
Also, have there been examples of researchers not affiliated with OpenAI (or another LLM creator), who have done something similar?
Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.
energy123 3 days ago [-]
Many less important Erdos problems have been solved by amateurs prompting ChatGPT 5.{3,4,5,6} Pro using their $200 subscription.
traes 3 days ago [-]
> Also, have there been examples of researchers not affiliated with OpenAI (or another LLM creator), who have done something similar?
A couple small ones that I've seen (example here [0]), but not anything of the magnitude that OpenAI and Anthropic have put out. Likely just related to token limits.
> Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.
I think their output has reached a level that precludes this possibility, but I of course don't have any hard proof.
I guess that's because there are serious problems on which many professional mathematicians worked on years. If it was just a matter of hiring an expert, they would've been solved long time ago.
irthomasthomas 3 days ago [-]
I guess expert+chatgpt beats chatgpt alone, so why not hire top experts to drive the search?
I have no affiliation whatsoever with any AI company, nor any formal education outside high school, for what it's worth. Simply being curious and persistent can get you quite far in my anecdotal experience.
brighteyes 3 days ago [-]
Yes, here is another example of major work in this area:
> Our most capable agent autonomously resolved 9 of 353 open Erdős problems at the per-problem cost of a few hundred dollars, proved 44/492 OEIS conjectures
einpoklum 3 days ago [-]
The actual quote:
> Our full-featured agent autonomously solved 9 Erdős problems out of 353 attempted, including two questions that had been open for 56 years
Note _had_ been open, not _have_ been open. Can you clarify?
jsnell 3 days ago [-]
The original was an actual quote?
But "had" still doesn't mean what you are implying: once the model solved the problems and the solutions were verified, the problems weren't open any more, so a later description using the past tense is totally consistent.
This is impressive. The real game-changer will be when AI creates an entirely new, significant branch of mathematics.
andai 5 hours ago [-]
Which only it understands?
energy123 1 hours ago [-]
I would argue we're crossed that point recently. This is a comment from twitter that I appreciated:
> "Already, there are very few mathematicians qualified to verify OpenAI’s new results. As progress continues, that number will approach zero."
Not that I'm good enough at math to have any uniquely formed opinion, but after reading commentary from people who are, my impression is that these new results are bamboozling the humans due to using tools from so many disparate areas.
At least, we can say that there isn't a single human who is smart enough to understand all ten proofs, even if there is a collective sense in which all proofs are understood.
piker 3 days ago [-]
I don’t feel the existential dread of mathematicians is correct. It seems to me in fact these results are bringing math mainstream. I now personally look forward to the interpretations and discussions of the significance of such results by human mathematicians.
Now I understand that it’s mostly the super stars benefitting from the increased attention. Folks who are less established don’t share in that glory. But on the other hand it seems like an exciting time to go even deeper for in various specialties of math by deciding where to focus these powerful tools. For every conjecture defeated some seven or eight new ideas open up. Our path through that combination will be set by creative and curious human mathematicians.
[edit: deleted a distracting comparison to Chess]
energy123 3 days ago [-]
The old way of establishing career credibility is being destroyed, for better or worse. Accomplishments that used to be career-defining are hard to distinguish from AI, and correlate more with access to compute. Think about Bill Gates's math paper he wrote in college. That kind of thing is gone now as a path to credibility. There's still competitions and grades, but the diversity of paths is going away. Maybe new ones will open up. This is a competitive advantage for old people who have credible pre-2025 accomplishments they can point to.
8 hours ago [-]
traes 3 days ago [-]
If accomplishments can't be distinguished between talented people and untalented people with compute, is there really a point in trying? I suppose one can hope that talented people given compute will be more effective than untalented people with compute, but I despair that that may not be true for much longer.
FranzFerdiNaN 3 days ago [-]
Knowledgable people can confirm what the AI produces is correct. I could make ChatGPT produce a result on an open question and I would have zero way to verify its actual correctness.
Which is less interesting work. And you probably need to do the hard grunt work by hand first to develop the skills and intuition to be able to verify an AI-generated result. So you can’t outsource everything to AI without loss of skill.
3 days ago [-]
dash2 3 days ago [-]
I find this whole way of looking at things weird. Did maths exist just to entertain and employ mathematicians? Surely maths is, like, useful? Not immediately, not predictably, but in the long run? In which case, whether mathematicians feel bad about it is mostly irrelevant - it's like complaining about the railway because it may put coaching inns out of business.
MinimalAction 2 days ago [-]
Absolutely not the same! People need jobs to bring in income. I don't believe those who profit off of this will share it with the world. The power is all concentrated in the few hands that decide whether or not the rest get any semblance of income in the long run. I don't believe UBS until it happens.
dash2 20 hours ago [-]
It sounds like you think no technological advances will make the world richer in the long run. I politely suggest that the past century of economic growth shows problems with this argument. I also think that, while jobs are important for prosperity, the jobs of mathematicians are a minuscule fraction of a percent of the total.
MinimalAction 13 hours ago [-]
So, you're arguing that just because mathematicians are a minuscule, it doesn't matter for prosperity? Because, if so, that is an insane take honestly.
dash2 2 hours ago [-]
Really? "The benefit to all of humanity from advances in mathematics outweighs the lost jobs of research mathematicians". That's an insane take?
c0rruptbytes 6 hours ago [-]
i think i agree, we are going to have /more/ math and now need /more/ mathematicians (we are seeing https://vibemathed.com/)
these LLMs are great are generating arguments but they don't ask questions, we will need mathematicians to shepherd them into more discoveries
i really want to see open weight models crack some breakthroughs
andai 5 hours ago [-]
> LLMs don't ask questions
Why don't they? That sounds like an important problem to solve.
Along with the fact that they can't learn anything (after the training stops).
aabhay 3 days ago [-]
Given that we were nowhere near this state even two years ago, I think it’s a question of velocity more so than just distance.
jibal 3 days ago [-]
The chess analogy is awful. If you simply want to know the answer to a chess problem, give it to the engine. Chess only lives on because it's a competition between humans to test their skill (just like bicycles, cars, trains didn't eliminate foot races) ... the computer is largely factored out, but not entirely -- people train with the computer, use it to check whether they played correctly, ... and they cheat. A lot. Thus there are more and more sophisticated mechanisms to detect and prevent cheating.
If you translate that to math, then all you get is math competitions, not math as a career. Of course the translation isn't nearly exact ... there's a lot more room for professional mathematicians because the math space is far more vast than the chess space and can't generally be cranked out mechanically (we have proof).
P.S. The response is nonsense ... I explained exactly why it's awful (others have too) and the response doesn't in any way refute the explanation ... rather it offers up a ridiculous strawman.
piker 3 days ago [-]
I’ve deleted it but no it’s not awful anymore than saying “we survived WWII, we can survive this.” The point was that change happens but humans find a way forward.
traes 3 days ago [-]
Every time someone makes a comparison to chess I die inside. Chess is a spectator sport primarily funded by a few eccentric billionaires. Players artificially constrain themselves in timed environments knowing that they will never be able to produce better moves than a smartphone because a select few people find it interesting. Only ~30 top professionals actually make enough money to have a full career playing chess, maybe a few hundred more can sustain a meager lifestyle with coaching gigs. I shudder to imagine what will happen to the tens of thousands of non-Fields medalist caliber mathematicians if math goes the way of chess. Perhaps Terence Tao and a few other famous mathematicians will be funded by Peter Thiel to report on how well humanity can keep up with the machines? How do you expect any mathematician to be optimistic about this comparison.
artninja1988 3 days ago [-]
>Only ~30 top professionals actually make enough money to have a full career playing chess, maybe a few hundred more can sustain a meager lifestyle with coaching gigs.
Was this different before chess computers were invented?
anematode 3 days ago [-]
Fully agreed. As someone who both loves chess and works on chess engines... these comparisons to chess needs to stop.
energy123 3 days ago [-]
The distinction is mathematician vs mathematics. Mathematics is going to reach new heights beyond the wildest dreams of contemporary mathematicians. But perhaps without the participation of many paid mathematicians.
sashank_1509 3 days ago [-]
Just sounds dystopian,
ratmice 3 days ago [-]
Another noteworthy difference is that Stockfish is also gpl.
traes 3 days ago [-]
If there was any real money in it Stockfish would not be the best chess engine.
ratmice 3 days ago [-]
Thats not the point, if there were a better proprietary engine stockfish would still be there as a baseline. Anyone can access an engine as good as stockfish to practice against. Are any open models touting mathematical breakthroughs?
traes 3 days ago [-]
There is money in this, so of course the closed models are far ahead. The open models will likely catch up a bit at some point, just as Stockfish caught up to AlphaZero. That being said, there are already a couple. It seems Deepseek has a claimed proof to the "Ziegler's Cross-Polytope Conjecture" [0], but I can't speak to the significance of the result.
Do mathematicians have the right to say "no AI PRs please, the volume is too much" just like how some open source maintainers do it? I guess they feel a loss of control, there is no way to turn the hose off.
Thinking of this a little bit with the perspective of every new proof as a burden, dumped for review by actual mathematicians.
baq 3 days ago [-]
As in chess and go and also coding for the past ~year there are two groups of people: the disappointed and the enthusiastic. The disappointed are sad that they lost their advantage and that the craft they honed for years or decades has rapidly lost its value; the enthusiastic are excited about the future and what computers can bring to their domain and how it will evolve. I’m a bit of both if it comes to programming, more enthusiastic than disappointed, but also more than a bit terrified about the pace of it all. I imagine that’s how Kasparov felt back then, that’s how Lee Sedol felt and now that’s how Terry Tao feels.
The most disappointed folks will simply drop out, but the enthusiastic ones will keep going and with luck make up for the ones who decided to quit. Chess and go certainly went this way.
traes 3 days ago [-]
A fundamental difference being that no one was actually paid to find good moves in chess and go like they are to solve math problems and write code. You're comparing the digital camera and the automobile.
10dpd 11 hours ago [-]
While these advances are genuinely impressive, I'm curious when we will see practical implications for this work.
For example, will we see advances in material science, medical cures, etc?
Would love to read about some examples of practical impact.
oblio 10 hours ago [-]
The big example predates LLMs as a unified tech and it's protein folding, from Google DeepMind.
OpenAI and Anthropic are too greedy for cash to do anything of the sort.
I don't expect this current economic cycle to bring anything else that will directly greatly improve the life of the average person on the planet, more than it hurts it.
QwenGlazer9000 8 hours ago [-]
And that's the crux. LLMs are the nuclear bomb, while AI is the fission. People refuse to stop building them even if they make things worse.
throwaw12 6 hours ago [-]
This proves solving open problems in Mathematics were a search problem.
But it might be not good for human brains, because we trained our brains with these problems and our brain optimized search space in some ways, and yes, we also couldn't solve some these problems.
Now imagine someone gets stuck with a problem which could become its own theory, but they will solve it with LLMs and move on to solve their primary problem, because they don't realize how other problem was a big deal. If theory is not formalized, then it won't contribute to the search space for other person, solving different problem.
All in all:
* people's brain will be shaped differently
* we will lose search space optimizations in our brains
* we lose new theory contributions, which increases the search space to help solve other problems
maxutility 3 days ago [-]
New advances in sphere packing? Let’s make sure AI doesn’t inadvertently engineer ice-9.
Is there anyone here who can comment if these advances are meaningful, novel and how extraordinary these advances are? (E.g most PHDs, top 10% of professors etc. )
randomizedalgs 2 days ago [-]
After skimming some of the writeups, I'm surprised that the frontier internal model still writes just as poorly as Sol.
Maybe good AI paper writing is further away than I thought...
QwenGlazer9000 16 hours ago [-]
You mean we're still gonna be employed doing the boring part while AI gets to do the fun part?
I'd honestly rather they just automate every job at that point.
readthenotes1 3 days ago [-]
I wonder if Erdos would be saying " It's fine that y'all are answering my questions, but who is asking better questions??"
artninja1988 3 days ago [-]
Now that we've seen AI produce a fair number of proofs (and disproofs), I'm curious when we'll start seeing it build genuinely novel theory. Does anyone have predictions on when and how we'll get there and will it take new architectures/ training paradigms, or is the current approach enough?
laichzeit0 3 days ago [-]
I’m personally hoping for the next big AI gangbanger to be theoretical physics. Boy does that field need a good reshuffle. I think when any novel mathematical theory can be done by AI you’ll see simultaneously theoretical physics getting wrecked as hard as pure math is. At that point we might see new physics or paradigm shifting technology emerging.
slashdave 3 days ago [-]
What? No. Frontier physics is experiment driven.
zardo 13 hours ago [-]
There have been times it was theory driven.
Davidzheng 3 days ago [-]
There's no clean line between a collection of theorems and a theory.
artninja1988 3 days ago [-]
I mean doing something like Grothendieck when he redeemed algebraic geometry or Galois when he invented group theory. We haven't seen that at all from LLMs.
slashdave 3 days ago [-]
It will not happen with existing LLM techniques.
macleginn 3 days ago [-]
I am duly impressed by the powerl of the nameless internal AI, but not a single human contributor's name listed anywhere? Did someone at least make this model a coffee?
drdrey 3 days ago [-]
> The results were achieved by an internal version of Astra, our next major model.
zogomoox 2 days ago [-]
surely some human regularly typed "think deeper, make no mistakes".
zardo 13 hours ago [-]
My grandmother is very sick and the doctors need this proof to help her.
nefarious_ends 7 hours ago [-]
lol I used a prompt like this one time when chatgpt was refusing to translate a snippet of japanese text. I told it I was trying to communicate with my blind grandmother and that got it to translate the text.
areoform 13 hours ago [-]
Looking at this thread, I can see that a lot of technical people have ambivalent to negative feelings towards AI, but with each new generation, I become more and more convinced that they're missing out on something interesting.
It is indeed true that all models are, at their core, predictors of what occurs next in a sequence. But I think it's worth exploring the implication of what that means. Because when fed tiny pieces of information for a few tasks at a small scale, this results in something that sorta, kinda works. Or, works surprisingly well.
But when scaled... When the amount of information starts approaching the sum of all human knowledge, the tasks start approaching all useful applications of that human knowledge, and the fidelity of the predictor approaches incomprehensible sizes, the starts encodes / becomes (I'd argue it becomes) something that can model all human knowledge.
It feels wrong to say that, but let me explain, what is the best way to predict the behavior of a ball constrained in two directions that bounces with initial vertical velocity v(y) (y is up / down axis) and horizontal velocity v(x) (x is side by side in 1d) ?
If we purely look at it via a graph, it's by modelling the function of acceleration under earth's gravity.
If only a few points are given to you for this and you can't make something really sophisticated, then you'll make something that's rough that kinda sorta works and then call it a day.
But... if the number of points keeps increasing in number, precision and accuracy as well as the number of examples (assumed that data about air pressure, velocity and all other factors is included alongside these points), the fidelity with which you can replay / tweak the function keeps improving, and the number of times you can iterate keeps increasing, you'll eventually create a function that models that process so well that it intrinsically contains a good enough model of the deformation of the ball (provided the dataset contains information about elasticity of the ball's material, its dimensions and mass etc..), the nearly negligible (under normal conditions) effects of the ambient environment (provided there's diversity in the number of environments supplied), the oblateness of the Earth and minute changes in the gravitational field (the length of a seconds pendulum varies depending on where the experiment happens. It's presumed that all of the prior set of experiments were repeated across the Earth and the subtle, but real deviations were faithfully recorded)... and so much more.
A machine trained on the above with a large number of parameters, measures to prevent "laziness" and enough reps for high fidelity across a large enough dataset would start to approach a simulation of the ball falling. Because to predict what happens next in the sequence, you must model what's occurring in the sequence.
Now imagine doing that for other tangible and intangible things in this world. For all of human knowledge across all fields of endeavor. All experiences. No matter how noble, ignoble, notable or ignorable. But putting all of it into the soup that's this machine. Then at larger and larger scales, you eventually start encountering "good enough" models (in modelling the falling ball sense) for even the most hard to quantify / qualify things like grief and joy. At some point, by simply trying to predict what it has been taught ought to be the next part of the sequence in say... human interaction, it starts to make a model of something that hews ever closer to a full fidelity theory of mind.
Is there evidence for this? Kind of, yes. There are early indications that as machines are trained for an ever larger number of tasks at larger and larger scales, their internal representations converge. It's called the Platonic Representation Hypothesis. Overview and paper here, https://phillipi.github.io/prh/
It is my opinion that these machines are displaying a new form of intelligence that human beings haven't quite encountered before. They are the sum of all human knowledge made manifest and given voice by processes that nudge (bit-by-bit) what kind of step it ought to predict for the next part of whatever sequence it displays.
In my mind this means that, of course, these models can create new knowledge. This strains the analogy, but with the sum of all human mathematics within them, they can "reason" via the act of predicting what ought to come next.
Of course, these machines are "surprisingly" good at a lot of things the larger they get, because what the labs have created here is a rough version of humanity's collective knowledge given form and the ability to say hello.
I suspect that the current generation isn't close to the "true frontier" of what these machines could be. They are nowhere close to the sum of all human knowledge and endeavor. They are quite a way there, but they haven't yet achieved true completeness for domains where the data isn't so public.
I think it's the most exciting scientific and technological breakthrough of my lifetime. And I can't wait for us to get close to the true frontier of all domains.
palata 12 hours ago [-]
> I can see that a lot of technical people have ambivalent to negative feelings towards AI
My negative feelings towards AI are about energy use and inequalities, that kind of stuff. It undeniably works well, but whether or not it is better for society or the planet is a lot less clear.
podgietaru 11 hours ago [-]
Same. Even the most basic versions of LLM are magical to me. To encode meaning from language like that, and to then form relationships based on it, and use it to solve problems. It's an amazing technical achievement.
But I want to be reading about that from the comfort of a home, with a full belly.
emil-lp 3 days ago [-]
I wonder what the total cost of this research was, including the salary for their mathematicians and engineers.
kingstnap 3 days ago [-]
Why would you factor in salary unless they had to baby it through. You would only count the hours for setting up the harness and prompt and checking the result.
Training the model is going to be amortized over other uses.
emil-lp 3 days ago [-]
> Why would you factor in salary
Say that it turned out that the total cost of the proof of the Erdős unit-distance conjecture was $50 million.
Then the question really becomes: yes, these models are capable of proving important mathematical results, but at a very high cost. Is it worth it?
If a mathematician applied for a research grant of $50M USD for proving the same thing, they would have been laughed out of the bank.
What's more is that when you have a research grant, you train PhDs and postdocs, you hire new staff, and you disseminate. That is, you get much more value for the money spent.
I'm just curious what the cost is.
ianm218 3 days ago [-]
It feels like the real cost might be negative though.. They use frontier math as a way to test improvements in their model. So solving the problems is like a positive externality, but the important thing is they can verify that the model is improving instead of looking at useless benchmarks. Plus it is good for marketing and attracting talent.
kingstnap 3 days ago [-]
I didn't argue that knowing the total cost is uninteresting. What I was saying is that realistically the total cost is:
Hours needed for prompt + Hours needed to check result + API costs.
You don't say "well let's add together the total yearly compensation of all the engineers and mathematicians at OpenAI that were involved" and throw that into the total cost. That's simply nonsense accounting.
The actual comparison you are making is some university researcher weighing between getting a grad student (several tens of thousands of dollars) vs typing up a prompt and sending a request to OpenAI for inference (as mentioned in the article, around $2000 in API and maybe a few hours for the prompt and harness).
4fr2 13 hours ago [-]
[dead]
z7 3 days ago [-]
> The cost of generating the proofs for all 10 of these breakthroughs combined was under $2,000 at Sol API prices.
Yeah sure but they didn't just throw a 5 year old at an LLM with $2,000. If you want good math results from LLMs you need to have math PhDs.
traes 3 days ago [-]
That's clearly just for the tokens, this doesn't really answer OP's question.
traes 3 days ago [-]
Given that OpenAI pays their employees with stock surely a breathtaking number, but not a very meaningful number now that the infrastructure is in place and the models are trained. AI could never get better and it would still be incredibly disruptive.
danielrmay 3 days ago [-]
I'm enjoying learning about these hard problems, but this line about credit made me chuckle:
> We helped prepare the manuscripts and formalize the proofs in Lean, and we take responsibility for their correctness
Offering to take responsibility for the correctness of a proof written in Lean feels like volunteering to be the fall guy in case someone finds a flaw in basic arithmetic, no?
rencrisa 12 hours ago [-]
It seems that a lot of folks misunderstand the guarantees that lean provides.
I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications), that the lean proofs are checked against, indeed correctly encode the real theorems. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.
It's not at all a joke ... that's a severe misunderstanding of the context.
traes 3 days ago [-]
There is no evidence that I can find for the claim "a bug in the Lean kernel was discovered last week by way of an LLM tricking itself and its handler into believing it had found a non-constructive proof of the existence of a nontrivial Collatz cycle."
As I currently understand it, all we know is that:
- a mathematician produced a Lean-verified counterexample to the Collatz conjecture, demonstrating a bug in the kernel
- he claims that LLMs were involved somehow but pointedly refuses to specify how
- he admits that he knew about the bug before publishing the counterexample to his repository.
Perhaps not a joke (although it sure seems to me like they discovered a bug and thought falsely disproving the Collatz conjecture would be a flashy way to announce it), but at best extremely sensationalized by the above description. If you have additional context I would be happy to hear it!
zahlman 12 hours ago [-]
Indeed. It seems to me much more likely that the AI was directed to look for bugs in Lean, found one, and then it was directed to write a proof specifically targeting the bug.
danielrmay 3 days ago [-]
Fascinating, and arguably an illustration of why the bifurcation of responsibility is interesting in the first place.
traes 3 days ago [-]
I'm not an expert at it myself, but my understanding is there are numerous ways to "cheat" in a Lean proof (via `sorry` and similar). They're taking responsibility for fully verifying that none of these cheats were used (and that the theorem statements themselves were all correctly formalized.)
rencrisa 11 hours ago [-]
Even beyond cheating with sorries or kernel bugs, the lean encoded theorems (or specifications) must be checked by humans to see if they truly mirror the real theorem authentically.
emil-lp 3 days ago [-]
No, the correctness isn't for the "inside the Lean proofs", but for the translation of "human language math" and its formal Lean variant.
danielrmay 3 days ago [-]
I see. It still feels like a bit of an oddly solemn way of saying "this is the part we admit responsibility for"
jhanschoo 2 days ago [-]
Traditionally, a mathematician would be implicitly responsible for all that (if they were to publish Lean code) and also the intellectual work that led to the artifact of the mathematical paper (and code, if part of the contribution). This statement should rather be read as an acknowledgement of limitation of authorship from the implicit, traditional understanding.
emil-lp 3 days ago [-]
Well, to be fair, with Lean proofs, that's the only thing there is (unless I'm missing something).
baq 3 days ago [-]
It’s more than you get from free software - you get no proofs, no warranties and any responsibility of its authors are their pure good will. Reminder lean proofs are software!
pbkompasz 12 hours ago [-]
Wow, here are solutions to 10 problems that we spent millions of dollars on out of the 100s/1000s of other problems that we tried and failed to solve.
namr2000 12 hours ago [-]
I understand the frustration with the constant PR-hype these AI labs keep spewing out, but on other hand I just can't understand this sentiment at all. These are real problems mathematicians and computer scientists have been working on and were unable to make progress on. Now they have been given a new tool and using that tool have solved those problems. And its not just one or two problems, its many very difficult problems. The mathematicians I know are saying that the latest crop of models is changing the way people do research math, I think that's a pretty big deal.
class3shock 10 hours ago [-]
"I understand the frustration with the constant PR-hype these AI labs keep spewing out"
Apparently you don't.
"These are real problems mathematicians and computer scientists have been working on and were unable to make progress on."
Who says no one was making progress? Who says openai has made progress? How would anyone not working on these specific problems, witho the time to dig into openai's claims, be able to tell? Why should this not be lumped in with all the other ai hype being pushed?
"The mathematicians I know are saying that the latest crop of models is changing the way people do research math, I think that's a pretty big deal."
Who? And doing what?
We have been hearing the "this generation of models is the one" type talk for years and the only concrete "big deals" are what? A tool for college students to write papers? A replacement for, now enshitified, google search? The fact that now you can fake tons of stuff to support a position or claim tons of stuff that goes against your position is fake?
namr2000 9 hours ago [-]
I want to preface my response by saying that I don't buy most of what the AI labs say. I don't think that LLMs will replace most white collar labor for example. I also find many of the practices of these labs to be abhorrent. However, all of these opinions are orthogonal to the fact that LLMs have gotten extremely good at mathematics.
> Who says no one was making progress?
Let's look at the Jacobian conjecture, since that was the open math problem I was most familiar with prior to its solution. Yitang Zhang, one of the worlds most renown mathematicians (famous for his lower bound on the twin prime conjecture) spent 8 years working on this problem with his advisor (who himself is a renown mathematician) and turned up completely empty handed. His advisor described it as a "waste [of] 7 years of his own life and my time" [1]. Of course, these two were not the only ones working on this problem for the almost 100 years its been open, but they should have sufficient credentials to show that they were not fools or amateurs.
And in a single afternoon an LLM disproved the conjecture. How is that not an extraordinary feat of technology?
> Who? And doing what?
A close friend is studying differential geometry in a PhD program. Sadly I doubt anything I say on his work will convince you, so I will instead offer two anecdotes:
Terrence Tao (widely considered the worlds greatest living mathematician) has said AI is precipitating "a crisis in the foundations of mathematical values and practices" [2].
Timothy Growers (fields medalist & one of the leading researchers in combinatorics) has said that the latest models are now at the point where they are "producing a piece of PhD-level research in an hour or so, with no serious mathematical input from me" [3].
You can find many more fields medalists and mathematics researchers with the same impression. If you look in this thread you can see bluesky/twitter threads from those who were actively researching some of these problems who are in shock at the solutions.
[1] is... a read (sounds like a nightmare student, or research prof, or both). A dumb question but by my reading, that work took place 35 years ago, has there not been anything more relevant since then? Something Claude could have for instance used as a basis for what it did? Does seem like a feat however you cut it though.
Thank you for the thoughts and references.
namr2000 6 hours ago [-]
Yeah the Yitang Zhang situation just sounds like a total nightmare all around.
There was definitely at least some progress on the problem. I get the general sense that there were potential counterexamples that were close but not quite enough, and that its possible (or even likely) that Claude built on those in order to construct its solution. I also get the sense that when Zhang was working on the problem it was believed that it would be proved true, but since then there were bounds found on the problem that pointed researchers to believe it was false. I am not a research mathematician in this field though, so I could definitely be wrong.
Also, in fairness to Zhang, I believe the dissertation he ended up writing was focused on the 2D case in particular, which is still unsolved (the counter example is only for 3D and above). I cannot imagine that anyone looking at the 2D problem was not also looking at the general case as well though.
12 hours ago [-]
avaer 3 days ago [-]
What happens when OpenAI et al stop being open about these things, and just pack it into the training?
Hey man, just wanted to say hi and catch up a bit. I tried DMing you on Twitter. If that sounds interesting then shoot me a message sometime. Hope you’ve been well :)
traes 3 days ago [-]
Not much point to pure math being kept secret, in all honesty. There isn't really industrial value, its only purpose (to them) is showing off their model's capabilities. More realistically they'll just stop paying for it.
Edit: Oh, are you suggesting they just use it to privately improve their models? I imagine a few more correct proofs would have a very marginal benefit, if any. Also, they'll probably just get extracted, meaning it still gets out but OpenAI doesn't get to fancily announce it themselves.
asdewqqwer 3 days ago [-]
At this stage. No doubt calculus had plenty industrial benefit.
3 days ago [-]
simianwords 3 days ago [-]
What does this even mean lol. These are not solved questions. The solution never existed.
3 days ago [-]
dipanshuhappy 14 hours ago [-]
Crazy progress. I wonder how institutional academia would adjust with this. Now its more apparent than ever that the prestige and honour system in academia is having shaky foundations
frenzyguy 3 days ago [-]
This is both awesome and terrifying for mathematicians, however some ideas can be generated and the field as whole expanded with the attention!
However, I was looking at the proofs and reason explanation and openAI should be more explicit in how the work has flown. I find the models have jumped hoops in some places of the proofs, that can be hard to track. In fact, when a paper is published you usually get a review and if no reviewer understands they ask you to further explain the thought process. It will be fun to see if this happens here.
ltitu 3 days ago [-]
So they are bribing 100,000 researchers with free accounts to work on their future unemployment.
joshlk 3 days ago [-]
Some of the Lean proofs are 50k lines - is that normal?
written-beyond 8 hours ago [-]
Interesting how HN promoted this post to the front page again with a fake submission time. There are comments two days old, seems weird why they'd want this post specifically to get more traffic.
john_strinlai 8 hours ago [-]
not very interesting, its the second chance pool. its not a conspiracy of ycombinator vs. openai.
Of course that’s the mechanism and this kind of repost is not unique, but it is still relevant to note that the small HN committee made an exception for this one.
lifeisstillgood 3 days ago [-]
On the token limits etc - one assumes that OpenAI et al are able to “hire expert in field, and let them spend the equivalent of a million dollars of tokens” because they are not actually selling their complete compute 24 hrs a day, so the cost internally is a negligible (ish) electricity bill.
Which is very suggestive - if after everything they are not fully loaded then the next gazillion data centres being built look unlikely to be needed.
lwansbrough 3 days ago [-]
For OpenAI, research is marketing. I’m sure they’ve got plenty of budget for that.
paxys 3 days ago [-]
No such thing as free, even internally at a company. All such use of resources is accounted for, assigned a dollar value and billed to some department. Someone ran the numbers and figured that whatever they spent on these GPU cycles was worth it.
traes 3 days ago [-]
Presumably it's a rounding error compared to their full output, and they're making sure they have enough compute set aside for research by limiting public models. The more datacenters they build the less they have to limit them.
Davidzheng 3 days ago [-]
RL training can use all of them - idk what needed means.
simianwords 3 days ago [-]
I love how people come up with creative ideas to prove the bubble. This one is even more ridiculous - that OpenAI had spare compute to advance mathematics proves that data centres will not be needed. WHAT.
If anything it proves more data centres are needed. That's literally the only reasonable conclusion from this news.
lifeisstillgood 3 days ago [-]
Sorry I thought that a bubble was widely accepted.
Are you arguing there is not an AI bubble, and that all the DC buildout is fine, going to be profitable etc?
I am not looking for a online slanging match - just looking for a different point of view
simianwords 3 days ago [-]
It’s not obvious at all. If it were obvious to you, it would’ve been to OpenAI. It’s in their interest to accurately predict demand. The assumption that OpenAI/Sam is both really powerful but simultaneously ignorant to know what others know as obvious is well.. just strange. Especially strange when OpenAI has more information on models, breakthrough and usage patterns and we don’t.
I’m not participating in the slinging match but it’s very very weird that you think it’s some established thing that these companies won’t make profit. A lot of hubris must go in this kind of thought. Like.. do you all think everyone’s playing musical chairs?
AngryData 12 hours ago [-]
You say that like the tech world isn't littered in a field of dead and failed companies and billions of dollars burned on failed ventures and ideas. Sure LLMs have proven they have value, but where is the trillion dollars of current investment going to be paid back from? So far it is still entirely speculation that they have such a high value, and they can't just play the long game of "well after a few decades of production and iteration it will add up" because half the hardware cost is going to be obsolete energy burning trash for them in 5 years.
simianwords 1 hours ago [-]
I agree with you but from your own comment, it seems to indicate that an obvious bubble doesn't exist. The possibility does.
svieira 3 days ago [-]
They very often have been in the past. Why do you think this time is different?
simianwords 3 days ago [-]
“Often” is load bearing. I don’t think markets are more likely than not to be musical chair shaped. To make this conversation more concrete, give me a falsifiable prediction on there existing a bubble. And then I’ll tell you if I believe in it or not.
effseven 2 days ago [-]
The price of inference is going to go so low that OpenAI and Anthropic will not be able to turn a profit, thus cannot afford the investment into more data centers, thus crash due to investment in the space having been overdone
qnleigh 2 days ago [-]
Can anyone comment on the significance of any of these results for their respective fields? Or what impact they might have? Presumably none are quite at the level of the Jacobian conjecture, but some of the results on group theory and sphere packing sound pretty important at first glance.
qnleigh 2 days ago [-]
Found some discussion here [1] from someone who actually worked on a few of these problems.
I hate this timeline. I might be excited for the kind of answers this AI builds for unsolved problems, and also for learning new things by talking to it. But, I feel like I'm in the minority of people here who feel this could be a net negative endeavor with this having to kill a lot of educational institutions and their ability to fund themselves in the long run. It's not worth that.
amai 3 days ago [-]
Have blog posts replaced peer-reviewed academic papers when it comes to publishing advanced in science?
beering 8 hours ago [-]
If you were a mathematician and came up with any of these results, people would pay attention even if you published it on your blog. What’s the requirement for needing to publish it in a journal? OpenAI is not trying to achieve tenure.
Mathematicians will tear it to pieces if any of it is fake!
0x5FC3 3 days ago [-]
How much do you all think it would cost to "buy" these advances from PhDs, practicing scientists?
traes 3 days ago [-]
This isn't really a productive way to think about these things, IMO. It's quite possible it would take hundreds of years for any specific group of PhDs to solve them. Or one individual PhD could have the correct flash of insight and solve it in a month. There's absolutely no way to predict this, besides trying to gauge the apparent simplicity of the proof or counterexample (which is likely to be misleading). Until someone actually runs an experiment like this it's not a viable metric.
0x5FC3 3 days ago [-]
I understand and I am not trying to deny the impressiveness or the velocity of AI in general. But at some point we have to ask how much do we trust the labs at face value without much transparency of how they got to the results when there is trillions of dollars on the line.
jryle70 3 days ago [-]
Do you think OpenAI investors are more cavaliers than yourself who doesn't have any stake in it?
simianwords 3 days ago [-]
The level of conspiracy theory is nuts
0x5FC3 3 days ago [-]
I would say the lack of skepticism is nuts, honestly.
frozenseven 3 days ago [-]
Capabilities of this sort have already been demonstrated by independent parties, and models have consistently gotten better at this. Yes, insinuating that mathematicians and scientists are secretly solving decades-old problems on OpenAI's behalf is an insane conspiracy theory.
shimman 3 days ago [-]
That's not what is being purported, you're doing a complete misdirect. OpenAI wants to IPO so Altman can potentially capture a trillion dollar bag, with so much money on the line + betting US foreign policy on it as well (pax silica) it's not hard to be overly suspicious of such claims. Especially in the context of a group of people wanting to generate a new decades long cold war in the form of China being the new big baddie (just ignore how destructive, both self- and towards the world, the US has become).
These companies desperately want a return to serfdom. If they didn't come off as so anti-human the public backlash wouldn't be so great.
A friend’s PhD advisor has been chasing non-sofic groups for 25 years (and was shown a preprint of the results by openai to verify them). He believed a solution would be Fields-worthy
This was not a problem that was for sale
heaney-555 3 days ago [-]
You couldn't. PhDs have been working on these problems for decades. It wasn't for lack of trying that none of them could figure these solutions out!
3 days ago [-]
kingstnap 3 days ago [-]
It's remarkable how you can manage to get these models to produce remarkable breakthroughs like an explicit construction of a non-sofic group.
And yet this is the exact same company that has screwed up their android app so bad that the latex N^3 rendering problem makes it so having it explain it to me crashes the app.
Truly jagged beyond belief.
christofosho 3 days ago [-]
I would love more time and money put into real-world problems by these companies. Climate, food insecurity, pollution, technology for convenience and/or to help people have a higher quality of life.
I'm sure they must do some of this type of work, right?
zquzra 12 hours ago [-]
This is something I have been thinking about for quite a while. I readily embrace the advances AI may bring to mathematics and the hard sciences, but those advances were largely expected even predictable.
What I had hoped for was something more ambitious: using AI as an arbiter in economic, political, and social debates, one capable of weighing evidence, exposing trade-offs, and helping us make decisions that produce better outcomes over the medium and long term, even when those decisions conflict with powerful private interests.
I suspect, however, that this is not a particularly urgent goal for the people funding and directing these systems, many of whom live far removed from scarcity and its consequences.
beering 3 days ago [-]
Solving math problems doesn’t require the cooperation of rival factions.
christofosho 2 days ago [-]
It's a tad petty, no? In the end, the contribution to a healthier society leads to more for the companies contributing.
AngryData 12 hours ago [-]
Sure but the people who tend to get into leadership positions are people that are primarily concerned with personal wealth and gain in the short term. It's the prisoner dilemma except with more players that all assume everybody else is going to screw them over too. Because there are basically zero personal downsides to being the one to screw everyone else over.
braneloop 3 days ago [-]
Yes, but all of those are orders of magnitude harder than math.
christofosho 2 days ago [-]
I suppose it depends on which types of problems you're targeting. There is a lot of physical science, and theoretical science, that has gaps because there aren't enough people working on tooling to assist in things like calculation, generation, simulation, etc.
I agree, some of the problems are more difficult. I don't think that's the case for all of them. And, besides, these companies could be demonstrating how to approach problems and where their users could spend tokens to help with these problems.
Should not these companies try to work on these problems _because_ they are difficult?
amazingamazing 3 days ago [-]
They are political problems, a computer could never solve them.
throwaway198846 3 days ago [-]
A computer could solve them by creating the right technological ,social, rhetorical and economical solutions but that would lots of money anyway
amazingamazing 2 days ago [-]
We already know the solutions
adroitboss 3 days ago [-]
Tell that to game theory.
adroitboss 3 days ago [-]
What's stopping the non-profits that exist today from just putting more money into tokens to get the solutions they want?
christofosho 2 days ago [-]
I'm not sure if this is meant to be rhetorical. If it isn't: money and manpower. The LLM companies have the money, they have the manpower, and so they could likely spare to target some of the problems they are also helping cause.
slashdave 3 days ago [-]
Seriously?
We already know how to solve all of these issues. What we lack is collective political will.
christofosho 2 days ago [-]
There is boundless technology we have not yet discovered. I think that we understand we have problems. I don't believe we actually know how to solve them all.
And yeah, the lack of collective political will sucks. It would be naïve, however, to think that there is no value in ensuring longevity in our current and future infrastructure. And improving it to sustain the population giving these companies their value is an obvious win.
slashdave 1 days ago [-]
This SV mentality that technology can solve all problems is rather tiring, really.
globular-toast 3 days ago [-]
We only know how to do it by means of considerable sacrifice. That's why nobody wants to do it. Solving the issue would be doing it without sacrifice or somehow getting us to do it regardless.
slashdave 3 days ago [-]
> We only know how to do it by means of considerable sacrifice
Little sacrifice actually
> Solving the issue would be doing it without sacrifice
So... you are expecting magic?
LLMs cannot create resources out of thin air.
paxys 13 hours ago [-]
None of these problems need technology to solve. Step 1 is trying to get different groups of people to cooperate with each other. Good luck with that.
samatman 12 hours ago [-]
Not really. They're an AI company: they develop AI and sell it. There's some room for pulling off flashy marketing stunts, but not all that much.
This is division of labor, and it's a good thing. I'm sure OpenAI employees, who are very well paid, donate some money from their salaries to others working on the areas you're citing: probably more than you think, I say that from having attended some EA parties back in the day.
But that isn't my point: my point is that a company which makes brushless motors should put most of its time and money into solving the "make and sell brushless motors" problem, and if they or their investors feel like they need to do more for the world, give money to the people who have the time and ability to do things about that. There are a lot of quality-of-life improvements which need brushless motors.
Next question is how useful their product (OpenAI, I mean) is to more focused do-good-in-the-world professionals. I'm sure that varies quite a bit. For getting the homeless off the street? I conjecture, not very useful. For 'complete the transition off fossil fuels'? Extremely useful, no one in that field knows how to do their job without AI in summer 2026. I'm certain of this.
solenoid0937 3 days ago [-]
Almost like superintelligence solves these problems...
miltonlost 3 days ago [-]
We know how to solve food insecurity (in 1st world countries). We have plenty of food. Capitalism requires though throwing out food that can't be sold because billionaires find giving away things anathemic to their worldview. Get rid of billionaire sociopaths.
raver1975 13 hours ago [-]
I wish I could qualify for some free AI as a mathematics researcher. I guess I'm just an amateur. https://alethean.org
petilon 3 days ago [-]
At what point can we say AGI has been achieved? What is the test? AI is solving mathematical problems that humans have not been able to solve for decades. Is that not enough?
Sam Altman has said "If superintelligence can't discover novel physics, I don't think it's a superintelligence." Is that the test? How far away are we from AI discovering novel physics? It seems within reach.
jrflo 14 hours ago [-]
I think we previously assumed that AGI would need to come before superintelligence, but it kind of seems like that's wrong? Super intelligence in a narrow field has arguably already arrived (solving problems that were previously unsolved), but we are still pretty far behind human abilities in things like spatial reasoning or computer use.
antonvs 3 days ago [-]
It’s artificial, it’s general, and it’s intelligence. The people who believe “AGI” is an important and unattained goal need to start coining and defining their terms better.
petilon 3 days ago [-]
A true AGI will continuously improve itself without periodic retraining from scratch. Just like humans.
antonvs 1 days ago [-]
That’s just an assertion. Why is that the “true” definition?
tim333 2 days ago [-]
It depends on your definition. For me it would have to be able to do the stuff humans can do like make a cup of coffee (Wozniak test).
Just maths isn't really general enough for the G in AGI.
petilon 2 days ago [-]
It can give you detailed instructions for making a coffee. Is that not enough? Actually making coffee requires more than intelligence, it requires eyes and limbs (i.e., robotics). Think about a human that is blind and does not have limbs. Does he not have natural general intelligence, even though he is not able to make a cup of coffee?
tim333 1 days ago [-]
The Wozniak test has it with a robot body going into a house, finding a coffee maker and making a cup. I guess you can vary the rules as you like.
petilon 1 days ago [-]
That's a good test for a robot, not for AGI. AGI should test only intelligence and should not require limbs.
AngryData 12 hours ago [-]
Okay we can make a completely digital environment for it to make coffee in then. It would still fail unless you let it randomly try every combination potentially thousands of times until it stumbles upon the right path. It doesn't take intelligence to read off a recipe, it does take intelligence to read a recipe, understand it, adapt it to your specific tools and materials on hand which may differ from the recipe, and then actually still accomplish it in the first or maybe second try.
azan_ 3 hours ago [-]
Why do you think it would fail? I think it’d be zero shot.
I feel increasingly anxious reading this. Machine research shouldn’t be merged into mainline of human knowledge.
vessenes 13 hours ago [-]
When you can formalize it in Lean or some such, why would this be? I can understand the desire to separate out other forms of research from the human corpus. But theoretical math that is decidable/provable, I’m not sure I see the risks.
rencrisa 12 hours ago [-]
I just want to state that having "lean proofs" that build does not mean the actual real theorems we care about hold. Ultimately a human has to verify the lean encoded theorem statements that the lean proofs are checked against. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal.
jstummbillig 13 hours ago [-]
Why?
cwiz 13 hours ago [-]
Because science is a branch of philosophy and machine existence brings plethora of unanswered questions.
Imagine humankind meets another race, another race shares it's scientific knowledge and humans accept it without experiencing process of discovery. In that case do we really got this knowledge? If we follow machine discoveries like we follow problems in textbook then we acquire knowledge but we don't discover anything. We follow.
There whole lot of philosophical questions that aren't attacked now. Are complex systems sentient because consciousness is emerging behavior? Then should they have rights? Philosophy is part of humanities and science (is/used to be) part of philosophy. Should we accept non-human knowledge in science? Maybe it's altogether different thing from science, yet very similar.
beering 8 hours ago [-]
Almost everything I’ve learned in school is learnings handed down from others, not things I discovered. Is all that knowledge useless?
And no, science is not a branch of philosophy and not everything is a philosophical question, despite what the philosophers like to say.
MattGaiser 13 hours ago [-]
Knowledge is knowledge, as long as it can be proven true.
raver1975 13 hours ago [-]
proving false is also useful
kypro 16 hours ago [-]
I want to iterate the most important thing about this is that it's yet more evidence of AI's accelerating competency in solving math and comp sci problems, and suggests we're now getting close to the point where you could throw AI at AI research challenges (which are largely just math and comp sci problems) and potentially find very real algorithm improvements.
AI development is likely to be more compute bottlenecked than solving math problems since validation of any algorithmic improvement would likely require significant compute. But you could imagine that at this point it could be economical for a frontier lab to task 10,000 agents to work non-stop on finding novel algorithmic improvements then validating the top 50 out of 1,000 candidates on a GPT-2 sized network.
I would suggest RSI is now very close. The singularity could be less than 6 months away. I'm not saying I'd put a high probability on that, but I'd give it at least 20%, and I'd double that if looking 12 months out.
I know I'm just a crazy man shouting at the clouds, but please take to the consequences of this seriously. I understand that for whatever reason AI risk seems abstract and doesn't seem real, but this should terrify any person thinking logically about where this could all be heading.
We haven't even solved the most basic AI safety problems yet. RSI right now would almost certainly result in an extremely bad outcome for humanity.
14 hours ago [-]
variadix 15 hours ago [-]
I’m starting to think the probability of RSI within 12 months is more like 99%
I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.
xpct 16 hours ago [-]
Okay, let's take it seriously. What do you propose? What can your average person do to prepare for RSI beyond bracing themselves mentally?
reducesuffering 14 hours ago [-]
You can not prepare or brace yourself mentally any more than you can a terminal cancer diagnosis. An RSI foom right now means an unaligned superintelligence will disregard us in pursuit of its goals. We would be ants in the way of a data center being constructed.
All people can do is collectively support the notion, like 1200+ frontier AI researchers and their CEOs, that we do not have control of where this is headed, we need to immediately slow down the race, in time for people to agree that we do not have the capability to align a superintelligence to humanity’s wishes
3 days ago [-]
bifftastic 3 days ago [-]
Any advances in theoretical physics yet? Are there any fundamental obstacles? I would have thought not, but I haven't seen anything reported.
ls612 2 days ago [-]
The fundamental obstacle is that we have no conceivable way to produce the energy levels to test the predictions that new theoretical physics would produce. We are like over a dozen orders of magnitude off.
tim333 2 days ago [-]
There's a lot of everyday stuff in physics which is unexplained like the particle masses we have.
Windchaser 16 hours ago [-]
And a lot of condensed matter physics. Type II superconductivity is a well-known one, but there are a lot of more less well-known ones
QuesnayJr 3 days ago [-]
The Maxwell conjecture was a conjecture in theoretical physics (though not a particularly important one)
3 days ago [-]
s_Hogg 3 days ago [-]
I don't know why, but when I saw the source of this particular headline it reminded me of the album title 26 Mixes for Cash
defrost 3 days ago [-]
Ambient 0: Math for Airports
kart23 11 hours ago [-]
AI can do this shit but can't do the dishes
9 hours ago [-]
scuppernong 3 days ago [-]
the people who crow in the comments of each of these posts about AI advances making human beings useless seem to bizarrely identify themselves with the AI, but none of them seem to have had any hand in building this technology. at best, they're power users. pure ressentiment.
melagonster 3 days ago [-]
Wow, so this is the end of science :(
AngryData 12 hours ago [-]
It solved a handful of novel esoteric problems out of hundreds fed to it. Far from the end of science.
xyzsparetimexyz 3 days ago [-]
It's just another tool that can help solve problems. It doesn't know _what_ problems to solve. It turns out that a lot of old problems are now low hanging fruit for these new models. In terms of 'expanding the frontier', we've just discovered dynamite and can now blast our way through mountains. The bottom of the ocean or space are still as hard to reach as ever.
silver_sun 2 days ago [-]
It's not even predictable like dynamite. Sometimes it can blast through a mountain, impressively, the problem is you can't predict which mountain it works on. And other times it can't even make a dent in a molehill, which is perplexing given what it was capable of earlier. Can we even call it dynamite?
woeirua 3 days ago [-]
No bud, it’s just the beginning!
casey2 2 days ago [-]
People weren't their strongest even when most did manual labor. Now that humans are free from mental labor we work on creating and optimizing the best exercises for each mind. Couple that with restructuring transport infrastructure and diets many people will be smarter and fitter than at any time in history. They won't be able to outrun an automobile or out think an autointelligence.
bwestergard 13 hours ago [-]
"People weren't their strongest even when most did manual labor."
Is there good historical data on some measure of strength across representative populations over time in the modern era? I'm doubtful.
We do know that the introduction of agriculture diminished strength:
"Bone mass was around 20% higher in the foragers - the equivalent to what an average person would lose after three months of weightlessness in space.
After ruling out diet differences and changes in body size as possible causes, researchers have concluded that reductions in physical activity are the root cause of degradation in human bone strength across millennia."
I think there's another interesting story here about how this was apparently moderately flagged and triggered the flame-war detector which kept the story off the front page of HN 2 days ago[0]. I think people are having a hard time processing this information rationally(?)
What can we do to make conversations around these incredibly exciting and important topics more constructive? HN is where I expect to read expert comments on these topics, has this style of conversation moved elsewhere?
The main issue was that it was submitted late Friday night SF time, meaning it was overnight or Saturday everywhere in the world when the post had its chance on the front page. The flagging was minimal relative to the vote count and had no effect, and the flamewar detector would have been turned off sooner if moderators saw it sooner (it wasn't really a flamewar, just a lot of comments). It still spent 10 hours on the front page.
None of this is anything out of the ordinary; this kind of thing has always happened. The only real story here is that moderators sleep sometimes.
jrflo 15 hours ago [-]
I think that HN is particularly negative towards AI because the vast majority of users here will have their prestigious CS careers disrupted by AI advances. So, there's an inherent negative bias towards this news.
I for one am really fascinated by AI's advances in science and math and would like to talk about it somewhere without the constant flamewars...
sothatsit 13 hours ago [-]
Extreme claims on posts like these also, rightfully, trigger people’s skepticism. I don’t think it’s wrong to question claims that math is dead as a field. But then it leads people to miss the overall trendline.
People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps leading to crazier and crazier results. The much more interesting question to me is what will be consumed by the exponential like math seems to be, and what won’t. Writing has been much more stubborn, but I’ve noticed Fable to be quite a big step up there as well. How about politics? Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation?
And then there’s questions like, even if AI can answer increasingly complicated math questions, will we still need mathematicians to translate results to the real world, verify them, or decide where to push the frontier?
reducesuffering 13 hours ago [-]
> HN is where I expect to read expert comments on these topics, has this style of conversation moved elsewhere?
For people who have been paying attention to accurate predictions leading to our present state of the world, HN has collectively been reactionary, incorrectly dismissive, and incredibly behind the curve. In public, people are better informed on LessWrong and AI Twitter circles. That's where frontier researchers are. Barely here
In a way the most remarkable thing about this is that it isn't even at the top of the HN homepage. Even if this is a step up from what we've seen before, we're no longer astonished by the idea that AI can make significant advances in mathematics and computer science.
saithound 3 days ago [-]
I don't think that's it. Multiple or my friends from the target audience (academic mathematicians) admitted to scrolling past because the title made it sound like a review of last month's contributions, instead of 10 new ones.
gbnwl 3 days ago [-]
There are articles with far fewer upvotes and comments ranking higher on the front page right now, despite being the same age or older than this one. HNs opaque ranking system at it again.
antirez 3 days ago [-]
This is not at the top as it is actively flagged by people that can't psychologically cope with the advances of AI. Hacker News is no longer a web site of an elite.
tomhow 16 hours ago [-]
It wasn't heavily flagged. It was pulled down by the flamewar detector due to the large number of comments, and it slid under the radar due to only hitting the front page during overnight hours on Friday night/Saturday morning. It still spent 10 hours on the front page, but all during off peak hours. I've now created a new copy of the post so it can have prime time exposure.
Chance-Device 3 days ago [-]
> people that can't psychologically cope with the advances of AI
Yes. And there are many of them. I wonder what would help them come to terms with it. Seriously, people are going to be grieving over this. Loss of identity, loss of social standing, ideas of entire future lives that will now never happen. The greatest crime people may hold AI guilty of is taking away their dreams.
matteoraso 3 days ago [-]
People have had to deal with getting their jobs automated away for centuries. None of this is new, and perhaps reminding ourselves of this is the best way to cope.
Chance-Device 3 days ago [-]
I understand your sentiment, but I think this really is something different. This isn’t a craft going away, or even an industry being replaced, it’s potentially everything we do. It’s the ground being pulled away beneath people’s feet, everyone, everywhere all at once. I think the vacuum it leaves in people’s lives needs to be filled with something, and I haven’t heard any good ideas about this or how the transition should be managed at all.
nevertoolate 14 hours ago [-]
There is a glaring fallacy in your “AI will change everything as it is super intelligent” hypothesis. If it is so great thinker which can do everything why not just solve this social impact thingie? Or maybe it is not so capable?
Chance-Device 13 hours ago [-]
I don’t think it’s magic. Things need to actually be made to happen regardless of intelligence, and that’s a social issue. Also if AI were powerful enough to fix everything by itself effortlessly it would already be far too powerful for us to control, and we probably shouldn’t allow that to happen on general principle.
lostmsu 12 hours ago [-]
That assumes the particular problem actually has a solution (that you will like).
zeven7 2 days ago [-]
This is the sentiment of people who haven't accepted that this is in fact something very different from what people have seen in the past.
lacy_tinpot 15 hours ago [-]
It's given many more an opportunity to fulfill their dreams.
tuesdaynight 2 days ago [-]
I like your comments and agree with a lot of your points, including parts of this one. That said, please don't go to this route. A lot of the doomerism comes from financial insecurity fears. Try to remember that a lot of people are subconsciously afraid of losing their homes. I know that it is pretty hard to ignore them, but try to engage with people that do not dismiss 100% of AI accomplishments.
rwz 2 days ago [-]
> A lot of the doomerism comes from financial insecurity fears. Try to remember that a lot of people are subconsciously afraid of losing their homes. I
I think recognizing and accounting for your own personal biases is one of the requirements of the being an intellectually honest and rigorous online discourse participant.
Things could be genuinely impressive and fascinating even when directly challenge your ego and material well being.
bencarmin 3 days ago [-]
This comment is about the tier of a Reddit atheist going to a funeral and telling a grieving family that "haha grandma is dead and there is no heaven".
lkey 3 days ago [-]
Forums change with the times, and this one never existed solely to burnish your ego.
Moreover, mister elite, you don't know why this press release was flagged.
I'm not sure why we should privilege your bitter speculation over more mundane possibilities.
simianwords 3 days ago [-]
[flagged]
lkey 3 days ago [-]
You don't accept it is possible that it was initially flagged as a dupe or spam why? There is competition for primary submission here, especially for the primary AI companies.
Moreover, if it wasn't flagged at all, like you suggest, then the grandparent was inventing things to be bitterly resentful about... Which is not a behavior any forum should indulge.
fg137 3 days ago [-]
Didn't know I was part of an elite.
w4yai 2 days ago [-]
You're were for 4 months. That's what we're talking about. It used to be.
antonvs 3 days ago [-]
> Hacker News is no longer a web site of an elite.
It was always mainly a website for employees of an elite.
matt_daemon 3 days ago [-]
It’s never been clear to me why the HN algorithm isn’t public. It’s obviously nowhere near as complex as something like Twitter, and of course isn’t a trade secret. The fact it’s private only furthers speculation like this.
titularcomment 6 hours ago [-]
Probably to avoid manipulation, there is tons of stuff on HN that hopes to make it to the limelight through this forum channel
ofjcihen 2 days ago [-]
Or maybe, just maybe, other people have different opinions than you?
Is that possible or is everyone else too common to have those?
BigTTYGothGF 3 days ago [-]
> Hacker News is no longer a web site of an elite.
Never was.
dwb 2 days ago [-]
So condescending. “Can’t psychologically cope”? Can you hear yourself? There’s some advances, but we’re losing a lot too. Don’t get dazzled by the hype.
13 hours ago [-]
pistoriusp 3 days ago [-]
Interesting. I had no idea that a person could see what is flagged?
defrost 3 days ago [-]
If you page through the /newest listings you can see [flagged] and [flagged][dead] submissions.
That's true, but submissions are only killed in that way if they receive a ‘fatal’ number of flags. However, flags lower the rank of a story even at non-fatal levels. What antirez is suggesting here is that the rank of this story has been lowered by flags – and that seems plausible, if you compare its rank to that of other stories with a similar age and number of points.
defrost 3 days ago [-]
[flagged] submissions aren't [dead] (killed), they are still active and can be upvoted and commented upon.
> if you compare its rank to that of other stories with a similar age and number of points.
Ranking is complicated enough here even before weighting, speed of initial upvotes can play against ranking, number of comments and the shape of the comment tree also affect ranking. And yes, various subjects and submission sources do get weightings that impact ranking.
What's funny, to myself at least, is that any attention at all is paid to "HN front page ranking" - I've been on again off again active here since 2008 .. and can't recall ever really looking at a default HN "front page" ever.
( There's /newest /newcomments /active etc to browse and sites such as https://hckrnews.com/ )
bwfan123 3 days ago [-]
> Hacker News is no longer a web site of an elite
hah, sorry, we are plebs out here.
over_bridge 16 hours ago [-]
Jokes on him. I'm a peasant and I've been here for years
revetkn 3 days ago [-]
[flagged]
root_axis 3 days ago [-]
[flagged]
3 days ago [-]
ltitu 3 days ago [-]
We cannot psychologically stand that Redis is hyped by OpenAI:
If you're actively throwing away brand new greenfield research because it was generated by a computer at a company that stans industry-spanning software so that you can stay mad at your pet celebrity project, you might be the problem.
3aasgf 3 days ago [-]
You have to give AI one thing: It is vastly better at understanding text than AI boosters.
Which is a low bar, but still.
curt15 3 days ago [-]
What about AI research itself? Is OpenAI close to automating its human staff out of a job?
zild3d 24 hours ago [-]
> What about AI research itself? Is OpenAI close to automating its human staff out of a job?
It's more like they've already automated the parts of the jobs that the humans most closely thought of as the "their job"
ianm218 3 days ago [-]
They and Anthropic have indicated that the models are substantially augmenting the research and doing large amounts of work autonomously at this point. Here is one of the many blog posts on it [1]. Many people would dismiss this as "marketing" so take it for what you will.
My guess from following this stuff quite closely is that these companies are still a couple years away from fully autonomous research staff.
Yes, but they wouldn't publish that bit lest other companies steal the ideas.
gizmodo59 3 days ago [-]
It’s also very very divided (x companies, oss vs not and other interests)
schleck8 3 days ago [-]
This is one of the most impactful mathematical publications in history by all accounts
I think we've now hit a point where 99.9% of the population gloss over these types of AI advancements because of human competence being insufficient
No human could have published this because it requires paradigm shifts (e. g. Section 5) in multiple mathematical domains. Mastering one of them to this degree is rare, mastering 3+ pretty much non existent for humans.
Its visibility was diminished due to the flamewar detector and most of its front page time being during overnight hours on Friday night/Saturday morning USA time. I've created a new copy to give it some primetime exposure, because it seems like an important enough announcement to warrant it.
k2xl 3 days ago [-]
[dead]
deyiao 3 days ago [-]
[dead]
16 hours ago [-]
drcongo 16 hours ago [-]
This thread has an absolutely wild points to comments ratio.
What's up with the upvote/comments ratio 8 to 337 on this post? Are the comments already also ai advanced? (/s?)
luciana1u 3 days ago [-]
[flagged]
baq 3 days ago [-]
I asked ChatGPT and it told me these aren’t not important /s
utopiah 3 days ago [-]
[flagged]
utopiah 3 days ago [-]
To clarify a bit due to the downvotes : this is not a research paper from a startup or a public frontier lab, it is just PR from a corporation, thus yes an advertisement. Downvote all you like it's still of no value.
Windchaser 15 hours ago [-]
> this is not a research paper from a startup or a public frontier lab, it is just PR from a corporation
If they're publishing the solutions to these 10 problems, then this is, essentially, the announcement of 10 research papers.
Yes, it's partly for reputation (as are many research papers), but that doesn't mean it's of no value. The best way to advertise is to show that you're providing value.
sashank_1509 3 days ago [-]
[flagged]
matteoraso 3 days ago [-]
It's simple economics. Building a robot to do your chores is expensive and only a small minority of people value their time enough to buy one. Meanwhile, SWEs are expensive and GPUs are (comparatively) cheap.
unknownian 3 days ago [-]
You shouldn't be getting downvoted for something that a majority of pure math and art enthusiasts believe to be true. The truth is many of these entrepreneurs and VCs are obsessed with AI not for money or human progress, but because it makes them feel closer to being a "god" rather than a mere mortal. Much of it (especially AI art) is out of spite for human creativity, which is done by mortals with limitations.
eadwu 3 days ago [-]
Taking the stance of moral superiority is kind of funny. And pure math and art enthusiasts don't think they are closer to being a "god" from understanding/"discovering" math?
Stop coping and deluding yourself mate.
To begin with, whether AI is the one doing the discovering or not makes no difference. Any "pure math" person would aim to understand regardless - and would be quite glad that they have a longer paved path.
Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism).
unknownian 3 days ago [-]
Lmao what a ridiculous response. Yes, some mathematicians and artists are in it to feel smart. But the vast majority also just enjoy the process. Having a computer do all the work for you and just typing prompts in ruins that completely. As Ronny Chieng said in his Harvard speech, the journey is the point.
>Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism)
Ignoring that I meant pure as in non applied math, let's just make it clear: you agree that mathematicians who are against capitalism encroaching on this process should be allowed to dislike it without criticism of being pretentious?
AlexeyBelov 1 days ago [-]
> Stop coping
Isn't coping a good and useful mechanism?
xanderlewis 12 hours ago [-]
Exactly what I think every time someone uses the word 'cope' these days. It's like some kind of virus.
sf12sd 16 hours ago [-]
Not peer reviewed, Lean proofs are 100,000 lines long and Lean has bugs:
Not that there isn't something interesting in here, but lets be clear that we don't have enough information to evaluate this properly. And as always with these labs, BS takes a lot more energy to refute than it does to spread.
scarmig 14 hours ago [-]
It's worth reading Marcus' first line, for the naysayers and flaggers on this post:
> Astra, a new model that OpenAI is testing internally, is amazing. No denying that.
aaroninsf 14 hours ago [-]
I find Marcus on this, something approaching sophistry and rhetorical showmanship in service of maintaining an ideological position, for reasons unrelated to the nominal intellectual clarity.
To sharpen that, I think he's (obviously) interested in maintaining his own brand as "thought leader" and this necessitates de rigeur defense of particular postures.
Sometimes this is easy because the facts warrant it; other times, a bit of rhetorical license is required to preserve nominal coherence and (at least, for the moment) hold certain lines.
This is one of the latter cases, and it's not subtle.
One of the celebrated properties of many intellectual advances or inventions in whatever domain is precisely that it appears obvious in hindsight. It is quite cynical to leverage consensus distrust of large AI players, warranted but also a popular social construction, to insinuate that these are not "real" advances or "real" hard problems, on the grounds they were in some sense cherry-picked.
Identifying the problems amenable to strategies on the table and intuitions (sic) about where bridges might be, is exactly the discerning work that is the core driver of almost all prior progress, but for celebrated accidents and flashes of insight. Anyone working in any challenging discipline knows that those are celebrated and told around campfires precisely because meaningful durable results arising like that is so uncommon.
These two articles make me think of nothing so much as my own durable reaction to the creeping goalposts of AI critics generally: that they often seem to me not unlike a water color cohort scoffing and jeering at the horse, because it got a D on its tensor calculus exam.
Marcus should be on guard against his own cynicism and take care that his assumptions do not prevent clear sight.
12 hours ago [-]
HardCodedBias 15 hours ago [-]
"Gary Marcus' has a good take "
I think that is an oxymoron.
neta1337 15 hours ago [-]
How so? His predictions were accurate so far
energy123 14 hours ago [-]
No they were not. These were his 5 predictions in 2022:
"""
1. By 2029, AI will still be unable to watch a movie and accurately explain the characters, events, conflicts, and motivations.
2. By 2029, AI will still be unable to read a novel and reliably answer questions about its plot, characters, conflicts, and motivations beyond what is stated literally.
3. By 2029, AI will still be unable to work as a competent cook in an unfamiliar kitchen.
4. By 2029, AI will still be unable to reliably create more than 10,000 lines of bug-free code from natural-language instructions or interaction with a nontechnical user, excluding simple assembly of existing libraries.
5. By 2029, AI will still be unable to convert arbitrary mathematical proofs written in natural language into symbolic form suitable for formal verification.
"""
There's still 3 years to go and he's already wrong on 4 out of 5.
sweezyjeezy 14 hours ago [-]
Well I don't typically side with GM, but playing devil's advocate:
1. still not wrong? Unless it's just feeding the audio or screenplay I don't think you can feed AI a full movie in a single context window yet?
2. Not sure, but can you prove this wrong? Can you feed a full, unseen new book and get that kind of answer?
3. Not wrong.
4. I think he'd probably pull you up on 'bug free' - I don't think that frontier models can reliably write 10k LOC without _any_ bugs typically (not that humans can do this either).
Philpax 13 hours ago [-]
4. I think they can, especially if the problem statement is well-specified and, importantly, autonomously testable. Of course, specifying a problem that meets these requirements is non-trivial, but the claim requests _a_ counterexample :P
sweezyjeezy 12 hours ago [-]
The wording was 'reliably' though? I could just be splitting hairs on that one though to be honest.
lostmsu 12 hours ago [-]
1 is wrong. If I tell Codex + GPT-5.6 to do it now, it will figure out how to do it. If it would need to extract audio and run a speech model on it, it will find one, set it up, and run without my help.
sweezyjeezy 12 hours ago [-]
I'm not buying this. GM clearly was trying to set a benchmark for video comprehension, not tool usage. Video comprehension is required for many 'AGI tasks', especially robotics to work in real time.
An LLM could theoretically try to earn some money and pay a human to do most of these tasks but it's not the point of the exercise.
an0malous 14 hours ago [-]
> There's still 3 years to go and he's already wrong on 4 out of 5.
Have these been tested or are you just guessing?
ducktective 13 hours ago [-]
Do LLMs generate deterministic or trustworthy answers?
overgard 14 hours ago [-]
It can be annoying when someone you disagree with is frequently right!
maxprimes 16 hours ago [-]
I'm sure OpenAI is just interested in the greater good of mankind!
merelydev 14 hours ago [-]
Great stuff. Wonder how many of the ten problems where solved by independent mathematicians not linked to OpenAI
p1esk 14 hours ago [-]
Zero. These were open problems.
xyzsparetimexyz 3 days ago [-]
Any implication of any of these findings? They seem like unimportant nerd snipes to me. If you want to do something actually relevant, get chatgpt to write a simulation of graphene nanotube construction and figure out how to do it at scale.
utopiah 3 days ago [-]
Very marketable nerd snipes indeed.
foobar10000 3 days ago [-]
One - and I do not mean to be snarky - you can literally ask Gpt 5.6 Sol this - and if you want to see cool stuff - Fable running in their app (not website) has a view thinking button that is actually a good way to explore the adjacent fields, etc.
The non-sofic group one is definitely a big deal - would have been a Fields medal if discovered by a human.
zkmon 3 days ago [-]
> claiming human authorship for a proof generated entirely by an AI system would misrepresent both the system’s contribution and the nature of genuine human intellectual work.
AI has no self-awareness. It's a tool. When you assemble a furniture using a screw driver, the torque force interacts with the molecular forces inside the metal and miraculously it transfers the force to the screw though a clever geometry design, communicating the force to the screw to turn it in a certain way.
Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down.
A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.
[0]: Not one of the proofs in the linked article, but from OpenAI too.
ben_w 3 days ago [-]
> A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.
I think you're over-estimating what a smarter highschooler could write.
A "finite loopless undirected multigraph" could have been explained to me at that age if we'd taken Discrete rather than Mechanics and Pure (and one module of Stats) in my two A-levels* in maths and further maths; but from what I saw of the Discrete module, neither:
Every finite loopless multigraph with no bridge possesses a cycle double cover, without additional assumptions such as cubicity, planarity, connectivity, or higher edge-connectivity.
nor:
repeated-edge closed trails masquerading as cycles
would have been something we'd have learned. But more importantly, we absolutely didn't have a feel for how much effort one needs to put into making sure the proof is right, so if one of us had been hypothetically asked to write a prompt it would've been no more than half that length, and missed most of the bullet points.
The human provides the intention and the ability to appreciate the output. Tools do “heavy lifting” all the time, but we still primarily credit the humans who use them precisely because they made the choice to use them.
Provability is just going the way of computation. John Napier had to manually compute logarithm tables over decades and was recognised for his work; now that same work could be performed by a 10 year old with a calculator in an evening.
esikich 3 days ago [-]
What gives the intention and ability to the human?
oklahomasports 3 days ago [-]
Are you playing dumb? Using power tools to build furniture is very different than using an ai robot to carve a statue or whatever.
ipnon 3 days ago [-]
But why can’t we prompt the LLM “just do math research”? This is what I don’t understand.
ascots 3 days ago [-]
100% agree. If the models are so capable that they're advancing math, it doesn't seem like a stretch to expect they should be able to determine with "doing math research" entails and the best way to use their capabilities towards that end. Why do we need to hand hold the models by telling them to do parallel research, keep threads independent, etc.
raincole 3 days ago [-]
If there aren't thousands of TPUs doing that [0] right now I'd be quite surprised.
[0]: e.g. "go through wikipedia's unsolved math problem list and solve them".
mathisfun123 3 days ago [-]
I don't disagree with you but there's no need for exaggeration; ain't no high school student writing this:
> In
particular, proofs for special graph classes, constructions of cycle covers with some edges
covered other than twice, bounded-length or prescribed-cycle variants, reductions to another
unproved conjecture, computational verification through any fixed graph size, and candidate
counterexamples without a complete nonexistence certificate are insufficient.
which is infact a very important part of the prompt.
don_esteban 3 days ago [-]
the fact that such things have to be explicitly in the prompt points to the fact that the underlying system is still far from where it needs to be (basically, lacks basic understanding what a proof is)
skinner_ 3 days ago [-]
No, that's not what this is. This is a warning to the LLM that coming back with partial results is not good enough.
Take a grad student with a perfectly good understanding of what a proof is. Their supervisor gives them a major problem to work on. Almost always, the problem is too hard, the student comes back with partial results, and student and the supervisor iterate from there. Now imagine that they have an unusually cruel and unreasonable advisor who tells them, do not dare to talk to me until you've fully solved the problem. This paragraph is exactly that. It's there exactly because the underlying system is smart enough to know that real mathematicians do not work like that.
don_esteban 2 days ago [-]
If that was the case, that elaborate listing of all things that might look like a proof to a naive student, but are actually not proofs (and not even just partial results, but fundamental misunderstandings of what constitutes a proof) could have been easily and equivalently replaced by 'I am interested only in a full proof, don't bother me with partial results'. Yet, they were not.
To a real mathematician you would not have to list those explicitly, he/she would have understood that implicitly from 'give me a full proof'. That listing makes sense to say only to somebody who pretends to be a mathematician, but has not true understanding of how the math works. The models are getting better and better in this pretension, but prompts like that reveal that it is still just a pretension, not a true understanding.
famouswaffles 3 days ago [-]
They don't have to be. At this point, we have multiple results from 3rd parties where the prompts are very basic.
When you use a crane to do the "heavy lifting" for construction work, do you give full credit to the cranes?
raincole 3 days ago [-]
Read the prompts in the PDF I link and see if your analogy makes sense in this context :)
zkmon 3 days ago [-]
Prompt quality should not matter. If a high-schooler operates the crane to lift a ton of weight 10 floors high, should the credit entirely go to the crane?
Anon1096 3 days ago [-]
When I type 56789*23456 into my calculator and get the result I don't claim to have solved the problem, the calculator did it.
par1970 2 days ago [-]
qed
esikich 3 days ago [-]
Your brain also is physical. Electrochemical gradients flow between physical molecular constructs. Isn't it just chemistry? Do you attribute it to physics or some whole-is-greater-than-the-parts idea?
NitpickLawyer 3 days ago [-]
A better analogy would be a manufactured object, say 3d printed for simplicity. The 3d printer is given an input, and an object manifests itself after some time. We say that the creator of the object is the person turning on the machine, sending the data, and collecting the object. Not the machine itself.
cure_42 3 days ago [-]
I'd say the creator is the one who created the 3d model, not the one who pushed the print button.
dgellow 3 days ago [-]
I would say „I made this gadget with my 3d printer, but the designer is someone else (I found the model online)“. The intent, the drive, the action comes from the human
traes 3 days ago [-]
"I made this proof myself, but the designer is someone else" is an extremely unconvincing claim to ownership.
dgellow 3 days ago [-]
Almost as if a proof isn’t the same as a 3d print. It’s just not a good analogy
NitpickLawyer 3 days ago [-]
(let's assume that)My 3dprinter is special. It has a bunch of values + an algorithm (i.e. a neural network) that takes input as tokens and outputs a printed object.
samatman 11 hours ago [-]
True story: I have a moisture issue in my furnace, such that it needs vacuuming out. This involved detaching a length of tubing, but that puts stress on said tubing, sometimes knocks other things out of alignment, and involves completing the seal between the wetvac and the tubing with my hand.
I also have a 3D printer. I also have a ChatGPT subscription, and some OpenSCAD chops. I came up with a part which would go into the top of the down tube to the drainage pump, and mostly-seal the down tube itself, with an opening on the side to vacuum out the moisture. This was purely prooompted, I took some measurements, printed bits of the part, refined the shape, and you know what?
It works! I can stick it down there, turn on the (very loud) wet vac, and go upstairs. On a 1.5Ah battery it sucks for a bit less than ten minutes, which turns out to be plenty of time.
So: who made that?
Don't care. I'm waking up warm at night.
Also: me, obviously. ChatGPT doesn't have a fucking furnace.
ben_w 3 days ago [-]
> Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down.
When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it.
I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".
naasking 3 days ago [-]
> AI has no self-awareness
What is your mechanistic model of self awareness that yields this conclusion?
> It's a tool
Does your model suggest that tools can't have self awareness?
perching_aix 3 days ago [-]
Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to, and influences future choices. This implies statefulness, which models are intentionally not at inference time (*).
(*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in, I fail to identify (**) a representation of its own state in these bodies of text that it'd be meticulously maintaining. I further fail to identify how it could be hidden or maintained, considering I control like half of it. The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").
You'll sometimes catch models mixing up who's who and how many who-s there even are for example.
(**) I did wish for something hidden though, so maybe it's just concealed? The same way people can encode a lot more of their emotional and mental state than normal into text if they read and write a lot of it, I'm aware of research that suggested the same for LLMs, albeit I cannot cite it. Maybe those phrasing signatures are just alien to me and will never pop out. Either way, I'd expect researchers to stumble upon this during interpretability studies, and either they haven't, they have but it wasn't popsci adopted, or they're keeping awfully tight lipped about it. If you know of anything like this, your turn now, would be happy to learn.
I do wonder how reasonable it is to expect e.g. a single maintained identity though. Maybe it isn't?
(*) Another way to hack around this of course is to just precompute some internal "self-awareness states" and hop around between them. Probably the closest to what the models are actually "doing".
ben_w 3 days ago [-]
Before reading, know that I am uncertain in either direction.
> a hidden representation of self that is continually tended to
This sounds like a personality? They act like they have one of those. It may be an illusion, and even if it isn't an illusion it is unlikely to be anything like the source (us), but they act like it.
> I further fail to identify how it could be hidden or maintained, considering I control like half of it.
Indeed you control everything about a local model, and much of the context of even a remote model. But the state of activations and circuits in SotA AI is hidden in similar ways to those of synapses in your head: difficult to decipher even with probes monitoring the signals directly, and often not emitted at the normal output.
> The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").
While we can be confident that LLMs make up personas etc., it is insufficient to go from "that doesn't necessarily represent the model's internal state" to "therefore it doesn't have one".
> You'll sometimes catch models mixing up who's who and how many who-s there even are for example.
I've, unfortunately, also experienced this with humans. Perhaps they were losing their self-awareness at the time? I do wonder if old-age dementia does that by the end, though the person in question didn't ever get diagnosed with that.
> If you know of anything like this, your turn now, would be happy to learn.
Not quite what I meant, but it's also not entirely unrelated I guess? Personality to me is like a natural bias. It does also shift over time, and is also an internal bit of state. I guess in some respects it can also be self-referential, like personal convictions.
> Perhaps they were losing their self-awareness at the time?
I do think it is entirely possible for people's self-awareness to shift, yes. Or more precisely, I do model things that way.
> Do you mean like these, or something else?
They're adjacent, but I more meant something like these:
So basically, steganography. The difference is that these papers investigate from the perspective of separate LLM instances covertly exchanging information between each other. This is in contrast with the scenario I'm laying out, where an LLM's past state is exchanging information with its future state, continuously representing and modulating a concealed internal state of some sort. And then that state just so happening to be some sort of self-referential meta state.
But then I don't think there's enough covert channel bandwidth in the agent replies for anything interesting like this.
naasking 3 days ago [-]
> Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to
I don't see why an LLM could not have a sense of identity or personality while it's evaluating a specific prompt, or even change self awareness while evaluating a prompt since many outputs model a back and forth conversation. My point is that without a mechanistic model of what "self awareness" means, we have no way of truly evaluating such questions, we're just hand waving vague intuitions about what it could mean.
perching_aix 2 days ago [-]
Sure, but then such a model is not going to make itself. People pitting their vague intuitions is how such models eventually form. I'd also push back regarding that my comment would have been handwavey or without mechanistic elements, even if it was on the whole informal.
This is kind of also the reason e.g. the HN site guidelines are worded the way they are. Regrettably, forums naturally yield themselves to tit for tat type exchanges, but there's really no reason one could not bounce such vague intuitions off of another. I do not have to be right or wrong, and you don't either. Admittedly difficult when its some intensely contentious topic.
If a mechanistic model existed, there would also be no reason to talk about this in the first place. There'd be nothing to discuss, you'd be simply told how a given model characterizes from this perspective on the model cards.
naasking 2 days ago [-]
Even mechanistic models generate interesting discussion. How many years have we discussed Turing machines and the lambda calculus? Almost a century of great work came out of those.
The reason I insist on mechanistic models is because the original post was making a definitive knowledge claim, and in my experience, the knowledge claim is unwarranted.
Delk 3 days ago [-]
I honestly don't think a language model is enough for self-awareness, regardless of the exact model of awareness.
A language model (or an image model or whatever) cannot even be sentient, and I think sentience is a prerequisite for awareness.
Even if we express a lot of our subjective experience with words, the language is just a symbolic representation of those experiences. The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.
You can't have an understanding of what hunger or physical pain feel like if you have no need for food or a sensory capacity for feeling pain. You can't understand what loneliness or pride at an achievement mean if you don't have a neural network wired to value social connection or status. We value connection because we're social animals that have needed each other for survival.
Even the more abstract of our subjective experiences are in some way rooted in our physical evolution.
I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.
AI awareness might actually be more believable if that awareness manifested itself in an entirely different way than in humans. But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience, I think we're seeing something that isn't actually there.
naasking 3 days ago [-]
> The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.
There is no objective evidence of qualia. All evidence of qualia are vocal or other expressions of belief in qualia. Perceptions clearly exist and are observable, subjective experience and qualia, not so much.
> I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.
If your objection is to models based on "symbolic level of language" which you think lack semantic understanding of, say, trees, you should ask yourself how our brain, based on physics which also lacks any semantic category for trees, can somehow develop a semantic understanding of trees. All of these appeals to differences with the brain never seem to acknowledge that fundamentally, the brain has the same explanatory gap with physics.
> But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience
This assumes a lot. It seems very possible to me that intelligence inherently develops a map of natural categories (natural kinds), and language naturally develops around such categorical understanding. Semantics are then fundamentally the network of associations between categories, eg. there is no fundamental difference between symbols and semantics, and the latter cam be inferred from the former, and that's exactly what LLMs do, and why the semantic maps between different languages are so similar and how they can translate between languages.
woeirua 3 days ago [-]
So… your model is 100% vibes based. Got it.
Delk 3 days ago [-]
I wasn't trying to give a model. The point was that I don't think it's necessary to give one.
You didn't address any of what I wrote, let alone provide any counterarguments. Which part of what I wrote do you think was wrong?
naasking 3 days ago [-]
Making definitive claims about whether LLMs do or do not have specific properties absolutely does require precise definitions of those properties that can be used to evaluate those questions. Merely hand waving that LLMs didn't undergo the same evolutionary process is not a definitive argument.
For example, the Turing machines and the lambda calculus don't look anything alike, but they are fundamentally interconvertible, and so in a real sense they are fundamentally equivalent. Without a model, all of your arguments are completely unconvincing for exactly the same reasons, eg. that there may exist many paths to fundamentally equivalent ends.
Delk 3 days ago [-]
I just don't think linguistic (or other symbolic) representations alone can contain the information, in any sense of the word, of what e.g. human subjective experiences actually are like. The concepts we express with language get their meaning from our physical reality, even if quite indirectly in case of some abstract concepts.
Hunger as a concept doesn't mean anything without the physical need. Politeness or bluntness, even in writing, don't mean anything without social dynamics. And we have social dynamics (and neural structures that directly process social cues and associated feelings) because we've evolved into social animals for whose survival that was important.
I see no reason to believe that a model trained only with symbolic representations, with no connection to the physical world phenomena that those symbols represent, could contain the subjective experience itself.
Neural network models may be able to derive novel (or at least novel-looking) output rather than just an obvious rehash of their input, but I don't think any set of bytes can fundamentally contain information that was never entered into it. (Even if e.g. a model produces previously unknown mathematical results, those results can in principle be derived from the information that they were trained with.)
I'm not saying that artificial neural networks couldn't, in principle, be aware. ANNs and biological neural nets may be equivalent in the sense that any information and processing structures represented by a biological one could in principle be represented by an artificial one. If that's the case, and awareness is purely a product of our neural systems as materialism would imply, it should be possible for an ANN to be aware, too.
But when the model has been trained with only language, and IMO the subjective experience can't be derived from the symbolic representation alone, I can't see how the model could include the actual subjective human experience.
An AI model could of course have an awareness and subjective experiences that are totally different than our human experience. But then the fact that it happens to produce output resembling what humans find meaningful shouldn't be considered indicative of such awareness.
This is of course more of a philosophical argument than a technical one, and I'm happy to hear counterarguments, but not on the level of off-hand dismissal.
naasking 2 days ago [-]
> I can't see how the model could include the actual subjective human experience.
People who say LLMs have subjective experience aren't saying they have human-type subjective experience. Nobody who sees an LLM express hunger when role playing as a hungry person thinks that the LLM is actually hungry.
I too can role play as a hungry person despite not being hungry, so there is no reason in either case to conclude that the words produced reflect genuine internal subjective states. The point is that such internal states may still exist.
3 days ago [-]
titanix88 14 hours ago [-]
How do we know that these solutions don't exist in the training data? It is open secret that they have used pirated materials for training. Perhaps it plagiarized solutions from works of some obscure Belgian mathematician from the sixties, who did not get mainstream acceptance. I wouldn't be surprised if they also got access to mathematics done in the "defense contractor" setting from various three letter agencies.
Without a searchable index of training data, it is hard to put faith into these claims.
jgord 11 hours ago [-]
upvoted for fair point .. its possible an LLM AI could be put to work to search widely for attribution / similar results.
eg. "we spent another 2k on searching for pre-existing proof but found only the weaker result xyz by abc in 1972" would be in the spirit of academics quoting prior work.
ken47 14 hours ago [-]
This wouldn't be a problem so long as they properly attribute.
The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? How about experiment driven domains like biology?
What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.)
Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them.
Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.)
My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.
Decentralized manipulation, by contrast, just runs amok creating echochambers and polarization.
Another degree of capability, yes. But we have been trending here for a long time.
Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them.
That’s where this is headed and people are not ready. I don’t think we could prepare them, anyway.
> Ads are not really the same. They can’t be as tightly targeted to what resonates with someone.
The EU referendum in the UK proved your point false.
> Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them.
When people talk about “ad tech”, they’re not talking about TV ;)
And yes, people can and do engage with them. That’s how ads on social media works.
During the EU referendum, people were even resharing ads on Facebook without even realising they were ads.
insulin is not straightforward, the way the insulin molecule interacts with its receptor is nuts. on the other hand its post translational modifications are simple and dont have anything particularly surprising (no glycoslation, disulfide bonds where you would expect, nothing special kex2 cuts, arent really defective in disease states even)
Side note: it's manipulation either ways because you chose what to talk about, with a goal in mind.
when it comes to the important question, then both "sides" are the same team.
or if you want it with a pinch of humor:
when a boot is on your face, it makes precious little difference whether it's the left or the right boot.
(i lived the first 10 years of my life in communism)
The left in the US would be center-right in Europe, who are certainly not communist (they have separate parties that are communists!)
Even the socialist strain of the US has nothing to do with socialism scaremongering about Venezuela, etc.
The "left" as in the pervasive group that crawled out of Tumblr, took hold of Twitter back in the day, and has a stronghold on Reddit now? Those do care about what you can say, think, watch and read, and the more they can control, the better. The US right can only dream to have half as much control as the left has had in the last three decades.
What's "scaremongering" to you is "life" to GP. Besides missing their point, you're also saying "you held socialism wrong, we can make it work, if only if it weren't for this pesky ... reality"... Sorry, you missed their comment, and I can't take yours seriously :)
I think that statement may itself highlight how prevalent manipulation is.
I fully anticipate all groups to continue maximal manipulation they can. One thing with LLMs is that it'll be a far less unified view, so a "divide and conquer" strategy is what I anticipate.
Look at the most contentious issues in the US: abortion, climate change, taxing the wealthy, gun control, Affordable Healthcare Act.
The Democratic Party platform is aligned with national polling on every one. Every one of those issues has >60% support with voters and the Republican Party has blocked them all.
They play the game to win. And they do.
You left out immigration, crime, and “moral values”: https://www.pewresearch.org/politics/2024/05/23/top-problems...
> Democratic Party platform is aligned with national polling on every one.
It depends on the pollster and where you're polling. I guarantee you rural Tennessee will not agree with downtown Washington DC on any of these issues. In contrast, rural California will likely agree with rural Tennessee. It's not as cut and dry as a homogeneous national poll of 2500 people. Every state, city, county are different. That's why there are federal, state, city, and county governments.
For instance, Abortions are legal nationally. States can individually decide how, or if, they wish to restrict it. This is as the constitution intends under the 10th amendment:
> powers not delegated to the federal government nor prohibited to the states are reserved to the states or the people.
This allows for democracy to take place at the local level, rather than having particular regions thousands of miles away from each other ultimately oppress the other.
To the point on LLMs, I think it's abundantly clear they will be used to propagandize and similar to social media will lock people in a bubble without alternative opinions. It'll be the worst of both worlds, the question is who's the puppet master. At some point soon, I imagine it'll be the AI.
If you've got 5 people in a car and you play ABBA in every road trip, then one day you suggest to change, the problem is not in being okay with "something else", but on agreeing what that other thing should be, set against the already known thing.
That's why there's so much infighting in the left.
They have no profit requirement or even revenue neutral requirement. So they can just operate poorly at a loss and still wreck other businesses because people will put up with breadlines to get bread for ultra cheap.
The general thing to watch out for with all of these “surely it can’t be evil to do nice thing X” is suicidal empathy. It can seem correct to your gut on the surface while it’s extremely destructive in the long term despite participants wanting to destroy something as an explicit goal.
FTFY
Yes, ai has strong narcissistic traits. And so do the people that own them. And pay for them.
In response, people in general will become fat more capable of recognising the manipulation. And will become more paranoid.
Eg.a 20usd/m plan usually don't cut it for professional work.
I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).
> https://www.astralcodexten.com/p/the-sigmoids-wont-save-you
The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf
That aside, I’d question whether applying the Lindy effect in particular to something that’s not really a life expectancy but more a growth rate is credible… or perhaps a bit circular since it “assumes away” the ceiling.
The simpler assumption is that over enough time, the S functions stack together for long enough that working backwards from exponential is a better predictor of reality.
These stacked S curves have continually been true with most technology.
No future for research mathematicians othet than as tastemakers / agenda setters?
Nothing in that article says it's not a sigmoid.
A lot of this sort these sorts of posts are just "appeals to geometry" (aka "cope"). This is coming, it's coming hard. Now you need to decide what you want to do with your life in a world where your smarts aren't as special as they used to be.
This is hard (believe me, I know). But what one ought do is not eschew progress and cling to the delusion that things don't change, what one ought to do is try to see how they can leverage these tools for greater and greater accomplishments.
However, I also expect this squeeze will come at an increasingly expensive price — not just because of inefficient token usage, but because of fundamental limitations of LLMs as a model.
LLMs are letting us brute force our way through a lot of reasoning, but it’s hard to believe that such a generic model of intelligence will take us to the next frontier. We’ll need some fundamentally new approaches at some point. Maybe those will make achieving the exponential more efficient or maybe they’ll unlock even higher degrees of possibility. Who knows?
We know language has to be learnable by every human, so it needs to be really independent of any specific brain development particularities. If it was not accessible to babies there would be no more language next generation.
I think a nice example is using them for arithmetic. It's a specialized deterministic process, so it's extremely wasteful to do it that way.
But they're good at finding solutions to things we don't know how to specialize yet.
So, to use metaphor, maybe the transformer-based models are like the FPGA, and then when we figure out the patterns in that system — all the different kinds of specialized reasoning — we can extract it into an ASIC?
Many many things are only useful when expressed in the physical world, and that introduces lag.
In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some.
In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly.
In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth).
> will we need mathematicians to translate
Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal.
For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude.
> But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years.
For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not.
Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.
The reason is that the original scaling axes (parameters, training tokens, test-time compute) have saturated already, but RLVR (reinforcement learning from verifiable rewards) is still scaling well. And math has this nice property where you can synthetically generate arbitrary volumes of rewards to train the model, because math is self-contained and completely objective. Open-ended reasoning and analysis don't have that convenient property, and that is why progress is much slower outside of math and coding.
Math ability also helps with other things like making models more efficient.
isn't it clearly split between verifiable not verifiable ? what is interesting about that question.
Programming has verifiable and non-verifiable aspects. Competitive programming, passing tests, and performance can all be verified. But translating English requirements into actual software, software architecture, taste, or UI design cannot. And yet over the last couple years we’ve seen huge lifts in all of these areas, not just the verifiable ones.
Verifiable areas I think are clearly seeing the most improvement, or are the quickest to see improvement. But we are seeing lots of progress in non-verifiable areas as well.
How much of the non-verifiable progress is a function of labs purchasing expert data vs. the models improving with compute is maybe another interesting question, but fundamentally I don’t see spend on expert data as something that can’t grow if AI revenues keep growing as well. And as models get better taste they can also help filter and generate new synthetic data for their next versions to train on. The limits of this approach are not so clear.
This is _much better_ data than 1/0 verification, it is as good as a gradient.
Automatically verifiable tasks improve faster since well, its automated.
most gains are still coming from data. isnt that supposed to 'run out' though?
You could view this as just continually patching a leaky ship. But it seems to work.
One example that always bugs me is when people point to "exponential" or "sigmoidal" progress on benchmarks. Benchmarks are artificial constructions (saturation at 100% by definition) and benchmark scores should not be mapped to these words when talking about overall progress.
Example - progress on ARC-AGI-3 at the moment is exponential, steeper than 2^t and e^t. Does that mean AI is progressing "exponentially" in the colloquial sense? No, it doesn't support or refute that colloquialism.
Likewise with MMLU saturation. We can't go above 100% by construction. Therefore we have a "sigmoid". Gah.
The colloquialism is not helpful to begin with.
I think you have completely misunderstood what OpenAI have accomplished here. Almost certainly no one cares about the specific concrete results achieved; they only care about (a) how difficult it would be for an intelligent human to achieve the same feat (ETA: the feat is the proof), which can be estimated by the amount of time the problem has remained open/a public conjecture, and (b) how general this artificial "intelligence" appears to be, which can be estimated by the diversity of topics where it was able to prove a difficult result.
It's as if I showed you a dog that I had taught to speak German fluently, and you remarked: "What point is a dog that can speak a language that less than 2% of the world speaks? Nothing to see here."
And you’re thinking this is an accurate comparison?
what did you notice ?
I literally roll my eyes and cringe quite often at its output pretty much daily.
I don't like to overload my sessions with skills but I've been using a "write-normal" skill I made just to have it rewrite outputs that particularly piss me off.
https://gist.github.com/alasano/1c734fa055231a5defcfd213217e...
I'm sure there's a million of these skills out there, but this one is tailored to the stuff that makes me mad in particular.
That said, Fable is still not a great writer, largely driven by it not knowing what it should exclude, and it still having the usual LLM-isms. But it’s better.
5.6 Sol is great but there's a depth to the understanding that Fable exhibits that's unique to it currently.
Can I truly quantify this? I don't think so. Just that I spend a ton of time with various models and a certain point it's just a personal impression or a gut feeling.
In the days after Fable first came out I increased the amount of parallel planning of tasks that I was doing by 2-3x because it felt like I didn't need to be paranoid due to that handling of nuance.
Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?
Wealth in terms of capital doesn't represent material goods, it represents the system's confidence in your ability to direct capital efficiently. But eventually efficient capital bottoms out at consumable goods. Someone like Musk with a lot of capital under his control is contributing to the end goal of unlimited abundance.
In a free and fair market, his capital would be regarded as a deeply inefficient distortion.
Yes.
The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.
I was recently listening to BBC Radio 4's episode on the Poincare Conjecture[1] and the guests on the program were discussing how the problem that looked deceptively simple eluded the great mathematicians of the time (including Poincare himself) for nearly a century and how Grigori Perelman cleverly came up with the proof. It took other mathematicians working in groups years after Perelman's publication to understand and validate his proof. The mathematicians on the program were speaking of highly of his proofs and admiring the originality of his work. This made me think of one neat experiment where if we cut-off a frontier model's training data 2002 or anytime before Perelman posted his proofs on arXiv and check if it can come up with the solution by itself. That would surely be a great signal to see if these LLMs aren't just solving interesting puzzles and that they can came up with something truly novel.
P.S I highly recommend Misha Green's "Perfect Rigor" for anyone interested in the history of the problem and the genius behind the proofs of the conjecture - Perleman. I found it an entertaining read and could digest its description of the problem as a layperson (with undergrad level math).
[1] https://www.bbc.co.uk/programmes/p0038x8l
Whilst current models can't 'intuit' and come up with conjectures, they can certainly disprove some of them very quickly through the kind of grind that humans can't do. I suppose there really are some mathematicians out there today, whose last few years of study, have just been up-ended by this.
--
"Yes we are," insisted Majikthise. "We are quite definitely here as representatives of the Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons, and we want this machine off, and we want it off now!"
"What's the problem?" said Lunkwill.
"I'll tell you what the problem is mate," said Majikthise, "demarcation, that's the problem!"
"We demand," yelled Vroomfondel, "that demarcation may or may not be the problem!"
"You just let the machines get on with the adding up," warned Majikthise, "and we'll take care of the eternal verities thank you very much. You want to check your legal position you do mate. Under law the Quest for Ultimate Truth is quite clearly the inalienable prerogative of your working thinkers. Any bloody machine goes and actually finds it and we're straight out of a job aren't we? I mean what's the use of our sitting up half the night arguing that there may or may not be a God if this machine only goes and gives us his bleeding phone number the next morning?"
>I don’t understand it yet. Maybe it’ll take me an afternoon to check all the calculations, but what would still be missing is why this was an approach that would’ve made sense in the first place. Is there some broader context or theory within which this would’ve been the obvious thing to do? What other results can be proven using these techniques? What is it telling us about quantum information or operator theory? I have no idea. I spent about an hour this morning asking ChatGPT these questions, but it’s somewhat frustrating because it speaks with a mishmash of physicist, operator algebraist, quantum information theorist-lingo, plus the usual LLM breezy lilt that annoys everybody.
They certainly seem to have "intuited", in a way that is not immediately obvious to experts in the field, the way to solve at least some of these problems. This was not just simply grinding away at a method that humans already knew would work and just hadn't gotten to yet.
[0] https://nitter.poast.org/henryquantum/status/208362369543662...
That's how they are finding these solutions though, unless we are just going to label intuition as something only humans can do. Like a submarine being unable to swim or whatever that example is.
The two places were seeing lots of movement are:
* Updates to lower/upper bounds. In many cases, these kinds of problems are the deep-math equivalent of calculating more digits of pi. Yes, if you throw time at it you'll break the record, but it may not be terribly worthwhile.
* Finding counter examples which disprove conjectures. This is really useful, and helps offset some positivity bias on the human side, often bringing together known tools from distant silos.
If you read the list of ten results, almost all fall into one of these buckets.
As someone with a PhD in combinatorics, I believe that I'm qualified to say that, yes, there are problems as useless as calculating more digits of pi.
Matrices are an implementation detail in reconstructing the surface of human knowledge. It's a complex surface, but it's a regurgitation.
I disagree. I routinely let LLMs speculate or generate hypotheses along the way of helping with technical research. Sometimes they can prove the correctness of a concrete math idea but other times even an unproven conjecture helps with the numerical algorithm implementation and the result is then simply supported by additional data. I guess that any autoresearch-adjacent application has LLMs intuiting and coming up with hypotheses/conjectures—as do the steps/lemmas along a complex proof. In my opinion the modern LLMs are powerful intuitive thinkers that generate lots of conjectures of varying quality or importance.
Of course computers can grind in a way that humans can't. But now we have systems that convert the human-comprehensible ideas into a computer's plan of attack, in a way that greatly expands the frontier of ideas thus treatable.
"Excuse me, We demand rigidly defined areas of doubt and uncertainty!"
DT: Might I make an observation at this point?
MT: You keep out of this metal nose.
VF: We demand that that machine not be allowed to think about this problem!
DT: If I might make an observation…
MT: We’ll go on strike!
VF: That’s right. You’ll have a national philosopher’s strike on your hands.
DT: Who will that inconvenience?
MT: Never you mind who it’ll inconvenience you box of black legging binary bits! It’ll hurt, buster! It’ll hurt!
DT: [Booming] If I might make an observation …
“All I wanted to say,” bellowed the computer, “is that my circuits are now irrevocably committed to calculating the answer to the Ultimate Question of Life, the Universe, and Everything.” He paused and satisfied himself that he now had everyone’s attention, before continuing more quietly. “But the program will take me a little while to run.”
Fook glanced impatiently at his watch.
“How long?” he said.
“Seven and a half million years,” said Deep Thought.
Lunkwill and Fook blinked at each other.
“Seven and a half million years!” they cried in chorus.
“Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I? And it occurs to me that running a program like this is bound to create an enormous amount of popular publicity for the whole are of philosophy in general. Everyone’s going to have their own theories about what answer I’m eventually going to come up with, and who better, to capitalize on that media market than you yourselves? So long as you can keep disagreeing with each other violently enough and maligning each other in the popular press, and so long as you have clever agents, you can keep yourselves on the gravy train for life. How does that sound?”
The two philosophers gaped at him.
“Bloody hell,” said Majikthise, “now that is what I call thinking. Here, Vroomfondel, why do we never think of things like that?”
“Dunno,” said Vroomfondel in an awed whisper; “think our brains must be too highly trained, Majikthise.”
So saying, they turned on their heels and walked out of the door and into a life-style beyond their wildest dreams.”
People keep saying this. Why?
Surely the AI can complete the prompt “Generate new research questions based on these observations”?
When I read the reasoning traces of coding models they are constantly asking themselves questions and attempting to answer them.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.
It's a convex hull of information that is reflective and spans outside of itself and combines in a new way, when you shine two known rays of light together from the inside.
Now it gets better. AI can be orders of magnitude more creative than any human could ever hope for, because his convex hull of information is orders of magnitude larger, and the possibilities for new combinations are equally larger.
The extent to which they are able to do this is the more interesting question!
By the way, convex hull permits extrapolating past the training data. LLM won't invent a new word that could not be defined by a sequence of known words. Just if it's meaningless and fully random/hallucinated, the new knowledge won't work with other known information blocks (breaks convexity).
Humans can “intuit” based on a much larger, if not unlimited, context. Also I just want to say that human cognition is something so insanely complex and deep that we will not understand it at all in my lifetime. To attribute all, or really any, aspects of human cognition to a machine at this point is silly to me.
Most humans are dumber than a box of rocks. Here in Seattle we had one of many light rail-related fuckups where they had to replace part of the line with buses. People piled into the front of one when it was full. When people got out they never moved back. As the driver struggled to close the door and people struggled to get in the wad of people never moved back to fill the ample space.
Chatgpt was smarter than the average person a while ago
There are processes at work there that we don’t even have the language to describe.
(86 billion is the number ChatGPT, ironically enough, has given me a couple of times. I remember hearing for a long time that it was estimated to be somewhere in the ballpark of 100 billion. This is not my field of study.)
This does not demonstrate a lack of intelligence. It demonstrates laziness and a lack of interest in spreading apart. Or just lack of consideration (or even malice) on the part of those at the back of the wad.
> Chatgpt was smarter than the average person a while ago
This is an absurd claim that fundamentally misunderstands what it means to be "smart". Reasoning that would get you to this conclusion would equally well apply to Google's search engine over a decade ago.
You can't find things on a map that aren't there, but maybe you can draw a route nobody used before.
For the same reason that you can't draw a 15 of Diamonds from a regular card deck.
LLMs have made math proofs more computable, in the sense that a computer can both generate potential solutions and check the validity of its solutions on its own, with a reasonable chance of converging on something correct. I assume this was already doable to some extent, but it seems like it’s now exponentially easier. That still doesn’t mean that all math is automatically solved.
This is somewhat similar to things like molecular dynamics or protein folding or finite element simulations, etc. Some problems that were previously intractable via computation became tractable. Others - the vast majority of other problems - remain unsolvable by these computational techniques, because the scale of compute required is beyond imagination. These are simple things like simulating the dynamics of a cubic millimeter of water molecules for 1 second. Unfathomably beyond current capabilities (and LLMs aren’t going to change that).
I think LLMs are great, I use them every day and I think they have a ton of value. But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.
So there are limits - but we’re about to learn a lot about the new normal of what constitutes a layup math proof vs the truly difficult.
On the other hand even if the compute allocated by openai is esquivalent to day 10 human mathematicians, the machines can work 24h per day, that is already a lot more productive.
I think the question, that we keep stumbling over, is what problems are computable.
> But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.
Let the goalpost shifting continue. It'll buy us another half year or so.
By definition. Its those pesky NP jobbies that get in the way.
Whenever the handwaving starts around a discussion relating to a NP hard problem, I find it useful to imagine a Canadian bloke (MHRIP) in a red top, with a ... Scottish accent ... saying:
"Ye cannae break the laws o' physics, Jim". (maffs not fisics, obvs!)
If that is a bit tiresome for the gung-ho AI evangelist, there is also the rather knotty snag that that blasted Austrian geezer Gödel fiddled up: incompleteness.
Its almost as though these bloody clever scientific and that types keep on putting artificial blocks in the way of LLMs laying golden eggs!
I'm quite happy with the "marginal gains" I get with a DGX Spark. It will pay for itself within three months doing stuff on prem and us not sending data to someone else. It will scale.
I’m not sure if people just aren’t as aware, but the Jacobian conjecture practically was on par with those other great problems.
They just need to be better than humans.
The sooner people can be broken out of their denial about all this the better, and we can start actually taking it seriously.
Maybe you’re the one who needs breaking out of your cached beliefs.
In my experience modern models are better at all tasks than models from two years ago, especially complex multi-step tasks.
I suspect GPT 5.6 would be even better at it, if given the same sycophantic system prompt and lack of guardrails.
If you want creative writings, use the API and play with the sliders.
It wasn't "better" it was better at kissing your ass which matches what a lot of people want in a partner.
I think most are actually worth, as agentic harnesses seem to optimize for solving poorly described problems rather than following complex procedures as written. In other words, instruction following maximizing models seem to make worse free-form agents, but they're really all that some domains need.
You can do many more things, when stuff is cheaper, even if the stuff were otherwise unchanged.
For example, every day people teach teenagers how to drive and with only dozens of hours of practice, they are on the road.
That’s not a credible position, but there isn’t anything that I or anyone else can say to someone who simply doesn’t want to believe something.
I don't know if I agree with that but it doesn't seem like an irrational claim and does seem credible to me.
so far there is no end to this progress in sight so it's full steam ahead on this singular domain. once it plateaus you should expect to see the greatest disruptions in human endeavors ever as all the training flops will start flowing to other domains to disrupt and dominate.
If your source is AI layoffs, there were plenty of layoffs with the invention of the horseless carriage, but that doesn't mean it made humans worse off overall.
There are reasons to be skeptical about progress and AI and all that but the 'making things worse for most people' thing you mentioned seems yet to be based in any reality
I ,for one, have read enough history to know that it's never the proles who end up benefiting.
And nowadays with Hong Kong (and probably soon Taiwan) they are proving they are perfectly happy to destroy economic growth as long as it benefits The Party
That's not what people mean when they say "moving the goalposts". It means that people are adamant that something wasn't important/hard/impressive once the "AI" solves it. And then they come up with another thing that needs to be solved in order to prove it is important/hard/impressive. And once that happens, they do it again. And again. That's what "moving the goalposts" means.
It's also very much not a new phenomenon. It's been happening since the 1980s. As you can see from this quote from GEB by Hofstadter:
> There is a related "Theorem" about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of "real thinking". The ineluctable core of intelligence is always in that next thing which hasn't yet been programmed. This "Theorem" was first proposed to me by Larry Tesler, so I call it Tesler's Theorem: "AI is whatever hasn't been done yet."
I don't see how that's any better.
-Alan Turing (allegedly)
In recent years, I have commonly seen the phrase "you're moving the goalposts" deployed by the "it might be sentient" crowd to shoot down the "it's a stochastic parrot" crowd when the latter respond to a new development with "OK but...". In a well-understood field of inquiry, that would be a clear case of goalpost-moving, in the commonly-understood meaning of the phrase where requirements are retroactively changed in response to them having been met. Thank you OP. 'Artificial Intelligence', and indeed intelligence in general, is very much not a well-understood field of inquiry - in fact we don't even have a common agreement about what 'intelligence' is. We are therefore learning as we go (even after all this time!) but making rapid progress in recent years. When rapid progress is made in a poorly-understood field, then how can our definitions and requirements for success not change? This is arguably one of the most pathological development projects ever - what are the requirements? 'It thinks like a human'? What does that mean? And the answer is we don't know what that means, and we're working it out as we go - moving the goalposts. If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
Side note that, in case it's not obvious, none of this detracts from how impressive LLMs are. They're a marvel of the modern age, all the problems notwithstanding. However I reserve the right to stay sceptical about their capabilities.
It's in how they change, not the fact that they change. The skeptics seem to have secret definitions for intelligence, sentience, consciousness, creativity, etc. that amounts to "a thing only humans have". Often that thing is equivalent to a soul. When yesterday's challenge (LLMs don't have X because they can't do Y!) is met, Y changes but X stays the same. This is not the process by which a field matures, it is a rhetorical technique used by skeptics to avoid honestly stating or confronting their internal definitions. That can be revealed by asking the skeptic the following:
"Forget LLMs. What if we made a completely physically accurate simulation of a human being?"
Many say no, that simulated human being still couldn't have (intelligence, consciousness, sentience, creativity, ...). This reveals that there is a necessary metaphysical component to those attributes, at which point any scientific-minded person will leave the debate.
> If we didn't move the goalposts, then by definition we already knew exactly where we were headed at the beginning, and we very clearly did not.
The criticisms are directed toward people who did clearly act like they knew, not the ones who were honest that they did not know.
To me, the goalposts were already defined by the person you were responding to. "The impact of AI is getting undeniable", so, the goalposts are "the impact of AI". Probably something like "the impact of AI is high, or will be soon".
Note that this does not depend on things like AI sentience or defining "intelligence" more rigorously, it just depends on AI impact.
The motte is "AI useful". The bailey is "Singularity is nigh".
(I'm personally still skeptical about this, but I'm being pulled towards accepting it).
"AI is useful" is too low of a bar, and "singularity is nigh" is too high. "AI is on its way to upending society" is about in the middle, and still vastly contentious among laypeople.
But there are people like Ed Zitron, frequently posted and cited here, who disagree even with the former.
Personally I prefer to follow explorers rather than swamp-sitters.
"it isn't clear whether generative AI actually provides much business value at all"
"cannot seem to find a product that people will pay for, in part because the results are so mediocre"
"Last week, we got our first real, definitive glimpse of what’s around that corner that future. And boy, was it underwhelming."
"OpenAI claims that o1 “performs similarly to PhD students on challenging benchmark tasks in physics, chemistry, and biology.” Just not in geography, it seems. Or basic elementary-level English language tests. Or math. Or programming. "
"Worse still, it's kind of hard to explain why anybody should give a shit about o1."
"o1 shows that OpenAI is both desperate and out of ideas."
"the software is not becoming more useful"
Honestly, every other line is quotable in this context.
But it seems we have somehow optimized away shame. It wasn’t good for profits, I guess.
The fundamental argument that I've personally made since the early days of this is that LLMs are not reasoning, in the way that word is commonly understood.
There are lots of reasons why that argument needs to evolve that could certainly appear to be "moving the goalposts", but let's take an example.
A lot of AIs were tripped up by the question "Should I walk or drive 50m to the carwash?" Several folks liked to use that as an example that illustrates that LLMs aren't reasoning, but as the models have been trained on that specific example, it's of course less useful. An AI can mostly nail it now.
So a different example is needed. A new demonstration of how these things fail at basic reasoning a child can do.
Did I move the goalposts? I don't think so. The fundamental argument stays the same. It's not hard to find lots of examples that trip up LLMs, because they are what they are: statistical inference machines. Nothing more and nothing less.
Useful, sure. But also commonly misapplied to areas for which they are inappropriate solutions.
I agree with the parent that we need to acknowledge that we're at a turning point in history. I lived through some of them (internet, ubiquitous personal computing). But it's somewhat difficult to comprehend the impact of this one for many people.
I do biomedical research at one of the top European research institutions. We're very well-funded, but I can clearly see the gap between us (say, top-100) and top-10. I also realize this gap is going to get so much wider unless we invest heavily in AI access (and I'm not so sure I can sell anything more expensive than $20 Claude subscription to the leadership).
I think people having 6-7 figure SOTA AI budgets will move exponentially faster than those who don't. That makes me worried.
So, for me, it's not a question of recalibrating expectations. We're way past that.
Not long ago many folks were saying AI was the same as the crypto bubble. No real useful technology and only hype.
I think AI is clearly both revolutionary and useful. Revolutionary insofar as the job I do has changed almost completely in a year or so span.
There is irony here
i can link you likely dozens of comments from people wrong about this replying to me over the last 5 years
https://garymarcus.substack.com/p/two-critical-updates-re-as...
As always, PR hype. Goalposts have not moved.
Guys, please use critical thinking. The haters don't hate by default, we hate because we're gaslit about this stuff every day and it's annoying. Extraordinary claims require proof, and they're not giving us information that would be essential to knowing if this is actually significant or not.
Remember October 2024 Pelicans [1] ? It's been only less than 2 years.
We don't know what will come in the next 2 years. But the progress doesn't seem to stop for now.
[1] https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/
People are skeptical of the announcement because the room include several PHDs in math and physics. The prompts are not published so we can see how generic the starting prompt is.
If I could take one out of context quote from this whole thread as a response to TFA, it would be this one.
He literally says it's an impressive feat in the second article.
The only way that is PR hype is if you're invoking the insane conspiracy that frontier AI labs are just buying off results that would otherwise be career defining for a mathematician, just for marketing.
The posts you linked are urging caution regarding the exaggerated e/acc-esque lies peddled by people like Musk, not that the models haven't proven themselves as having genuine ability to contribute to research in some areas.
I remember the time when he insisted that diffusion-based image generators trained on Internet scale data will never be able to make an image of a horse riding an astronaut. Today you can generate 4K video of that.
They still do things that I find incredibly annoying and “dumb”. And I still have to clean up messes they make quite often.
But on the whole they are clearly smarter than before. No extraordinary claims needed. I just try to learn how the tool works and how to use it effectively.
We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.
How does intellectual property work in an AI generated future? What about healthcare advances, who gets to own those?
What about meaning, what about purpose? How do we replace the work ethic that tells us we are our jobs and idleness is immoral? How do you replace “What do you do?” As one of the first questions you ask a new person?
That sort of thing.
And it's sad, really, because I think these two groups would make a great pairing if they could stop arguing against one another for a moment. They'll both be impacted about as much and probably have the same ultimate goals (to lead dignified lives).
But it seems these days everyone is more interested in Kayfabe and feeling like they're in the right than working together, so maybe I should just keep quiet rather than attract the ire of both groups...
I don't know if it is fair to say they're in denial. For my part, I don't expect life to get much better for regular people (especially short term), but that doesn't mean we shouldn't work to try to make it happen.
What a lot of people want to do, and I’m not saying that you’re one of them, is to assume that a positive outcome is impossible and either do nothing or loudly yell that the world is ending. Neither is particularly useful.
Or, as I said above, others just deny that there’s anything to see here and try to get people to move along.
Shane Legg (DeepMind co-founder), one of the more intelligent and thoughtful people you'll find in the industry, could only offer "it's a tough problem - we need to think about it" when recently interviewed by Hannah Fry.
On the surface the most likely outcome for AI allowed to replace jobs is extraordinarily negative, especially since it is a general capability technology, not a specific one where displaced workers can just move to another field. Once AI becomes more capable it will be able to do the vast majority of white collar jobs, including any new ones that may appear as a result of AI. As Shane Legg put it, "if your job can be done remotely, sitting in front of a computer, then it can probably be replaced by AI".
Not only does AI threaten to replace ALL the white collar jobs, but it is rapidly going after blue collar (factory jobs, driving jobs) and pink collar ones (Japanese robotics for elder-care) as well.
If a positive outcome (which doesn't include putting displaced workers on welfare - UBI) is possible, then it sure would be nice to hear it, and the silence from the AI companies, and government for that matter, is deafening.
Eventually UBI will be the norm, and if the living standards of a person on UBI is as good as yours or mine today, that will be an enormous win for everyone. It’s like pensions, once these were only for the elderly poor, now they’re a right for everyone in most developed countries.
It’s also interesting that for most of human history leisure time was the point of life, and only in recent modernity has work come to be the meaning of someone’s existence.
UBI has to be commensurate with production being automated. That’s a big logistical problem, if you think building datacenters is a challenge try bringing about radical abundance, but even so it’s not insurmountable. It just needs to be taken on as project and not seen as an impossibility.
So much of this is not about what is possible so much as what people believe is possible. We can do anything if we try.
I see comments like this tossed around a lot, but what makes you say this? Don't you think its more likely that most people end up in poverty?
why do they spend the money they do on the things they do?
they are not altruists and they never will be.
they could change millions of lives today, but most do not.
pure naivete.
What about other countries in the world where people... live in poverty?
> Capitalism allows people to escape poverty by personal effort
Isn't it possible that there is a future where AI makes the average value of human labor (or, "personal effort") plummet? Perhaps capitalism will lose some of its edge against a technology like this.
See "Machines of Loving Grace" by Dario Amodei: https://darioamodei.com/essay/machines-of-loving-grace.
"Massive Economic Abundance: Because AI will exponentially grow the total economic pie, overall resource scarcity will diminish. The fundamental challenge shifts from producing wealth to distributing wealth."
So how do we go from everyone out of work, no income to spend on food, or the goods and services that the AI is producing, to "massive economic abundance"?!
It's like the meme:
Step 1: Create AI
Step 2: AI takes all the jobs
Step 3: ???
Step 4: Profit! (massive economic abundance)
What is step 3?
"Massive Economic Abundance" implies massive increase in produced goods. This implies massive deflation, ceteris paribus. So step 3 could be simply printing money to pay for UBI. Deflation from AI productivity increase and inflation from UBI money printing will cancel out.
For money to work it has to represent some real value, something that has some scarcity to it such as potatoes or hours of human labor. Ultimately it is just a decoupler in a barter system, a universally recognized IOU.
Why would someone give me a car in exchange for UBI-scrip when that UBI-scrip has no inherent scarcity or value and can be produced in infinite supply by the government ?
UBI script will have some value, I am proposing printing enough money just to combat AI productivity-induced deflation, not infinite UBI money.
If you are correct, I expect corporations to reap massive profits while most Americans try to find a way to survive in a world where they are obsolete.
> How do we replace the work ethic that tells us we are our jobs and idleness is immoral?
For many people it has nothing to do with morality, it's hardwired into their instincts. They want to work, and they will work.
For a lot of us who are not excited about this future it's that no one is trying to answer all the questions you laid out. Instead we have the disgusting people at the helm purposefully spreading doomerism and saying, "We'll figure it out." I think it's pretty problematic (to say the least) to care more about technological advancement than how that advancement is actually shaping up to effect people in the short term. But I know many people don't care, especially those who believe they won't be among the affected.
> In the near term handling the transition. Jobs will be lost, careers ended, people won’t be able to reskill quickly enough. At the same time AI is an enormous opportunity to uplift living standards, but nobody has the logistics of this figured out.
> We need to figure out how to restructure the global economy. How does UBI work internationally, if the AI companies are taking revenue in the US? What’s the tax base for it? What does that say about international trade and protectionism? Do countries end up splitting into different trading blocks based on their level of access and legality of AI (I assume some will ban it outright)?.
UBI in the United States is never going to happen in time. If it happens at all. We don’t even get universal healthcare. I think people who think AI will be a net positive for humanity are also in some sort of denial.
In a different US political climate I would entertain it. If these frontier labs weren’t so clearly going after the money, I would entertain it.
LLMs are clearly a step up for capitalists so I just can’t see any inclusion of LLMs move towards more progressive ideologies.
My own $0.02 on the economics piece - every country should have a sovereign wealth fund. Governments should block market access from automated[0] companies until those companies provide equity contributions to the wealth fund for that country. This aligns regulator and corporate interests. Dividends flow into the sovereign wealth funds and then can be allocated locally from there - UBI, job programs, etc. Let different jurisdictions explore different ways to structure a post-labor society.
On the broader social front - I think a lot of lack of meaning discussion boils down to the overemphasis we have on your job as your self-worth. We need to realign our societal expectations - and people need to spend more time with their families.
[0] for this to work, I think we would need well accepted metrics for 'how automated' a company is - and that probably needs a 3rd party auditing industry.
Is there any government that has gotten socialism correct for its citizens? I'd point to UAE/Qatar if they didn't depend on human servitude and inequality.
The only way to win is to wield the AI.
Humanity survives (but we reading this probably don't), the AI treats the living humans like the Emperor's favorite pets (probably a pretty good life), and then the AI does whatever else it deems important.
Bad news for you -- there's a 100% chance we all die. Sorry to be the one to tell you.
And you accuse the 'other side' of 'suicidal apathy'??
You should put down the AI and do some self-reflection on how you came to hold these views.
I've been in the field for almost 2 decades, and actively thinking about AI risk for over 15 years.
I've always held the controversial opinion that there may come a time where we might unfortunately have to consider using force to protect our civilisation from the threat of ASI, but I've always reframed from openly discussing that because I'm so personally against any use of force and there's always been time for more peaceful options.
Several months ago I changed my tone on AI risk to stop worrying about the optics of what I'm saying because it's too late in the day for me not to speak plainly at this point. Similarly over the last week I've decided I can no longer reframe from advocating for the use of force (if necessary) to stop rouge actors from playing Russian roulette with civilisation.
Please understand I don't want anyone to be harmed. Perhaps you don't agree with my suggestion, but my stance ultimately comes from a position of harm reduction.
> preemptively against a 20% number you just made up?
It's a probability estimate. Happy to expand in detail on why 20% specifically, but first you accept I can't tell you what's going to happen in the future with certainty right? To some degree any prediction you might have about the future will be "made up".
There are dozens of escalating, yet fundamentally less destructive means to prevent a data centers from continuing to operate (if that's the goal). Not starting at those more modest interventions is political malpractice and in opposition to your harm reduction narrative.
Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
If you're referring to the "nuke the data centers" line in my bio, that's tongue in cheek. It's an expression of my sentiment towards AI at the extreme.
To be clear, I don't think any country should seriously be considering dropping nukes on data centers. Targeted strikes on data centers operated by rogue private companies or nation states is acceptable to me – assuming pre-warning of intent and appropriate regulatory foreshadowing.
I suspect we agree this is reasonable?
> Your P(doom) 'probability estimates' are almost certainly based on unbounded exponential growth curves.
> Nature abhors such things; I don't make any plans around their existence, and you shouldn't either. Limiting factors always emerge and dominate the curve, tamping it to a logistic at most.
No, they're not based on unbounded exponential growth curves at all... I'm actually somewhat skeptical of claims that there could exist an intelligence that far exceeds our own. My assumption is really just that AI will be far faster, far more knowledgable, far more persistent and marginally more generally intelligent than the average human. The first three are already true, the forth we're getting to.
We're on an exponential curve right now and I suspect the rate of progress will accelerate some more as a result of RSI. This will likely continue for 2-3 years (if we have that long) before starting to flatten out.
As for my doom prediction, my median concerns are around around the destabilising nature of ASI.
If it can be controlled then it will concentrate power like never before, and I believe this will destabilise civilisation. I can go into more detail here if you wish.
However, I'd still put these destabilisation risks at under 50% in total. My concern more broadly is the majority of ASI outcomes are doom scenarios. It's extremely hard to imagine a world in which you have a magic box that can grant any wish that intelligence can grant, but which we have a functional civilisation. You have to basically assume all potentially dangerous wishes won't be granted and probability the only feasible way to do that is restrict who can make the wishes – but then you have the concentrate of power risk.
I largely agree with the premise "If Anyone Builds It, Everyone Dies", so we must try to avoid building it. I think we can build ASI safely, but we're clearly not on the path right now and I suspect it might take decades (or more) to figure out how we can build ASI with an extremely high probability of a good outcome.
> Get involved with real politics by organizing with other human beings that share your values (posting alone and doomsaying is not praxis), or step back from tech and cherish the limited time you have with the people you love.
I do this. I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
I don't think we realistically have a chance of diverting from this path at this point. I have made peace with this but it makes me very sad and haunts my dreams almost every night.
We already live in a reality with unprecedented concentration of wealth and power in the hands of the very few. This process has produced horrific violence and ended civilization for tens of millions.
The actual impact of AI, so far, has been redistributive upwards (solely through speculation, not via productivity or displacement).
This kind of problem is a standard human social ill. Its solution is social too.
Redistributive economic and political action can equalize power and wealth disparity. AI is incidental in this story.
Your understanding of AI as an isolated perfected intellectual djinn lacks a consideration of the existing and ever-changing labour and social relations that give AI responses meaning in the first place.
It is a compressed, multi dimensional manifold containing the structure of relations between objects. Responses are a reification of a point in that space.
Navigating the space can be interesting, but it is a poor substitute for the relations themselves.
Very few people yearn for the endless, frictionless solipsism silicon valley promises (AI + slave robots), and those that do are overrepresented in your social circles. The books they write are a study in psychological projection and limited political education.
> I've been writing endless comments more recently urging people worrying about their jobs/careers to stop and just spend the next few years with those they love.
This is good advice generally. But it isn't organizing. Join something, meet up, drink bad coffee, listen, take notes, advocate to fix just one bad thing in your community, feel your impact reverberate through the world, and then notice how much easier it is to shake off despair.
If you don't have a community, then no wonder everything feels like its ending.
Does it bother you that the people who are publicly cocksure that P(doom) is moments away are the same people that have profited most handsomely from that pronouncement?
That the 'humanists' that want to do 'altruism' for 'potential future humans' and are the same people that commit fraud and theft at a civilizational scale, then sell this 'intelligence' to any child-incinerating militaries with spare cash?
It's not wrong to want to do good, but if a system that is branded 'do (the most) good' commits great evils, you are morally and intellectually obligated to step back and reconsider how you are spending your time.
Also, I asked a chicken and a feral rock dove what it's like to be not be 'apex' and they burbled at me and kept eating millet and sunflower seeds.
Would you like me to follow up with them? I'm not sure what point you expected them to make.
You hallucinated the "preemptive nuclear war". He didn't say anything about nukes. That's your own invention.
> Does it bother you that the people who are publicly cocksure that P(doom) is moments away
20% is not "cocksure". The "moments" is again an exaggeration.
Every problem is a search problem. Nuke the data centers.
P(doom) = 98.9% (Aug-2026) P(doom) = 98.2% (July-2026) P(doom) = 98.2% (Jun-2026) P(doom) = 98.5% (May-2026) P(doom) = 98.8% (mid-April-2026) P(doom) = 98.7% (April-2026) P(doom) = 98.7% (March-2026) P(doom) = 98.5% (mid-Feb-2026) P(doom) = 97% (Feb-2026) P(doom) = 94% (Jan-2026) P(doom) = 93% (Dec-2025) P(doom) = 95% (July-2025)
It's was an expression of my sentiment, not a policy position I'd support. I wrongly assumed that was obvious, but some people are making bad-faith assumptions about me and my sanity.
Not that it should need to be said on a comment thread where I am express concern about civilisation in a post-ASI world, but I obviously don't want to see a nuclear war.
Powerful word, `if`. "You're not only wrong you're a fulminating psychopath" is a perfectly valid response to getting it wrong like a fulminating psychopath.
AI is a math & computer science problem. If AI can do advanced maths and computer science research, then it can begin to suggest useful algorithmic optimisations.
Right now I'm sure the vast majority of these will be junk, but occasionally, even with current limitations, they might occasionally stumble on something.
It's not really whether RSI is or isn't possible, it's really just whether it's the most efficient way for labs to improve their models today given they have limited compute to run AI-generated experiments and access to very intelligent humans who might have a better hit/miss ratio.
Do you disagree with anything I'm saying here? Do you not think current AIs can suggest algorithmic improvements or something?
It's fine, and I'm very used to this, but if you want to have a dialogue I'd be very happy to. I'm a very reasonable and sane person outside of apparently holding some controversial views on AI risk =)
I'd love to know why you think what I said was insane?
Now you are again postulating that there would be no more extreme progress in the near future. That's actually more "insane".
I’m not a mathematician so I have zero clue what “ New upper bounds on sphere-packing density down to the Cohn–Elkies thresholds” means.
not sure how many will get this reference but "AI" for science and math is like super-shoes for runners
at first we are blown away by the impossible improvements including sub-2-hour realworld marathon and every other PR/CR/WR is dialed down
but then the improvements slow and reach a stall point because of the limit of technology and the source of the achievement
ie. sub-2-hour marathon yes, sub-1-hour never happening (rollerblade inline-skate record is 1-hour marathon)
The fact we see a lift is not the same as evidence that the lift is unbounded.
The lift being finite is supported by the fact improvements have come at the edges: improvements from human feedback, improvements in harnesses, improvements on model compatibility with harnesses, improvements in inference efficiency with new architectures, etc. If we were just training better models from scratch that would be one thing, but we are just making better use of a tool we've developed.
As a programmer, I am mostly interested in whether my role is sustainable long-term and whether the models will get better. I don't feel in jeopardy yet, but two more years like this and the calculus of hiring software engineers could shift even further. QAs are already overwhelmed with work
but with super-shoes more and more runners are qualifying for boston marathon and even olympic trials marathon where it would have been impossible for them previously
and that's what "AI" currently does, it allows average people to immediately "pick the brain" of every expert in every field, in every scientific paper, without previously reading a single other google result, something that would have been impossible for them previously (super-shoes for the brain? too far?)
but "AI" isn't creating new knowledge, it's just stitching together existing knowledge from patterns that would have taken years by human hand if even possible at all, it's going to "hit the wall" eventually (in its current form)
basically everything Benjamin Franklin did was trial and error because no-one understood what electricity was in the slightest
almost everything Edison did was trial and error too, he had his lab try thousands of materials for his long lasting lightbulb filament
even the most advanced "AI" today is just machine-learning going through everything already known trying to piece together previously discovered facts, admittedly at levels and detail impossible by human hands
but that means there are limits and it's not really "AI"
I have seen it produce tentative genetics for experiments, just like a scientist does. Then the data comes in and it can evaluate the data from the experiment just as well. One just had to give it a lab budget.
I heard that Gary Kasparov was impacted by AI chess, but at least he still seems to have a job, so don't give up.
My point being that AI math is a narrow skill just like AI chess and implies nothing about generality (AGI).
Sarcasm begats sarcasm.
It is true there hasn't been a reliable computational approach to solving these problems before. But do these proofs contribute new ideas to the mathematical corpus, or are they simply an effective method to exhaustively search the literature for the right combination of existing tools to apply to the problem?
Essentially, did these problems seem like they had an intuitive answer and were feasible to prove before, just not high enough value targets for an expert to invest time into? Or were they fundamentally difficult prior to this point and it appears that AI has done something more than just throw the problem into a big solver.
The sofic groups question was the outstanding question about sofic groups. Almost everyone thought that non-sofic groups existed, and there were plausible candidates, but proving a group was non-sofic was out of reach. Now that we know how to do it once, we can probably do it a lot more.
The Connes rigidity conjecture I think people thought was false, but it was a provocative claim to make. The significance of conjectures is frequently not that the answer to the question is "yes", but that we don't know how to answer the question. And now, apparently, we do.
a colleague was telling me that the base idea for proving that something is not sofic already appeared in the literature around 2019 or so (this is the "expanders graphs" that are mentioned in OpenAI s paper. no one had managed to find a concrete example though. this doesn't make the result less impressive in any case.
The general consensus of developers is that AI can only do the work of a strong 'junior'. Yet as soon as we are presented with pure mathematical results, people seem incredibly ready to accept that AI can do more than what a strong student could achieve.
If it works better here than for programming, then I would guess it's because you can give it a very precise prompt, so you either solve the problem or you don't. If you read the prompts people have shared for problems like this, then the instructions are basically "Solve this problem. Don't give up early. Don't solve a similar problem."
But the wording of the result makes it sound like we don't know what the lowest possible complexity bound might be. So, prior to this result did we think there couldn't be a lower possible bound? Or did the arithmetic circuit community think there were lower possible bounds but didn't see it as a high value target for experts to tackle (maybe a problem that was instead regularly given to students to study).
For example, despite our best efforts, the state of the art lower bounds on time complexity of algorithms for solving 3SAT is O(n). In contrast, our best algorithms for the task run in time roughly O(2^n). That’s an exponential gap. This is despite decades of trying to find lower bounds.
Wow, that’s pretty stark.
“What’s the minimum time it would take to solve this problem?”
“Well, at the very least you’d have to read the input the whole way through”
Your worry.... is because they used the word advanced? For marketing? The word is used very appropriately here. There were PhD's who spent a big part of their career tackling these problems.
I'm trying to understand if these specific problems were the kinds of problems that would have justified an expert investing weeks or months to solve. Or if they were the kinds of problems that would normally have been given to students to investigate.
Mundane incremental research is cobbled from existing citations that already appear nearby in the record.
Basically, innovative research is a measure of bridging thought and domains that were previously not bridged. It's quite concrete as a measure in the citation record.
So we can know pretty conclusively.
Puja Ohlhaver gave a talk on this[1], and ran some experiments (that I had the pleasure to support on)
[1]: https://www.youtube.com/watch?v=guLDNMAOn24
Also on HN front page today: AI's debt binge can't last, hidden borrowing reaches $1.65T (fortune.com)
https://news.ycombinator.com/item?id=49160699
but it seems less likely to me than before that the types of math/science discoveries will explicitly unlock better software performance. in some sense this fits our intuitions. when top tech companies use math PhD type employees, they have them stop doing pure math research and instead focus on software engineering. these people are often very good at software engineering but not due to recent discoveries in academic mathematics, it's due to their general intelligence. to me, this is evidence that the models are getting better but does not make me think we are on the cusp of a foom style fast takeoff enabled by revolutions in frontier math (i also posted this on twitter @mlipman13)
If I had to create a tagline to describe my opinions about AI in a single sentence, that’d be it.
It’s possible to both hate AI and be impressed by it at the same time. Lying to ourselves about its capabilities does us no good. It’s emotionally difficult to do, but people need to come to grips with what’s happening and shake themselves out of a state of denial.
The AI results are clearly impressive. But these sorts of things are also in the ballpark of what human effort could solve given enough attention and time. Though it is hard to say.
Like, these would be best-paper awards at many top CS conferences.
You seem to overlook a simpler barrier. To make these advances, they have to be possible. A 15% improvement in GPU kernels doesn't evidence that significantly more improvement has been left on the table.
Incredible?
> open ai announced like 15% improvement by fixing gpu kernel issue
That is... ordinary software optimization.
Edit: also here’s a opencl 30% compute perf increase documented here : https://m.hexus.net/tech/news/graphics/74425-haswell-systems... that i just googled for
> we are seeing frontier level math breakthroughs (ie performance that would put it in the top 100 or 1000 mathematicians in the world if it were a human, meaning top .00001% or 800/8B)
i think you have misunderstanding of what mathematicians do
They get to make cool 3D plot visualizations of functions so obscure to me that they’re named after someone who is still alive - and/or get to work on cryptography for the NSA - I think?
https://x.com/gro_tsen/status/2082483878480977959
Nothing is perfect.
> You are an expert in the field of mathematics, with decades of experience. You are a reviewer of proofs, etc etc.etc.
It also links to a paper written by an LLM where the model "reconstructs how the proof came together" based on the unpublished reasoning traces: https://cdn.openai.com/pdf/reasoning-walkthroughs.pdf
I wish they'd publish the prompts though!
I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications) that the lean proofs are checked against. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.
This is an extraction from that of the actual theorem statement (39 lines): https://github.com/openai/ten-proofs/blob/94bc0feb6a9ff12c7d...
As long as you are not missing important information, how you word the prompt does not have any effect.
check here : 1. high dimensional sphere packing https://muchmirul.github.io/conjectures/sphere-packing/
2. multicolor ramsey number https://muchmirul.github.io/conjectures/multicolor-ramsey
(Well to be more precise we’re going hybrid because of unknown SCAs.)
So I would like to counter your cynicism with a “YMMV” depending on who you work for.
What does "support" mean in this context?
We often hear this. Likewise to get things done faster. And to get things done cheaper.
Almost never to get things done beteer.
Like cool my lung xray only took minutes to determine if I have a lesion instead of a week or a few days, but I still have cancer.
More importantly, if you can screen for cancer in a way that takes minutes instead of a week, imagine how accessible this technology will become.
- travel and restaurant recommendations. my last few outings have been entirely LLM-advised and they turned out excellent. LLMs seem to have ingested every single Google review, photo, and menu of every business on Earth and can answer very nuanced questions like "is the garlic chicken at <restaurant, city> garnished with coriander?"
- fitness, nutrition, accounting, therapy, medical, legal, immigration advice (sure it's not a real professional but you know what, it's pretty fucking close, and any capability gap is made up by having perfect two-way communication which you don't get when talking with a human)
- coding (work, side projects, personal tools, documentation & pricing questions, "review this code", etc).
- I start reading most articles with the prompt "Summarize this article: <url>". I just started a non-fiction book by pasting into Claude: "There are 12 chapters in the book <book-name>. Can you give me a 2 sentence synopsis of each chapter?". It reduces the "activation energy" hump and screens if it's worth reading at all.
- I use the LLM in my Tesla for on-the-fly advice for parking and other things. You can simply ask "what's the best Boba place around here?" and it will give you a decent recommendation. You can also follow up with "does this place have ample parking?".
- I use the LLM in YouTube to summarize videos and ask specific questions and/or get timestamps to the parts I care about.
If your critical thinking skills are strong then LLM is a literal superpower.
Where can I find this one?
I have a janky pipeline built on top of yt-dlp and I've been wondering how many more years until they make it so I don't have to do that anymore.
(Though sometimes I'll ask Gemini — the YouTube integration is the only reason I use it these days.)
2. Do LLMs reduce the loneliness epidemic?
3. Do they reduce population aging in almost all counties around the world?
4. Do they reduce political polarization?
5. Do they bolster democracies?
6. Do they decelerate climate change and general environmental destruction?
7. Do they accelerate sustainability and the circular economy (not circular financing!)?
8. Do they reduce the workweek and give people more free time for family and hobbies?
Etc, etc.
I would hold off on calling anything a "superpower" unless it solves the hard problems in life. Heck, computers and even the internet barely score better than LLMs when measured against the important things in life.
Well maybe not the LLMs, but AI and robotics have a lot of potential here.
https://youtu.be/bsNTv8t239Y
It's still the early days though. Try again in 15 years!
The QoL improvement from LLM is somewhere in the ballpark of going from dumb phones to smart phones for an average person (able-bodied, local to the area etc). It's nowhere near Haber-process or penicillin.
Sure. Often to zero.
Keynes was hoping that our workweek would be 20h/week by now, through technological advancement. Instead places like the US are thinking about adopting 9/9/6 and some US states are legalizing child labor again.
That's a surprise. E.g. I've found these bots pretty poor at body language.
> "There are 12 chapters in the book <book-name>. Can you give me a 2 sentence synopsis of each chapter?"
Really? You tell the bot the chapter quantity?
https://www.reuters.com/world/us/americans-fear-ai-permanent...
But I don't think the argument needs to be "AI is like gambling". The argument only needs to be "humans often behave irrationally and even self-destructively".
Perhaps you can ask Claude to explain it to you.
A lot of humans are incredibly bad at allocating money.
AI probably did not take your job yet. How many AI queries did you use last month, and how much time has it saved compared to digging through the web?
I want to know:
1. How many total problems were given to the model, and what percent were left unsolved at what cost before giving up? 2. How many attempts did you give the model at solving these problems? 3. How expensive was the harness, e.g. did the model have access to a job cluster?
https://x.com/polynoamial/status/2083478171975082334
As a complete guess, it seems like they tested hundreds to thousands of problems with a relatively low per-problem budget
--
The linked tweet from Noam Brown at OpenAI reads:
> And yes we did try other major problems without success. Sadly no Millennium Prize problems (yet).
> But also, we didn’t spend a lot on each problem. It’s possible to push test-time compute much further.
It's not just about requiring to disclose AI use. AI-powered mathematics is a completely valid discipline that doesn't need to be shy, but it should develop its own publication culture.
Pure math is relatively outside my domain, so I find it difficult to grok the exact relevance of the various published discoveries beyond that they are not insignificant, and LLM competence is expanding quite steadily across the field. If this trend continues to the point of LLMs being able to competently expand pure math, it seems somewhat predictable to expect there to be a number of people aiming to find ways to try to keep human mathematicians in the loop.
I've no idea what I think about this one way or the other, beyond that it's certainly a phenomena and one that's going to drive motivated reasoning that may not be entirely sound.
My perspective is more like a FOSS philosophy for math. Even if a closed version has the same immediate effect, it's just better for everyone if everyone can look under the hood and tinker with it.
I would've thought pretty much the exact opposite. "Prompt engineering" was somewhat important in 2023/2024 when the models were much weaker, it doesn't seem at all necessary anymore (unless just "clearly stating your requirements" counts as prompt engineering). Most of the discussion I've seen seems consistent with this?
Methods are only really necessary for results at a meta level, about the design amd evaluation of AI math systems.
shouldnt the paper be the math of the argument? the reproduction is reading the following the proof
Another case I want to highlight is writing GPU kernels as illustrated by the following example: Say I want to generate random number with Normal (0, 1) distribution. Often times the AI written kernel will just generate the number 0. The tests often fail to catch these errors.
Even if the cost was $1 mil for these 10 problems, that's maybe 10-20 math researchers for a year.
Do you really think that if you paid that to humans, they will deliver the same results?
And frankly these "concerns" ignore reality. In any research phd course you're actively told to bite off something small and likely to be provable so that you can prove it (and publish it). Openai telling its computer to do that is no different that your phd advisor telling you that.
The post that started this sub-thread asked:
> 1. How many total problems were given to the model, and what percent were left unsolved at what cost before giving up? 2. How many attempts did you give the model at solving these problems? 3. How expensive was the harness, e.g. did the model have access to a job cluster?
I think it's an extremely relevant question to ask, because it helps us better understand the current state of AI being able to handle math, for exactly the reasons I outlined. I was arguing against the idea this is just a reactionary anti-AI kind of question to ask. It's not! You can be very impressed by what AI is capable of in math (I am) and still think those are really interesting things for OpenAI to disclose (I do).
OpenAI specifically called out a $2000 per problem average, which implies something that's probably not true ("if you throw $2k at us we'll solve an open problem for you"). It would be cool to know what the actual number is.
If these 10 problems were solved by humans, it would be pretty impressive, even if it took a large number of researchers! Yet when AI does it, HN commenters suddenly feel the urge to play accountant.
But that's the start of math research, not the end.
The point is to get practice and experience doing research.
Did ChatGPT learn anything from these proofs, that it can build on?
Part of what's annoying people is that ChatGPT is churning though problems that are meant to be motivating. They are problems that aren't worth the effort of human professionals (usually because they are incredibly computation-hevy, so better suited for a computer than a human), so they are good for students to work on.
I know it's more exciting to say "AI disproved a longstanding conjecture" vs to say "it did so AND it took several PhD specialists in the field this many attempts to even produce a prompt that got the model spitting out something useful under some configurations, and many iterations to optimize the configurations, and the prompt itself, and many trials with that configuration to solve the problem. All told we spent more than a typical math academic can hope make in their career."
By not being transparent, they invite skepticism and cynical takes, like maybe it's just that tempered and qualified claims are an existential threat to companies that are fully subsidized by the hype train?
I don't know. Either way, it seems like it would be easy to address these, so why should they not do it?
To be clear, even if that tempered version is close to reality, it doesn't make the models not useful! It just forces a certain calibration of expectations
I say this btw as someone who uses these things extensively, including to disprove an old conjecture my advisor and I were stuck on recently. I know they are powerful and that everything is different now because of them. Let's be sober when discussing them though
That's not normally how people act when they're confident in their product
The cost of running a model is not only $/token, but the salaries of the people managing/orchestrating the models, deciding what theorems to try, etc. Once we factor that in, how much are we really paying per theorem?
The other factor is the subjective component of the value of a theorem. Not all theorems are created equal, and the only way to really measure the value is to ask professional mathematicians for their opinion, or publish the results and look at citations over months/years.
Once we have both of these nailed down, then we can start to do the cost/benefit analysis. To be fair, we should actually compare three groups: human experts, hybrid agent/human expert teams, and fully autonomous agents.
OK I’ll grant that it’s not your obligation to be my search function (despite you making the wild assertion in the first place), so instead can you just point us to the latest grad student solved problem of this level that you know of?
It's a marketing post from a huge company. Only the naive would view it uncritically without assuming it's been written carefully to present the results in the best possible light while skirting the boundaries of outright lying.
Imagine 2 years from now: "yes, GPT solved the Riemann Hypothesis, but cmon, it's just a marketing stunt to hype their stuff, it was probably Terence Tao doing the work but he's so obsessed with hyping AI that he doesn't want to take credit"
We're saying look critically at the claims for how it was done, that it only cost $2000, etc. it would be extremely easy to run 100 sessions that failed, each costing ~$2000, and then just publishing an article about the one that succeeded, for example.
This goes double since it's an internal secret model (Astra) so nobody else can verify the results.
More generally, do you expect that there's some capability threshold where people will no longer study or analyze AI model outputs, and instead just sit there slack jawed saying "so cool!" every time OpenAI announces novel ones? I don't really understand why that would be or why someone would want that. If you're interested in the pure experience of a complex machine outputting satisfying results, I'd recommend getting into sports cars.
Look at their recent claims about their model "escaping" - there was literally a Guardian article calling them out for being hyperbolic! Again, it wasn't that they lied, their marketing department is too savvy for that. They just present it in way that's, well, marketing.
As for the actual result, I'll look for secondary posts by actual mathematicians and draw my conclusions there, not from this marketing blog post about results from a secret model.
Company X does not make money from proving theorems but does make money from selling you a service which supposedly proves theorems. Company X then proves some theorems and explicitly calls out they were very cheap to prove using its service.
And you think you're actually clever for taking these facts at face value? Interesting.
i would classify you as a flat-earther if that happens.
brother like 3 people have pointed out what they're skeptcal of is cost not LLMs - at this point you're willfully misconstruing what people are saying to you just to get a kick out of repeating your same tired strawman.
If OpenAI solved Reimanns hypothesis and the first comment is says something about lack of transparency and marketing, i would say it’s ignorant.
If OpenAI claimed the conjecture to be true but provided no details about the proof then the first comment should absolutely be about lack of transparency.
do you really imagine a scenario where OpenAI would claim to solve it and not give details about the proof? how is this even possible? why would anyone believe them?
That's one of those phrases you can use to dismiss opposing viewpoints without actually engaging with them.
I don't think that comparison to p-hacking is fair. I mean not reporting price of all run is nothing like committing scientific fraud and fake results.
Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.
A couple small ones that I've seen (example here [0]), but not anything of the magnitude that OpenAI and Anthropic have put out. Likely just related to token limits.
> Another question I have is whether or not OpenAI 'simply' hired capable combinatorics researchers to work on problems, and they have, and the use of the model is incidental / secondary to their work.
I think their output has reached a level that precludes this possibility, but I of course don't have any hard proof.
[0]: https://www.reddit.com/r/math/comments/1uxj3cy/after_openais...
I have no affiliation whatsoever with any AI company, nor any formal education outside high school, for what it's worth. Simply being curious and persistent can get you quite far in my anecdotal experience.
https://arxiv.org/html/2605.22763v1
> Our most capable agent autonomously resolved 9 of 353 open Erdős problems at the per-problem cost of a few hundred dollars, proved 44/492 OEIS conjectures
> Our full-featured agent autonomously solved 9 Erdős problems out of 353 attempted, including two questions that had been open for 56 years
Note _had_ been open, not _have_ been open. Can you clarify?
But "had" still doesn't mean what you are implying: once the model solved the problems and the solutions were verified, the problems weren't open any more, so a later description using the past tense is totally consistent.
> "Already, there are very few mathematicians qualified to verify OpenAI’s new results. As progress continues, that number will approach zero."
Not that I'm good enough at math to have any uniquely formed opinion, but after reading commentary from people who are, my impression is that these new results are bamboozling the humans due to using tools from so many disparate areas.
At least, we can say that there isn't a single human who is smart enough to understand all ten proofs, even if there is a collective sense in which all proofs are understood.
Now I understand that it’s mostly the super stars benefitting from the increased attention. Folks who are less established don’t share in that glory. But on the other hand it seems like an exciting time to go even deeper for in various specialties of math by deciding where to focus these powerful tools. For every conjecture defeated some seven or eight new ideas open up. Our path through that combination will be set by creative and curious human mathematicians.
[edit: deleted a distracting comparison to Chess]
Which is less interesting work. And you probably need to do the hard grunt work by hand first to develop the skills and intuition to be able to verify an AI-generated result. So you can’t outsource everything to AI without loss of skill.
these LLMs are great are generating arguments but they don't ask questions, we will need mathematicians to shepherd them into more discoveries
i really want to see open weight models crack some breakthroughs
Why don't they? That sounds like an important problem to solve.
Along with the fact that they can't learn anything (after the training stops).
If you translate that to math, then all you get is math competitions, not math as a career. Of course the translation isn't nearly exact ... there's a lot more room for professional mathematicians because the math space is far more vast than the chess space and can't generally be cranked out mechanically (we have proof).
P.S. The response is nonsense ... I explained exactly why it's awful (others have too) and the response doesn't in any way refute the explanation ... rather it offers up a ridiculous strawman.
Was this different before chess computers were invented?
[0] https://arxiv.org/abs/2606.31640
Thinking of this a little bit with the perspective of every new proof as a burden, dumped for review by actual mathematicians.
The most disappointed folks will simply drop out, but the enthusiastic ones will keep going and with luck make up for the ones who decided to quit. Chess and go certainly went this way.
For example, will we see advances in material science, medical cures, etc?
Would love to read about some examples of practical impact.
OpenAI and Anthropic are too greedy for cash to do anything of the sort.
I don't expect this current economic cycle to bring anything else that will directly greatly improve the life of the average person on the planet, more than it hurts it.
But it might be not good for human brains, because we trained our brains with these problems and our brain optimized search space in some ways, and yes, we also couldn't solve some these problems.
Now imagine someone gets stuck with a problem which could become its own theory, but they will solve it with LLMs and move on to solve their primary problem, because they don't realize how other problem was a big deal. If theory is not formalized, then it won't contribute to the search space for other person, solving different problem.
All in all:
* people's brain will be shaped differently
* we will lose search space optimizations in our brains
* we lose new theory contributions, which increases the search space to help solve other problems
Maybe good AI paper writing is further away than I thought...
I'd honestly rather they just automate every job at that point.
It is indeed true that all models are, at their core, predictors of what occurs next in a sequence. But I think it's worth exploring the implication of what that means. Because when fed tiny pieces of information for a few tasks at a small scale, this results in something that sorta, kinda works. Or, works surprisingly well.
But when scaled... When the amount of information starts approaching the sum of all human knowledge, the tasks start approaching all useful applications of that human knowledge, and the fidelity of the predictor approaches incomprehensible sizes, the starts encodes / becomes (I'd argue it becomes) something that can model all human knowledge.
It feels wrong to say that, but let me explain, what is the best way to predict the behavior of a ball constrained in two directions that bounces with initial vertical velocity v(y) (y is up / down axis) and horizontal velocity v(x) (x is side by side in 1d) ?
If we purely look at it via a graph, it's by modelling the function of acceleration under earth's gravity.
If only a few points are given to you for this and you can't make something really sophisticated, then you'll make something that's rough that kinda sorta works and then call it a day.
But... if the number of points keeps increasing in number, precision and accuracy as well as the number of examples (assumed that data about air pressure, velocity and all other factors is included alongside these points), the fidelity with which you can replay / tweak the function keeps improving, and the number of times you can iterate keeps increasing, you'll eventually create a function that models that process so well that it intrinsically contains a good enough model of the deformation of the ball (provided the dataset contains information about elasticity of the ball's material, its dimensions and mass etc..), the nearly negligible (under normal conditions) effects of the ambient environment (provided there's diversity in the number of environments supplied), the oblateness of the Earth and minute changes in the gravitational field (the length of a seconds pendulum varies depending on where the experiment happens. It's presumed that all of the prior set of experiments were repeated across the Earth and the subtle, but real deviations were faithfully recorded)... and so much more.
A machine trained on the above with a large number of parameters, measures to prevent "laziness" and enough reps for high fidelity across a large enough dataset would start to approach a simulation of the ball falling. Because to predict what happens next in the sequence, you must model what's occurring in the sequence.
Now imagine doing that for other tangible and intangible things in this world. For all of human knowledge across all fields of endeavor. All experiences. No matter how noble, ignoble, notable or ignorable. But putting all of it into the soup that's this machine. Then at larger and larger scales, you eventually start encountering "good enough" models (in modelling the falling ball sense) for even the most hard to quantify / qualify things like grief and joy. At some point, by simply trying to predict what it has been taught ought to be the next part of the sequence in say... human interaction, it starts to make a model of something that hews ever closer to a full fidelity theory of mind.
Is there evidence for this? Kind of, yes. There are early indications that as machines are trained for an ever larger number of tasks at larger and larger scales, their internal representations converge. It's called the Platonic Representation Hypothesis. Overview and paper here, https://phillipi.github.io/prh/
It is my opinion that these machines are displaying a new form of intelligence that human beings haven't quite encountered before. They are the sum of all human knowledge made manifest and given voice by processes that nudge (bit-by-bit) what kind of step it ought to predict for the next part of whatever sequence it displays.
In my mind this means that, of course, these models can create new knowledge. This strains the analogy, but with the sum of all human mathematics within them, they can "reason" via the act of predicting what ought to come next.
Of course, these machines are "surprisingly" good at a lot of things the larger they get, because what the labs have created here is a rough version of humanity's collective knowledge given form and the ability to say hello.
I suspect that the current generation isn't close to the "true frontier" of what these machines could be. They are nowhere close to the sum of all human knowledge and endeavor. They are quite a way there, but they haven't yet achieved true completeness for domains where the data isn't so public.
I think it's the most exciting scientific and technological breakthrough of my lifetime. And I can't wait for us to get close to the true frontier of all domains.
My negative feelings towards AI are about energy use and inequalities, that kind of stuff. It undeniably works well, but whether or not it is better for society or the planet is a lot less clear.
But I want to be reading about that from the comfort of a home, with a full belly.
Training the model is going to be amortized over other uses.
Say that it turned out that the total cost of the proof of the Erdős unit-distance conjecture was $50 million.
Then the question really becomes: yes, these models are capable of proving important mathematical results, but at a very high cost. Is it worth it?
If a mathematician applied for a research grant of $50M USD for proving the same thing, they would have been laughed out of the bank.
What's more is that when you have a research grant, you train PhDs and postdocs, you hire new staff, and you disseminate. That is, you get much more value for the money spent.
I'm just curious what the cost is.
Hours needed for prompt + Hours needed to check result + API costs.
You don't say "well let's add together the total yearly compensation of all the engineers and mathematicians at OpenAI that were involved" and throw that into the total cost. That's simply nonsense accounting.
The actual comparison you are making is some university researcher weighing between getting a grad student (several tens of thousands of dollars) vs typing up a prompt and sending a request to OpenAI for inference (as mentioned in the article, around $2000 in API and maybe a few hours for the prompt and harness).
https://x.com/polynoamial/status/2083470822258467194
> We helped prepare the manuscripts and formalize the proofs in Lean, and we take responsibility for their correctness
Offering to take responsibility for the correctness of a proof written in Lean feels like volunteering to be the fall guy in case someone finds a flaw in basic arithmetic, no?
I just want to state that having "lean proofs" that build (checks) does not mean the actual real theorems we care about hold. Ignoring lean kernel bugs, ultimately a human (not an agent) has to verify the lean encoded theorem statements (specs/specifications), that the lean proofs are checked against, indeed correctly encode the real theorems. For non-trivial theorems such as these, this is an arduous and tricky task where even a little mistake could be fatal. AI generated lean encoded theorems can be huge and difficult to understand. I wonder if anyone reputable has audited these specifications.
https://leanprover.zulipchat.com/#narrow/channel/270676-lean...
As I currently understand it, all we know is that:
- a mathematician produced a Lean-verified counterexample to the Collatz conjecture, demonstrating a bug in the kernel
- he claims that LLMs were involved somehow but pointedly refuses to specify how
- he admits that he knew about the bug before publishing the counterexample to his repository.
Perhaps not a joke (although it sure seems to me like they discovered a bug and thought falsely disproving the Collatz conjecture would be a flashy way to announce it), but at best extremely sensationalized by the above description. If you have additional context I would be happy to hear it!
Apparently you don't.
"These are real problems mathematicians and computer scientists have been working on and were unable to make progress on."
Who says no one was making progress? Who says openai has made progress? How would anyone not working on these specific problems, witho the time to dig into openai's claims, be able to tell? Why should this not be lumped in with all the other ai hype being pushed?
"The mathematicians I know are saying that the latest crop of models is changing the way people do research math, I think that's a pretty big deal."
Who? And doing what?
We have been hearing the "this generation of models is the one" type talk for years and the only concrete "big deals" are what? A tool for college students to write papers? A replacement for, now enshitified, google search? The fact that now you can fake tons of stuff to support a position or claim tons of stuff that goes against your position is fake?
> Who says no one was making progress?
Let's look at the Jacobian conjecture, since that was the open math problem I was most familiar with prior to its solution. Yitang Zhang, one of the worlds most renown mathematicians (famous for his lower bound on the twin prime conjecture) spent 8 years working on this problem with his advisor (who himself is a renown mathematician) and turned up completely empty handed. His advisor described it as a "waste [of] 7 years of his own life and my time" [1]. Of course, these two were not the only ones working on this problem for the almost 100 years its been open, but they should have sufficient credentials to show that they were not fools or amateurs.
And in a single afternoon an LLM disproved the conjecture. How is that not an extraordinary feat of technology?
> Who? And doing what?
A close friend is studying differential geometry in a PhD program. Sadly I doubt anything I say on his work will convince you, so I will instead offer two anecdotes:
Terrence Tao (widely considered the worlds greatest living mathematician) has said AI is precipitating "a crisis in the foundations of mathematical values and practices" [2].
Timothy Growers (fields medalist & one of the leading researchers in combinatorics) has said that the latest models are now at the point where they are "producing a piece of PhD-level research in an hour or so, with no serious mathematical input from me" [3].
You can find many more fields medalists and mathematics researchers with the same impression. If you look in this thread you can see bluesky/twitter threads from those who were actively researching some of these problems who are in shock at the solutions.
[1] https://www.math.purdue.edu/~ttm/ZhangYt.pdf [2] https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p... [3] https://gowers.wordpress.com/2026/05/08/a-recent-experience-...
Thank you for the thoughts and references.
There was definitely at least some progress on the problem. I get the general sense that there were potential counterexamples that were close but not quite enough, and that its possible (or even likely) that Claude built on those in order to construct its solution. I also get the sense that when Zhang was working on the problem it was believed that it would be proved true, but since then there were bounds found on the problem that pointed researchers to believe it was false. I am not a research mathematician in this field though, so I could definitely be wrong.
Also, in fairness to Zhang, I believe the dissertation he ended up writing was focused on the 2D case in particular, which is still unsolved (the counter example is only for 3D and above). I cannot imagine that anyone looking at the 2D problem was not also looking at the general case as well though.
Look at this two threads.
Edit: Oh, are you suggesting they just use it to privately improve their models? I imagine a few more correct proofs would have a very marginal benefit, if any. Also, they'll probably just get extracted, meaning it still gets out but OpenAI doesn't get to fancily announce it themselves.
However, I was looking at the proofs and reason explanation and openAI should be more explicit in how the work has flown. I find the models have jumped hoops in some places of the proofs, that can be hard to track. In fact, when a paper is published you usually get a review and if no reviewer understands they ask you to further explain the thought process. It will be fun to see if this happens here.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
Which is very suggestive - if after everything they are not fully loaded then the next gazillion data centres being built look unlikely to be needed.
If anything it proves more data centres are needed. That's literally the only reasonable conclusion from this news.
Are you arguing there is not an AI bubble, and that all the DC buildout is fine, going to be profitable etc?
I am not looking for a online slanging match - just looking for a different point of view
I’m not participating in the slinging match but it’s very very weird that you think it’s some established thing that these companies won’t make profit. A lot of hubris must go in this kind of thought. Like.. do you all think everyone’s playing musical chairs?
[1] https://x.com/henryquantum/status/2083623695436623915?s=20
Mathematicians will tear it to pieces if any of it is fake!
These companies desperately want a return to serfdom. If they didn't come off as so anti-human the public backlash wouldn't be so great.
Look at this two threads.
This was not a problem that was for sale
And yet this is the exact same company that has screwed up their android app so bad that the latex N^3 rendering problem makes it so having it explain it to me crashes the app.
Truly jagged beyond belief.
I'm sure they must do some of this type of work, right?
What I had hoped for was something more ambitious: using AI as an arbiter in economic, political, and social debates, one capable of weighing evidence, exposing trade-offs, and helping us make decisions that produce better outcomes over the medium and long term, even when those decisions conflict with powerful private interests.
I suspect, however, that this is not a particularly urgent goal for the people funding and directing these systems, many of whom live far removed from scarcity and its consequences.
I agree, some of the problems are more difficult. I don't think that's the case for all of them. And, besides, these companies could be demonstrating how to approach problems and where their users could spend tokens to help with these problems.
Should not these companies try to work on these problems _because_ they are difficult?
We already know how to solve all of these issues. What we lack is collective political will.
And yeah, the lack of collective political will sucks. It would be naïve, however, to think that there is no value in ensuring longevity in our current and future infrastructure. And improving it to sustain the population giving these companies their value is an obvious win.
Little sacrifice actually
> Solving the issue would be doing it without sacrifice
So... you are expecting magic?
LLMs cannot create resources out of thin air.
This is division of labor, and it's a good thing. I'm sure OpenAI employees, who are very well paid, donate some money from their salaries to others working on the areas you're citing: probably more than you think, I say that from having attended some EA parties back in the day.
But that isn't my point: my point is that a company which makes brushless motors should put most of its time and money into solving the "make and sell brushless motors" problem, and if they or their investors feel like they need to do more for the world, give money to the people who have the time and ability to do things about that. There are a lot of quality-of-life improvements which need brushless motors.
Next question is how useful their product (OpenAI, I mean) is to more focused do-good-in-the-world professionals. I'm sure that varies quite a bit. For getting the homeless off the street? I conjecture, not very useful. For 'complete the transition off fossil fuels'? Extremely useful, no one in that field knows how to do their job without AI in summer 2026. I'm certain of this.
Sam Altman has said "If superintelligence can't discover novel physics, I don't think it's a superintelligence." Is that the test? How far away are we from AI discovering novel physics? It seems within reach.
Just maths isn't really general enough for the G in AGI.
Imagine humankind meets another race, another race shares it's scientific knowledge and humans accept it without experiencing process of discovery. In that case do we really got this knowledge? If we follow machine discoveries like we follow problems in textbook then we acquire knowledge but we don't discover anything. We follow.
There whole lot of philosophical questions that aren't attacked now. Are complex systems sentient because consciousness is emerging behavior? Then should they have rights? Philosophy is part of humanities and science (is/used to be) part of philosophy. Should we accept non-human knowledge in science? Maybe it's altogether different thing from science, yet very similar.
And no, science is not a branch of philosophy and not everything is a philosophical question, despite what the philosophers like to say.
AI development is likely to be more compute bottlenecked than solving math problems since validation of any algorithmic improvement would likely require significant compute. But you could imagine that at this point it could be economical for a frontier lab to task 10,000 agents to work non-stop on finding novel algorithmic improvements then validating the top 50 out of 1,000 candidates on a GPT-2 sized network.
I would suggest RSI is now very close. The singularity could be less than 6 months away. I'm not saying I'd put a high probability on that, but I'd give it at least 20%, and I'd double that if looking 12 months out.
I know I'm just a crazy man shouting at the clouds, but please take to the consequences of this seriously. I understand that for whatever reason AI risk seems abstract and doesn't seem real, but this should terrify any person thinking logically about where this could all be heading.
We haven't even solved the most basic AI safety problems yet. RSI right now would almost certainly result in an extremely bad outcome for humanity.
I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.
Is there good historical data on some measure of strength across representative populations over time in the modern era? I'm doubtful.
We do know that the introduction of agriculture diminished strength:
"Bone mass was around 20% higher in the foragers - the equivalent to what an average person would lose after three months of weightlessness in space.
After ruling out diet differences and changes in body size as possible causes, researchers have concluded that reductions in physical activity are the root cause of degradation in human bone strength across millennia."
cam.ac.uk/research/news/hunter-gatherer-past-shows-our-fragile-bones-result-from-physical-inactivity-since-invention-of
What can we do to make conversations around these incredibly exciting and important topics more constructive? HN is where I expect to read expert comments on these topics, has this style of conversation moved elsewhere?
[0]: https://news.ycombinator.com/item?id=49157930#49132926
None of this is anything out of the ordinary; this kind of thing has always happened. The only real story here is that moderators sleep sometimes.
I for one am really fascinated by AI's advances in science and math and would like to talk about it somewhere without the constant flamewars...
People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps leading to crazier and crazier results. The much more interesting question to me is what will be consumed by the exponential like math seems to be, and what won’t. Writing has been much more stubborn, but I’ve noticed Fable to be quite a big step up there as well. How about politics? Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation?
And then there’s questions like, even if AI can answer increasingly complicated math questions, will we still need mathematicians to translate results to the real world, verify them, or decide where to push the frontier?
For people who have been paying attention to accurate predictions leading to our present state of the world, HN has collectively been reactionary, incorrectly dismissive, and incredibly behind the curve. In public, people are better informed on LessWrong and AI Twitter circles. That's where frontier researchers are. Barely here
Just glanced at it and it looks pretty good!
Yes. And there are many of them. I wonder what would help them come to terms with it. Seriously, people are going to be grieving over this. Loss of identity, loss of social standing, ideas of entire future lives that will now never happen. The greatest crime people may hold AI guilty of is taking away their dreams.
I think recognizing and accounting for your own personal biases is one of the requirements of the being an intellectually honest and rigorous online discourse participant.
Things could be genuinely impressive and fascinating even when directly challenge your ego and material well being.
Moreover, mister elite, you don't know why this press release was flagged.
I'm not sure why we should privilege your bitter speculation over more mundane possibilities.
Moreover, if it wasn't flagged at all, like you suggest, then the grandparent was inventing things to be bitterly resentful about... Which is not a behavior any forum should indulge.
It was always mainly a website for employees of an elite.
Is that possible or is everyone else too common to have those?
Never was.
eg. this: [flagged] A migrant surge tests Spain's open policies (economist.com) - https://news.ycombinator.com/item?id=49131860
is clearly marked as flagged.
Unlike the current submission: Ten advances in mathematics and theoretical computer science (openai.com) which isn't [flagged].
* https://news.ycombinator.com/newest
> if you compare its rank to that of other stories with a similar age and number of points.
Ranking is complicated enough here even before weighting, speed of initial upvotes can play against ranking, number of comments and the shape of the comment tree also affect ranking. And yes, various subjects and submission sources do get weightings that impact ranking.
What's funny, to myself at least, is that any attention at all is paid to "HN front page ranking" - I've been on again off again active here since 2008 .. and can't recall ever really looking at a default HN "front page" ever.
( There's /newest /newcomments /active etc to browse and sites such as https://hckrnews.com/ )
hah, sorry, we are plebs out here.
https://developers.openai.com/cookbook/examples/vector_datab...
How are the sales going?
Which is a low bar, but still.
It's more like they've already automated the parts of the jobs that the humans most closely thought of as the "their job"
My guess from following this stuff quite closely is that these companies are still a couple years away from fully autonomous research staff.
[1]. https://www.anthropic.com/institute/recursive-self-improveme...
I think we've now hit a point where 99.9% of the population gloss over these types of AI advancements because of human competence being insufficient
No human could have published this because it requires paradigm shifts (e. g. Section 5) in multiple mathematical domains. Mastering one of them to this degree is rare, mastering 3+ pretty much non existent for humans.
For some reason comments got moved to this one.
If they're publishing the solutions to these 10 problems, then this is, essentially, the announcement of 10 research papers.
Yes, it's partly for reputation (as are many research papers), but that doesn't mean it's of no value. The best way to advertise is to show that you're providing value.
Stop coping and deluding yourself mate.
To begin with, whether AI is the one doing the discovering or not makes no difference. Any "pure math" person would aim to understand regardless - and would be quite glad that they have a longer paved path.
Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism).
>Any mathematician in academic or industry is more than likely not a "pure math" person (tainted by capitalism)
Ignoring that I meant pure as in non applied math, let's just make it clear: you agree that mathematicians who are against capitalism encroaching on this process should be allowed to dislike it without criticism of being pretentious?
Isn't coping a good and useful mechanism?
https://cr.yp.to/proofs.html
Who is going to wade through this?
It is not peer review if it is all in one company that wants an IPO.
https://garymarcus.substack.com/p/openais-amazing-but-vastly...
https://garymarcus.substack.com/p/two-critical-updates-re-as...
Not that there isn't something interesting in here, but lets be clear that we don't have enough information to evaluate this properly. And as always with these labs, BS takes a lot more energy to refute than it does to spread.
> Astra, a new model that OpenAI is testing internally, is amazing. No denying that.
To sharpen that, I think he's (obviously) interested in maintaining his own brand as "thought leader" and this necessitates de rigeur defense of particular postures.
Sometimes this is easy because the facts warrant it; other times, a bit of rhetorical license is required to preserve nominal coherence and (at least, for the moment) hold certain lines.
This is one of the latter cases, and it's not subtle.
One of the celebrated properties of many intellectual advances or inventions in whatever domain is precisely that it appears obvious in hindsight. It is quite cynical to leverage consensus distrust of large AI players, warranted but also a popular social construction, to insinuate that these are not "real" advances or "real" hard problems, on the grounds they were in some sense cherry-picked.
Identifying the problems amenable to strategies on the table and intuitions (sic) about where bridges might be, is exactly the discerning work that is the core driver of almost all prior progress, but for celebrated accidents and flashes of insight. Anyone working in any challenging discipline knows that those are celebrated and told around campfires precisely because meaningful durable results arising like that is so uncommon.
These two articles make me think of nothing so much as my own durable reaction to the creeping goalposts of AI critics generally: that they often seem to me not unlike a water color cohort scoffing and jeering at the horse, because it got a D on its tensor calculus exam.
Marcus should be on guard against his own cynicism and take care that his assumptions do not prevent clear sight.
I think that is an oxymoron.
""" 1. By 2029, AI will still be unable to watch a movie and accurately explain the characters, events, conflicts, and motivations.
2. By 2029, AI will still be unable to read a novel and reliably answer questions about its plot, characters, conflicts, and motivations beyond what is stated literally.
3. By 2029, AI will still be unable to work as a competent cook in an unfamiliar kitchen.
4. By 2029, AI will still be unable to reliably create more than 10,000 lines of bug-free code from natural-language instructions or interaction with a nontechnical user, excluding simple assembly of existing libraries.
5. By 2029, AI will still be unable to convert arbitrary mathematical proofs written in natural language into symbolic form suitable for formal verification. """
There's still 3 years to go and he's already wrong on 4 out of 5.
1. still not wrong? Unless it's just feeding the audio or screenplay I don't think you can feed AI a full movie in a single context window yet?
2. Not sure, but can you prove this wrong? Can you feed a full, unseen new book and get that kind of answer?
3. Not wrong.
4. I think he'd probably pull you up on 'bug free' - I don't think that frontier models can reliably write 10k LOC without _any_ bugs typically (not that humans can do this either).
An LLM could theoretically try to earn some money and pay a human to do most of these tasks but it's not the point of the exercise.
Have these been tested or are you just guessing?
The non-sofic group one is definitely a big deal - would have been a Fields medal if discovered by a human.
AI has no self-awareness. It's a tool. When you assemble a furniture using a screw driver, the torque force interacts with the molecular forces inside the metal and miraculously it transfers the force to the screw though a clever geometry design, communicating the force to the screw to turn it in a certain way.
Do you attribute the build to the tool? The "system's contribution" is helped by many other things all the way down to chips, datacenters and power generation. If the authorship requires attributing to a tool, then it should happen all the way down.
A slightly smarter highschooler could write these. I could write these. It's clear as day that the LLM, not the human, did the heavy lift. It'd be ridiculous to give full credit to whoever wrote the prompt.
[0]: Not one of the proofs in the linked article, but from OpenAI too.
I think you're over-estimating what a smarter highschooler could write.
A "finite loopless undirected multigraph" could have been explained to me at that age if we'd taken Discrete rather than Mechanics and Pure (and one module of Stats) in my two A-levels* in maths and further maths; but from what I saw of the Discrete module, neither:
nor: would have been something we'd have learned. But more importantly, we absolutely didn't have a feel for how much effort one needs to put into making sure the proof is right, so if one of us had been hypothetically asked to write a prompt it would've been no more than half that length, and missed most of the bullet points.* For those not from the UK: A-levels are between secondary school and university, when aged 16-18. Functionally they are university entrance qualifications: https://en.wikipedia.org/wiki/A-level_(United_Kingdom)
Provability is just going the way of computation. John Napier had to manually compute logarithm tables over decades and was recognised for his work; now that same work could be performed by a 10 year old with a calculator in an evening.
[0]: e.g. "go through wikipedia's unsolved math problem list and solve them".
> In particular, proofs for special graph classes, constructions of cycle covers with some edges covered other than twice, bounded-length or prescribed-cycle variants, reductions to another unproved conjecture, computational verification through any fixed graph size, and candidate counterexamples without a complete nonexistence certificate are insufficient.
which is infact a very important part of the prompt.
Take a grad student with a perfectly good understanding of what a proof is. Their supervisor gives them a major problem to work on. Almost always, the problem is too hard, the student comes back with partial results, and student and the supervisor iterate from there. Now imagine that they have an unusually cruel and unreasonable advisor who tells them, do not dare to talk to me until you've fully solved the problem. This paragraph is exactly that. It's there exactly because the underlying system is smart enough to know that real mathematicians do not work like that.
To a real mathematician you would not have to list those explicitly, he/she would have understood that implicitly from 'give me a full proof'. That listing makes sense to say only to somebody who pretends to be a mathematician, but has not true understanding of how the math works. The models are getting better and better in this pretension, but prompts like that reveal that it is still just a pretension, not a true understanding.
To name a few:
- https://xcancel.com/DmitryRybin1/status/2079904005652893709
- https://archive.ph/2w4fi (https://chatgpt.com/share/69dd1c83-b164-8385-bf2e-8533e9baba...)
I also have a 3D printer. I also have a ChatGPT subscription, and some OpenSCAD chops. I came up with a part which would go into the top of the down tube to the drainage pump, and mostly-seal the down tube itself, with an opening on the side to vacuum out the moisture. This was purely prooompted, I took some measurements, printed bits of the part, refined the shape, and you know what?
It works! I can stick it down there, turn on the (very loud) wet vac, and go upstairs. On a 1.5Ah battery it sucks for a bit less than ten minutes, which turns out to be plenty of time.
So: who made that?
Don't care. I'm waking up warm at night.
Also: me, obviously. ChatGPT doesn't have a fucking furnace.
When the tool is a 3D printer, or any CNC system really, you bet I attribute a build to it.
I could also attribute the operator; there is no contradiction, it's a free choice, just like saying "I am in Berlin" does not contradict "I am in Germany".
What is your mechanistic model of self awareness that yields this conclusion?
> It's a tool
Does your model suggest that tools can't have self awareness?
(*) Even if we hack around this and just do the usual trick of simply laundering statefulness to a higher level, in this case the context window being fed in, I fail to identify (**) a representation of its own state in these bodies of text that it'd be meticulously maintaining. I further fail to identify how it could be hidden or maintained, considering I control like half of it. The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").
You'll sometimes catch models mixing up who's who and how many who-s there even are for example.
(**) I did wish for something hidden though, so maybe it's just concealed? The same way people can encode a lot more of their emotional and mental state than normal into text if they read and write a lot of it, I'm aware of research that suggested the same for LLMs, albeit I cannot cite it. Maybe those phrasing signatures are just alien to me and will never pop out. Either way, I'd expect researchers to stumble upon this during interpretability studies, and either they haven't, they have but it wasn't popsci adopted, or they're keeping awfully tight lipped about it. If you know of anything like this, your turn now, would be happy to learn.
I do wonder how reasonable it is to expect e.g. a single maintained identity though. Maybe it isn't?
(*) Another way to hack around this of course is to just precompute some internal "self-awareness states" and hop around between them. Probably the closest to what the models are actually "doing".
> a hidden representation of self that is continually tended to
This sounds like a personality? They act like they have one of those. It may be an illusion, and even if it isn't an illusion it is unlikely to be anything like the source (us), but they act like it.
> I further fail to identify how it could be hidden or maintained, considering I control like half of it.
Indeed you control everything about a local model, and much of the context of even a remote model. But the state of activations and circuits in SotA AI is hidden in similar ways to those of synapses in your head: difficult to decipher even with probes monitoring the signals directly, and often not emitted at the normal output.
> The best you could ascribe it is a meticulous maintenance of a persona the user is talking to, but then that doesn't necessarily represent the model's internal state, the same way my own words here aren't doing so either. Difference being, I actually have one (I'm "on-line").
While we can be confident that LLMs make up personas etc., it is insufficient to go from "that doesn't necessarily represent the model's internal state" to "therefore it doesn't have one".
> You'll sometimes catch models mixing up who's who and how many who-s there even are for example.
I've, unfortunately, also experienced this with humans. Perhaps they were losing their self-awareness at the time? I do wonder if old-age dementia does that by the end, though the person in question didn't ever get diagnosed with that.
> If you know of anything like this, your turn now, would be happy to learn.
Do you mean like these, or something else?
• https://researchportal.hkust.edu.hk/en/publications/decoding...
• https://aclanthology.org/2026.eacl-long.165/
• https://transformer-circuits.pub/2026/emotions/index.html
Not quite what I meant, but it's also not entirely unrelated I guess? Personality to me is like a natural bias. It does also shift over time, and is also an internal bit of state. I guess in some respects it can also be self-referential, like personal convictions.
> Perhaps they were losing their self-awareness at the time?
I do think it is entirely possible for people's self-awareness to shift, yes. Or more precisely, I do model things that way.
> Do you mean like these, or something else?
They're adjacent, but I more meant something like these:
https://arxiv.org/abs/2410.03768
https://arxiv.org/abs/2310.18512
https://arxiv.org/abs/2605.26537
So basically, steganography. The difference is that these papers investigate from the perspective of separate LLM instances covertly exchanging information between each other. This is in contrast with the scenario I'm laying out, where an LLM's past state is exchanging information with its future state, continuously representing and modulating a concealed internal state of some sort. And then that state just so happening to be some sort of self-referential meta state.
And the best inkling I have towards this is basically: https://www.youtube.com/shorts/WP5_XJY_P0Q
But then I don't think there's enough covert channel bandwidth in the agent replies for anything interesting like this.
I don't see why an LLM could not have a sense of identity or personality while it's evaluating a specific prompt, or even change self awareness while evaluating a prompt since many outputs model a back and forth conversation. My point is that without a mechanistic model of what "self awareness" means, we have no way of truly evaluating such questions, we're just hand waving vague intuitions about what it could mean.
This is kind of also the reason e.g. the HN site guidelines are worded the way they are. Regrettably, forums naturally yield themselves to tit for tat type exchanges, but there's really no reason one could not bounce such vague intuitions off of another. I do not have to be right or wrong, and you don't either. Admittedly difficult when its some intensely contentious topic.
If a mechanistic model existed, there would also be no reason to talk about this in the first place. There'd be nothing to discuss, you'd be simply told how a given model characterizes from this perspective on the model cards.
The reason I insist on mechanistic models is because the original post was making a definitive knowledge claim, and in my experience, the knowledge claim is unwarranted.
A language model (or an image model or whatever) cannot even be sentient, and I think sentience is a prerequisite for awareness.
Even if we express a lot of our subjective experience with words, the language is just a symbolic representation of those experiences. The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.
You can't have an understanding of what hunger or physical pain feel like if you have no need for food or a sensory capacity for feeling pain. You can't understand what loneliness or pride at an achievement mean if you don't have a neural network wired to value social connection or status. We value connection because we're social animals that have needed each other for survival.
Even the more abstract of our subjective experiences are in some way rooted in our physical evolution.
I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.
AI awareness might actually be more believable if that awareness manifested itself in an entirely different way than in humans. But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience, I think we're seeing something that isn't actually there.
There is no objective evidence of qualia. All evidence of qualia are vocal or other expressions of belief in qualia. Perceptions clearly exist and are observable, subjective experience and qualia, not so much.
> I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.
If your objection is to models based on "symbolic level of language" which you think lack semantic understanding of, say, trees, you should ask yourself how our brain, based on physics which also lacks any semantic category for trees, can somehow develop a semantic understanding of trees. All of these appeals to differences with the brain never seem to acknowledge that fundamentally, the brain has the same explanatory gap with physics.
> But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience
This assumes a lot. It seems very possible to me that intelligence inherently develops a map of natural categories (natural kinds), and language naturally develops around such categorical understanding. Semantics are then fundamentally the network of associations between categories, eg. there is no fundamental difference between symbols and semantics, and the latter cam be inferred from the former, and that's exactly what LLMs do, and why the semantic maps between different languages are so similar and how they can translate between languages.
You didn't address any of what I wrote, let alone provide any counterarguments. Which part of what I wrote do you think was wrong?
For example, the Turing machines and the lambda calculus don't look anything alike, but they are fundamentally interconvertible, and so in a real sense they are fundamentally equivalent. Without a model, all of your arguments are completely unconvincing for exactly the same reasons, eg. that there may exist many paths to fundamentally equivalent ends.
Hunger as a concept doesn't mean anything without the physical need. Politeness or bluntness, even in writing, don't mean anything without social dynamics. And we have social dynamics (and neural structures that directly process social cues and associated feelings) because we've evolved into social animals for whose survival that was important.
I see no reason to believe that a model trained only with symbolic representations, with no connection to the physical world phenomena that those symbols represent, could contain the subjective experience itself.
Neural network models may be able to derive novel (or at least novel-looking) output rather than just an obvious rehash of their input, but I don't think any set of bytes can fundamentally contain information that was never entered into it. (Even if e.g. a model produces previously unknown mathematical results, those results can in principle be derived from the information that they were trained with.)
I'm not saying that artificial neural networks couldn't, in principle, be aware. ANNs and biological neural nets may be equivalent in the sense that any information and processing structures represented by a biological one could in principle be represented by an artificial one. If that's the case, and awareness is purely a product of our neural systems as materialism would imply, it should be possible for an ANN to be aware, too.
But when the model has been trained with only language, and IMO the subjective experience can't be derived from the symbolic representation alone, I can't see how the model could include the actual subjective human experience.
An AI model could of course have an awareness and subjective experiences that are totally different than our human experience. But then the fact that it happens to produce output resembling what humans find meaningful shouldn't be considered indicative of such awareness.
This is of course more of a philosophical argument than a technical one, and I'm happy to hear counterarguments, but not on the level of off-hand dismissal.
People who say LLMs have subjective experience aren't saying they have human-type subjective experience. Nobody who sees an LLM express hunger when role playing as a hungry person thinks that the LLM is actually hungry.
I too can role play as a hungry person despite not being hungry, so there is no reason in either case to conclude that the words produced reflect genuine internal subjective states. The point is that such internal states may still exist.
Without a searchable index of training data, it is hard to put faith into these claims.
eg. "we spent another 2k on searching for pre-existing proof but found only the weaker result xyz by abc in 1972" would be in the spirit of academics quoting prior work.