From a cursory glance the bootstrap paper is really hard to read, however the blog post is easier and seems more human written.
The first result is cool (but basically a complex error-prone calculation), the second is much much cooler, Claude proved the upper limit for the mass of a black-hole which is consistent with a given central charge. (thereby making it a consistent theory of quantum gravity), any theory predicting above this is not a theory of quantum gravity of that universe. This is very exciting!
> thereby making it a consistent theory of quantum gravity
Careful. Thereby making a not-inconsistent-via-bootstrap theory. Such a theory could be inconsistent for other reasons shown by other methods. The article expresses a desire to show inconsistency for M1 above c/12 for example, but it's now been shown that the boostrap method can't get you that far.
Hardly seemed sad to me: standard physicist coming-of-age story maybe.
Most who study physics at this level, especially theoretical physics, end up moving on to something else quantitative. Most of them do quit on some level: there's usually a way to keep going, and usually life's circumstances mean it's no longer worth it.
But very few actually regret the time they spent in it, from what I've seen.
Physicist here. I don't have time right now to read the papers in depth, nor are the results directly in my area, but I would say a few things.
One is that there are a near-infinite number of "open problems" in physics because the surface area of research programs in theoretical physics is extremely large. Before I was in the world of physics, I would have thought that "solving an open problem" meant that physicists around the world would be rejoicing and discussing the result. In reality, other than for the big, field-defining questions, only a small handful of people might care about any problem being solved. That holds true even for the Millennium problems.
Even if I were to read these papers in depth and understand them, without talking to somebody that knows the area and what has been done in it already, I would have no sense of the significance. The way to assess significance is to see how the community responds. Only time will tell whether people care about these results enough to cite them, incorporate them in future work, etc.
And so, like many other people I know, I plan to ignore all of these results until somebody I know comes and tells me to check one out: the cost of assessing any of them is too high, and there are too many other papers to read.
Maths starts from axioms / existing proven work, and extends knowledge with newly proven facts.
Theoretical physics starts from (incomplete) observation and compatible theories plus targets an open question. There often are many solutions, but the real question is, which one holds up if the broader field is examined?
Consider what happened to the observation of the Higgs boson? There were numerous string theory flavors, all (more or less) consistent with current observations, but predicting different mass for the Higgs. Clear path for elimination of a lot of previously "correct" models.
In that empirical approach, making experiments and comparing to old/current/future work is a necessity to judge a models in a given regime.
In fact, it is the differentiating factor in maths and theoretical physics.
Is it? I’m pretty sure most mathematicians working on research level math are similarly tucked into specializations. perhaps the only major difference is that in physics the language and jargon differs (and this is purely anecdotally) more across subfields.
In physics if you have a important open problem, you can probably just throw numerics at it to compare theory with experiment. The mathematical insight on the other hand is there to illuminate what's going on in nature, which is a much more amorphous target than the lists of very concrete problems mathematicians have just lying around.
Many open problems in physics are open because we lack experimental data. No amount of numerics is going to help you. Finding a way to probe the right energy scales is usually the obstacle, or observing enough events of something, collecting enough data, to get a statistically significant result. Either that, or the equations are so complex that numerical calculations are completely unfeasible.
I would actually argue that many open problems in physics are open because we cannot throw numerics against them to compare against experiment, and the major difficulty lies in developing methods that sidestep severe computational difficulties. (This is the foundation of much of modern nuclear physics for example)
I would also had that in math currently you can verify proofs, like Lean, which also means the tooling itself is a good match for automation, I don't know what's the situation in theoretical physics.
It tickled me that this was a bullet point in the Fable 5.2 announcement. I like that we're now at a point where "solved two open problems in the physics of black holes" is the kind of thing that shows up in a frontier model launch.
Those theoretical physics papers look like the real thing (i.e. not mathematical physics like the OpenAI Yang-Mills result). It looks like AI is finally branching out of math.
In my (limited) experience it does not struggle with empirical research — at least in the procedural aspects thereof — as long as it has access to the right data.
It is very good at understanding research goals, identifying appropriate datasets to work with, doing data wrangling and data science, determining appropriate metrics, running simulations, analyzing results, writing up reports with charts and tables, and proposing follow ups or “high potential next steps” which show a surprising amount of judgment, and if instructed to, executing on those too.
I use Astra and Fable (who delegate to cheaper models for coding) and have them review each others work, and only spot check things when they are done calling each other out. I rarely find issues, and when I do, it’s usually a misunderstanding of intentions.
I’ve only explored a few areas though, including computer vision, vector search, and wireless signal processing, all in applications that, as far as I can tell, are genuinely novel (though of varying levels of novelty and economic value, as the models are fond of pointing out to me!)
I prefer being in the loop and setting the direction — because I do not have unlimited tokens, hah! —- and I regularly identify new directions they had not considered or had prematurely dismissed… so I am a bit more useful than just a financier of tokens! But I can totally see how they could grinding ahead by themselves and brute force useful results, especially if given very concrete goals like a theorem to prove.
I don’t think we’re at Recursive Self Improvement yet, but I don’t doubt for a minute that this is behind the insane pace of development at Anthropic and OpenAI.
In empirical research, getting the right data represents the bulk of the intellectual effort. Designing the right experiment and collecting good data is painstaking, backbreaking work.
I agree, and this may be nitpicking, but would it not be more accurate to say “bulk of the effort” rather than “intellectual effort”?
My perception is that most data collection, once designed (which does take considerable thought, as you point out), is a slog of executing rather mechanical processes over a long time. (Which is what RA’s and lab assistants are for!) For sure, regular analysis is needed to ensure the right data is being collected and no confounding factors are sneaking in, but seems to me the majority of time goes into the relatively rote work of setting up equipment, running experiments and recording the results.
I do think that with access to MCP-enabled equipment, AI would be ideal for this too, given its infinite patience. The problem, as with all other fields, is this would rob novices the opportunity of building the necessary intuition that comes from spending time in the weeds.
> bulk of the effort” rather than “intellectual effort”?
No, it is intellectual. Designing and implementing an experiment that eliminates confounding variables requires a great deal of creativity and ingenuity. Reality is messy and complicated.
Some notable examples include: Michelson-Morley, Millikan, Wrights, Cas-9 for CRISPR, CERN (one giant experiment).
I guess in my mind I'm separating out the cycle into experiment design -> experimental setup + data collection -> data analysis phases and feeling like the middle phase appears to be the longest, most painstaking and most mechanical. But as I mentioned in a sibling comment, my perceptions are based on limited second-hand experience, so yeah I can believe they're off-base.
> but seems to me the majority of time goes into the relatively rote work of setting up equipment, running experiments and recording the results.
Sadly, no. There are surely exceptions, but most data collection for experimental research is highly skilled and intellectually demanding.
The good RAs are the ones that could have aced a PhD if they'd wanted to (there's one in my lab, and they're worth their weight in gold) -- but they are rare. There's a reason that the majority of research effort in the sciences comes from PhD students and postdocs, not from RAs. This stuff is not simple, and definitely not rote.
Fair enough. My earliest exposure to scientific experimental work was working (on an unrelated engineering project) alongside some scientists and PhD students in an NMR lab. I got the impression that they spent the bulk of their time tinkering with the mechanical aspects of the NMR machines to set up experiments, then waiting hours while the experiments ran, and then staring at the results to figure out what to do next.
Then I had friends studying biotech who seemed to spend hours in labs slicing up flies or preparing cultures for experimental assays.
They were all definitely very smart and hard-working, and the science was extremely deep, but setting up and carrying out the experiments seemed to be the most time-consuming aspect, which may have skewed my perceptions.
I agree, but it is the kind of math done the physicists' way (which I found recently asserted that AI cannot do) and not the kind which is formalizable in Lean.
From skimming the papers, it’s not clear to me what it is actually doing outside of proving what can be considered mathematical theorems (even if that’s done the physicist way). It doesn’t seem to be making testable predictions (but maybe I missed those).
For people not in the know, it's common - both now and historically - for physicists to use "mathematical techniques" that have not been proven valid.
I took grad level physics courses, and this drove me bonkers. I was accustomed to the comfort mathematics provided me. If something was unclear, I could always dig deeper and find a proof. But a lot of the stuff the physics professors would do - I couldn't, and nor could they. They just responded with "intuition". Some of them had dug deep earlier in their careers and stated that the technique was likely invalid mathematically, "but still useful in getting answers experiments can verify".
I won't even talk about relying on diverging series...
Can you give examples? Sometimes a simplification is enough to illustrate a theorem, especially to an undergraduate class, without the full proof being needed. The mathematics is a tool to understand what’s happening.
More mathematical rigour would be required for formal papers, though it wouldn’t need to be derived from first principles.
Trouble is you can declare things to be true that are strictly speaking mathematically false, and from there get to proving any falsehood.
The Dirac delta function is a famous example of something mathematically non-rigorous at its inception, and mathematicians objected to it, but a few decades later Laurent Schwartz developed generalized function theory, that made it work.
Or going even further back, calculus was done in terms of ratios of infinitesimals (like dy/dx) or infinite sums of them (integrals). A century or two later the formulas of calculus were proven rigorously through classical analysis (delta-epsilon proofs) but that lost sight of the original motivations. A century or two after THAT, nonstandard analysis made a rigorous theory of infinitesimals, but reliant on relatively high powered mathematical logic that didn't exist til the 1950s or so.
Right now I think quantum field theory (renormalization, path integrals etc.) have been put on a sound mathematical basis but physicists mostly don't care. And very recently, an AI proved something about Yang-Mills theory that helped rigorize part of the Standard Model of particle physics. I don't understand that last part at all.
Besides physics, a lot of the procedures of machine learning and AI are mathematically suspicious but people use it anyway because it empirically works. People in the field have told me that yeah, they expect mathematicians will someday get around to backing it up, but for now, practitioners are moving forward with what they have.
Let's just say he didn't motivate it very well, not to speak of deriving it. And it was just the final drop in the bucket, I never understood much of it in the first place.
If you attempt to formalize math involving such assumptions, one expects to encounter a proliferation of contradictions. So I’m curious to see how the contributions of AI authors is handled by the physics community. With pure math, you can ostensibly write a compact statement of a theorem in Lean, and if so, trust the process.
Yeah the use of divergent series in modern physics is certainly something. There is an equality that includes the sum of the natural numbers in it, and that evaluates to -1/12, but it's not equal to the sum of the natural numbers on its own. I can see reasons for why it might produce sensible answers in practice, but it does make me (admittedly not a physicist) suspicious.
We used to joke in my physics undergrad that they didn’t need a math department because we were doing all the stuff they were doing but with actual applications. Then the mechanical engineers said the same thing about us!
Maybe “thrilled” except in the sense that you’re in a SciFi thriller with a giant alien spaceship hovering overhead in the sky, in much the same way bricks don’t, and you don’t know if they’re friendly or hostile or just doing a survey for their new hyperspatial express route ;-)
> I like that we're now at a point where "solved two open problems in the physics of black holes" is the kind of thing that shows up in a frontier model launch.
Because they want you to think that you will be able to do the same, too.
Except, you don't have a dedicated cluster for a week with an unrestricted model.
Yep really makes you wonder what researcher's non-public data they stole to rush publishing this like anthropic admitted to stealing with the navier-stokes solution.
Apart from Claude/Anthropic no attribution whatsoever - neither on the Hexagon entry nor in the PDF. Isn't this a novelty, even for papers that are 100% the product of AI?
Not giving paper authorship to the human(s) involved in the research seems like academic fraud to me, or were there no human involved at all beyond choosing the category of "interesting open problems in theoretical physics"?
Without the prompts, which seem like they might be needed to reproduce the results, I am not sure how we can ever know.
And then there is this cryptic bit on the company web page article:
"Weeks after one of the agents reached a solution, a physicist, who had independently used Claude to tackle the same problem, shared his result, and a colleague realized that Hongbin’s agent had reached the same answer earlier. The two solutions used different methods, so they were a useful check on each other."
I have questions:
Who was the other mystery physicist?
Does that mean there were at least 2 humans involved in the research?
Or were they not involved enough to even understand the abstract, in which case it seems like academic fraud to include any human as an author?
I would not be surprised at all if the prompt was basically "do a breakthrough". In that case, all the humans did was having access to SOTA llms... which does not appear credit-worthy to me.
The entity making the progress should be credited.
"Claude did it" doesn't cut it. What's Claude anyway? Some client app talking to Anthropic API's? That's just shuttling prompts and output back & forth.
Researchers x, y or z doesn't cover it either. In this case they'll have relied on Claude for some heavy lifting.
Suggestion: name all the parts involved. "This research was done by Alice, Bob & Carol, using Claude with compute provided by Anthropic" - or something along those lines.
Credits given, clear who did what, no implicit assumptions.
That's the most unfortunate part of these recent discoveries. Even if the human "only" proofread the entire thing, that's still worthy of credit, because without their work, none of that output would be worth writing about.
Even through after all my skepticism about AI in science, Anthropic proved they can search for and find non-trivial discoveries with their tech, they've done very little to dissuade my fear they are actively hurting the scientific community for PR.
We could treat AI as just a helpful fuzzer for open-problems that occasionally opens new interesting paths for the actual scientists, but we know the end-goal of these people is a future where there are no qualified people left to review, let alone make new discoveries independently from the machine.
My professors hate these paywalls specifically so much bypassing them in added computer lab (lol) was added to the mandated bits of the curriculum on writing and reading for academia.
The hn comment guidelines say don’t complain hn is turning into Reddit, but if dang and co. still believe that’s an “illusion”, they’re not doing much to combat it.
There are numerous classes of things, such as reimplementations of utilities that already work quite well, on some convoluted protocol that doesn't seem to actually address any of the standing issues, which get rocketed to the FP every time, it's the same people spewing koolaid in the comments every time. You can't even ask what problems it actually addresses without getting attacked. Very strange
> but if dang and co. still believe that’s an “illusion”, they’re not doing much to combat it.
They actually encourage it with their authoritarian enforcement of decorum (you're not allowed to criticize people for being wrong and etc). The result is everything turns into exultant startup-y bullshit.
The physics of black holes is far beyond the typical education or capacity of people here to understand or discuss intelligently. Outside of the occasional expert post most of the “technical” talk here is limited to development of software at varying levels of intellectual rigor.
edit: To be clear I don't mean this as an insult, it's a site with millions of people, and the overlap between "working with software" and "understanding Cauchy surfaces" is pretty damned small.
This is just untrue? At the time of your writing; my comment was the third root comment and was about the paper. (Yours is currently the first root comment and mine is 4th).
So perhaps you are seeing what you’re biased towards, or you didn’t scroll past the first root comment?
Yes, but "Good thanks, how about you?" is a typical response to the question "how are you doing?". One would not respond in that way to the question "what are you doing?". So I think it is a safe assumption that the respondent misread the original question as "how are we doing?", to which the pedantic response would be a grammatical correction as the one I provided. Although in actuality I was quoting a television program.
All the code I have seen written by academics, PhDs especially is some of the worst code I have ever seen aha. All finished the night before some deadline for submission and "works" on their machine.
when i first discovered this site i spent most of my time in the articles themselves because i had no idea how much or little to value the comments
nowadays i open the comments first and often dont open the article at all
i dont think this is a good thing, but i do find it interesting. I think a big part of it is how often i would read an article and develop a bunch of thougths about it and then come into the comments and see someone correct a fundamental premise of the article rendering all of my thoughts kind of useless as they are all based on misunderstanding and so i would start checking the comments to see if the article had been 'overruled' so to speak before i waste my time on it. this seems to have slowly shifted me to not knowing if the articles are even worth reading to begin with
I get that, but I think we can try. I know there are at least some physicists here, from other threads about AI and physics, but we could at least talk about what this article is.
For example, the metaphor they give about walking from Penn Station to the High Line comes off as somewhat out of touch.
I would make a comparison to illuminate the phenomenon I'm seeing in the rest of this thread, but unfortunately there is a specific rule artificially preventing me from doing so.
Don't worry, we all see it. Just not much point in talking about it here, I guess. I still manage to get sniped by a subthread occasionally like this one, though.
I mean FWIW, I came here to see if this actually happened, or if it's yet more marketing where a prompt engineer managed to get Claude to spit out an unreleased paper/in-progress theory/whatever from it's training data, and they're now calling it a physicist, as has happened like four times that I can recall right now.
It'd be cool if that WASN'T the case, but reality is often disappointing.
I notice that AI agents can work together in a group without problems of individual egos and author ownership getting in the way of advancing collective knowledge
Are you trying to imply that egos are one of the major problems holding back science? And not the lack of funding and evaporating public trust in science?
I think egos do exist and they probably do cause problems here and there but academia has much bigger problems that hold it back.
"Neither result describes our own universe directly, and neither has practical applications, but both contribute to a long-running theoretical effort to understand how quantum mechanics and relativity fit together."
For example the famous Schwarzschild solution (that predicts black holes) doesn't apply to our universe, because it assumes a black hole that's been there forever, not one that formed. But it was still very useful for our understanding.
It also assumes zero angular momentum, which we know is essentially impossible in our universe. And rotating black holes (the Kerr metric) are much stranger beasts.
Schwarzschild allows you to still calculate a whole bunch of things outside the event horizon. The part where it changes from our universe is the physics inside the black hole and their formation
"Claude solves" feels reflective of the constant, problematic anthropomorphizing of the machine. For how it's problematic, look at "it broke out of the sandbox" vs. "we failed to build an adequate sandbox".
What does it mean for Claude to solve it? What was the prompt? I didn't read it all, but the word "prompt" does not appear in the blog post, either abstract, or either paper.
It means someone set up the question with the Claude tool and let it iterate until it came up with an acceptable solution.
When my accountant whips out a calculator, it is the accountant that defined the problem and set the solution in motion. Fantastic tools are still tools. Claude does not love me nor you no matter how its trained and encoded patterns express.
Right, but in that situation, the email from the accountant isn't titled "TI-34 finished your taxes". The accountant did all the interesting work. (This metaphor is getting stretched, since accountants usually use software to do the taxes, and that software does a lot more of the interesting work than a pocket calculator does. But even then, the accountant has done the most important work in telling the software about your tax situation and the summary is still "the accountant has finished your taxes".)
That's definitely not how it happened anyway. It wasn't until "it" came up with an acceptable solution but rather when whoever prompted it do so felt it was good enough according to whatever criteria they thought was sufficient. Knowing when to stop is in itself extremely challenging so unless there is an objective way to verify, which I doubt it's the case when it's the state of the art, surely someone went through the output and stopped the process, namely no more iteration (which again does show involvement from qualified someone).
"Spinning Jenny-spun yarn" feels reflective of the constant, problematic anthropomorphizing of the machine. For how it's problematic, look at "the boiled exploded" vs. we failed to build an adequate boiler".
What does it mean for Jenny to spin it? Who provided the power to it? I didn't read it all, but the word "power" does not appear in the patent, or the newspaper articles about it.
I'm confused by this. Are you comparing an LLM to the spinning wheel? In that case wouldn't the analogous headline be "Spinning wheel spins skein" or something? And if there were an article about it, I would definitely expect it to explain that Jenny operated the wheel and some basics about the steps involved. I don't know enough about spinning to know if it's purely mechanical or if operator skill is involved. If spinning yarn is mechanical, then it's not analogous because it's not mechanical to get Claude to prove a novel result in physics. If spinning yarn with a wheel involves a lot of operator skill, then I wouldn't expect us to say "the wheel spun the yarn", but "Jenny spun the yarn [aided by her spinning wheel]".
To pick yet another example: you _can_ say "the car drove from California to Florida", but that'd an unusual stylistic choice and so would probably imply that you didn't know for sure who drove it or something. In ordinary conversation people would be much more likely to say "Jenny drove from California to Florida [in her car]".
Is the boiled/boiler thing related or just some other example? That isn't analogous either. The analog would be "the pressurized gases escaped the vessel" vs. "the vessel failed to contain the pressurized gases". These are different. If you want to emphasize how out-of-spec the internal conditions were, you say "the pressured gases escaped the vessel". If you want to emphasize that the chosen vessel was obviously wrong for the job, you say "the vessel failed to contain the pressurized gases". Either way though there's no risk of the reader anthropomorphizing the pressurized gases. That's different from "the agents escaped the sandbox" because that implies ... agency.
Agree to disagree, I don't think it makes sense to say any of the options you listed solved it. I agree the various aspects you mentioned were prerequisites that enabled Claude to solve it, but none of those particular inputs solved it in isolation.
it's also meaningful though, in that it's a tool trained on enormous amounts of stolen human thought. so recognizing that it's the stolen thing that did the thing, and not the company itself, gives people back some of the credit
i cant say i fully understand this, i do quantum chemistry work so i only kinda understand portions of the first paper on black hole phase transitions. it is interesting that they're doing things kinda similar in black hole theory! i had no idea! the idea of black holes having phase transitions is one that my thermodynamics brain enjoys, and its nice to see perturbation theory in the wild. i am unfamiliar with much of the vocabulary used + would need days to build proper intuition on the problem setup. i do wish i could form a proper critique, the paper is at a weird space for me because some of the terminology is familiar, techniques used are similar, and i feel like if i really locked in i could develop an understanding of the paper in a few months. makes me want to go back to grad school.
i feel solipsistic saying this, but the more i understand about physics, the less awed i am by papers? obviously i am impressed, i know firsthand the amount of thought and calculations that goes into these sorts of results. but i dunno im just not awed. awe implies an incomprehensible greater than thou incredulity, and some of the physics in this paper feels really normal and achievable to me. i think im just too young to properly understand the effort this takes, maybe i should be awed, but im not.
i think i am trying to get at something where, it feels when ai labs put out big press releases on topics, they are making a big show of something only truly interesting to a small portion of people. if a human came up with these same results independently it would yield conference talks and grants, but ai is getting more eyes and a bigger press release just because its ai that solved it and not a human. which is impressive from an ai development standpoint! but less so from a physics standpoint (insert deintensifiers of this statement here im just stupid), so the physics feels like a prop.
still really interesting results though, again, i wish i understood enough of the underlying theories theyre connecting to feel really moved by the results. sorry for being so self centered in my analysis lol, wish i could do better but i cant.
The notion that an LLM, a machine with no connection to the physical world whatsoever, could "solve" physics problems is absolutely ludicrous. The level of quackery represented here is nauseating. If this is what "science" looks like under the aegis of these fraudster companies, we're done for.
I think everyone asking for this kind of thing doesn't really understand the scale and complexity of solving a big problem with many agents.
For any of the big things I've worked on, it isn't one "prompt", it's a goal that keeps getting updated and a bunch of back-and-forth steering of the model and sub agents, where what the human is saying to guide the model is heavily context dependent on what the model is currently doing or saying. Much of that guidance will ultimately be useless guidance on dead-end explorations. Also, the model will likely have written huge programs to solve things symbolically or simulate things numerically, and both the programs that were written, and the output of those programs are critical to understanding how the model came up with the things that led to a solution.
Now, I'd be all for publishing the full transcript, but the transcript is likely to be massive. Gigabytes of text, and reproducing the local computation (often sympy or numpy) may be hundreds or thousands of CPU-hours.
It is common in physics and astronomy to share your code that takes millions of cpu hours to solve problems, and to share your resulting data that can be in the petabytes.
Gigabytes of text seems perfectly shareable nowadays. Just as with most computational physics stuff of that scale, sharing code that can reproduce the result and roughly describing the computational requirements would be fine.
Oh it’s totally shareable. And I’d love to see it all shared.
But it isn’t “just a prompt”, and sharing it doesn’t make the results reproducible (since you don’t get control over the model’s “seed” from run to run).
LLMs are not deterministic, which to my dumb mind, seems to lessen the value of those inputs since they will not inherently get you down the same path, no?
I imagine the value here would be developing general strategies on how to use the tool to accomplish a task rather than reproducibility. That was part of the rationale for doing so in math at least. You could probably make an LLM deterministic if need be but with significant performance penalty.
I wonder how this works out in other parts of physics codes. If you need a Monte Carlo integrator, is it common to guarantee a seed for reproducibility? How do these codes handle distributed systems for rng? Is it common to use a fully reproducible distributed rng and GPU kernel for scientific compute?
If the human is involved, "where what the human is saying to guide the model", then the prompts are even more important to include. Is this a case of "solve an open problem in theoretical physics, make no mistakes", or a case where a human had to interact frequently with the model?
Think any novel idea coming out of Claude publically shared should legally require prompts to show up as part of the disclosure. That's more or less how science works with the papers already.
Over a decade ago I saw a talk about a cohesive and complete explanation for gravity. It didn't involve quantum mechanics nor a particle called a "graviton".
That talk did make enough sense.
I'm still waiting to see a "picture" of a black hole that didn't involve a "pre-trained model" developed with "synthetic data" or something like that.
I'm not feeling sufficiently persuaded that I can feel confident that gravitons are a thing.
Nonetheless, I am remaining open minded to this black hole stuff.
Funny because it can’t solve complex coding tasks but can solve unsolved problems in physics. Hopefully, not by stealing ideas from physicists working on those problems in Claude.
AI is good at pattern recognition but some problems require a different kind of inference
Also: The main result was first obtained autonomously by an internal Claude research model. The publicly available Claude Opus 5.5 later obtained the same result independently. Claude then wrote this paper, following brief guidance from a physicist, who also checked every main result.
Yeah yeah, we'll believe it once someone else actually reports on it. "Windows the best OS, says microsoft", "Big Macs the healthiest food, says McDonalds". "Our AI is solving open problems in physics, exclaims AI manufacturer".
Climate change too. In particular game theoretic solutions. I.e. technical solutions that can withstand global tensions and commercial and government constraints.
People are buying solar systems worldwide without a king. AI could look into lots of things related to renewables, storage and fuel generatiom and carbon capture.
Please stop applying classical ideas to quantum mechanics. The particle like you are imagining doesn't exist like that. We just have its description by a function (wavefunction). The notion of particle you have is just an emergent property in the classical limit.
What are the odds it stole from some random researcher that used the chatbot to do exploratory research? People talk to chatbots like they were trusted peers. Who can be sure one can't be scooped because the model connects the dots between your thoughts and the related unpublished thoughts of hundreds of other researchers also relying on it as trusted peer, with the model owner publishing the combined results behind the backs of everyone.
> What are the odds it stole from some random researcher that used the chatbot to do exploratory research? People talk to chatbots like they were trusted peers.
They do but we've seen enough solutions by now that surely one of these researchers would come forward to point it out?
The "stolen" work and strategy of the solution itself was there from 2023 and even buckmaster used ai who "stole" the idea from Córdoba–Martínez-Zoroa programme. There's no proof that the solution even used much of the solution of the buckmaster. You people blindly repeat everything like a sheep without critical thinking. AI + humans solving it within days compared to decades of humans only solving is what should be commended. Instead we got people like you who will take humanity to stone ages due to anti capitalist ideology
I hope you one day come to appreciate the value of human effort and how it has allowed these models to exist and continue to exist. And that people should be looked after by the community for which they contribute with their effort rather than dismissed and discarded. It's a quality of life thing. For everyone.
I do value humans like crazy so I'm ok with ubi or some sort. Just because some rust belt workers got shelved due to technological and economic progress, I ain't gonna stop using foreign manufactured products. I sympathize and help them to upskill but not take whole future hostage because of unemployment or creative destruction. I'll probably be next to get booted and be unemployed forever but I'll try to reskill instead of demanding the whole world to stop embracing ai enhanced progress. Even AI can't solve it alone without the years of human knowledge and human prompter enhancing its search space for better output. It's exciting what all humans can do with it. And I'm just getting it for just 20$? The deal is too insane to not be awestruck.
You may be over-simplifying the opposing argument if you think it's just about demanding that the world halt progress. There's a middle ground between being amish and being relentlessly accelerationist. Pacing is important for reacting to side effects as you go. Otherwise, they stack up. We have a backlog of cultural issues that won't be fixed with more tech.
The other person is the one who over simplified by calling it stealing, even though it's evident that ai used many new ideas to reach its solution. Look at mathematicians freaking out about how good it is. We can look at middle ground only if the opposing side provides honest criticism instead of calling it stealing etc when original authors themselves didn't accuse openai of stealing. The actual issues are just cyber security, alignment, humans not coming to terms with the intelligence of ai and upskilling of new engineers to have minimum required knowledge and something along those lines. The solutions are not stopping construction of data centers/nuclear or banning ai. They propose these shit solutions because it threatens their ideology/property values and they are still in denial about the astonishing human-like intelligence displayed by AI.
Ask them about capitalism and its achievements and you will get your answer. This reddit/twitter tier analysis is pretty indicative of their motives. They'll be hyper critical free market fundamentalists when talking about anthropic pushing for regulation or any corporations but bootlick the state when it strangulates housing/healthcare industries with shitty regulations where more freer market can solve such issues without too much ugly side effects.
A lot of us are willing and today have already taken the bet that a model or system that can solve hard problems across math, physics, biology is genuine intelligence. It will command a great premium or it may even figure out in the not so distant future how to make money on its own.
> a model or system [...] can solve hard problems across math, physics, biology
...is all that has been proven. That may one thing that intelligent humans can do, but it doesn't mean it requires human level intelligence. All computers are superhuman in some narrow sense.
The first result is cool (but basically a complex error-prone calculation), the second is much much cooler, Claude proved the upper limit for the mass of a black-hole which is consistent with a given central charge. (thereby making it a consistent theory of quantum gravity), any theory predicting above this is not a theory of quantum gravity of that universe. This is very exciting!
Careful. Thereby making a not-inconsistent-via-bootstrap theory. Such a theory could be inconsistent for other reasons shown by other methods. The article expresses a desire to show inconsistency for M1 above c/12 for example, but it's now been shown that the boostrap method can't get you that far.
https://nautil.us/what-does-any-of-this-have-to-do-with-phys...
Most who study physics at this level, especially theoretical physics, end up moving on to something else quantitative. Most of them do quit on some level: there's usually a way to keep going, and usually life's circumstances mean it's no longer worth it.
But very few actually regret the time they spent in it, from what I've seen.
One is that there are a near-infinite number of "open problems" in physics because the surface area of research programs in theoretical physics is extremely large. Before I was in the world of physics, I would have thought that "solving an open problem" meant that physicists around the world would be rejoicing and discussing the result. In reality, other than for the big, field-defining questions, only a small handful of people might care about any problem being solved. That holds true even for the Millennium problems.
Even if I were to read these papers in depth and understand them, without talking to somebody that knows the area and what has been done in it already, I would have no sense of the significance. The way to assess significance is to see how the community responds. Only time will tell whether people care about these results enough to cite them, incorporate them in future work, etc.
And so, like many other people I know, I plan to ignore all of these results until somebody I know comes and tells me to check one out: the cost of assessing any of them is too high, and there are too many other papers to read.
Theoretical physics starts from (incomplete) observation and compatible theories plus targets an open question. There often are many solutions, but the real question is, which one holds up if the broader field is examined?
Consider what happened to the observation of the Higgs boson? There were numerous string theory flavors, all (more or less) consistent with current observations, but predicting different mass for the Higgs. Clear path for elimination of a lot of previously "correct" models.
In that empirical approach, making experiments and comparing to old/current/future work is a necessity to judge a models in a given regime. In fact, it is the differentiating factor in maths and theoretical physics.
I would also had that in math currently you can verify proofs, like Lean, which also means the tooling itself is a good match for automation, I don't know what's the situation in theoretical physics.
Presumably Anthropic wouldn't have chosen to work on these if the problems weren't significant?
This seems carefully timed with their IPO.
Those theoretical physics papers look like the real thing (i.e. not mathematical physics like the OpenAI Yang-Mills result). It looks like AI is finally branching out of math.
https://news.ycombinator.com/item?id=50430239
Is it really, though? It's still a heavily mathematical domain, rather than one involving empirical research, where AI still struggles.
It is very good at understanding research goals, identifying appropriate datasets to work with, doing data wrangling and data science, determining appropriate metrics, running simulations, analyzing results, writing up reports with charts and tables, and proposing follow ups or “high potential next steps” which show a surprising amount of judgment, and if instructed to, executing on those too.
I use Astra and Fable (who delegate to cheaper models for coding) and have them review each others work, and only spot check things when they are done calling each other out. I rarely find issues, and when I do, it’s usually a misunderstanding of intentions.
I’ve only explored a few areas though, including computer vision, vector search, and wireless signal processing, all in applications that, as far as I can tell, are genuinely novel (though of varying levels of novelty and economic value, as the models are fond of pointing out to me!)
I prefer being in the loop and setting the direction — because I do not have unlimited tokens, hah! —- and I regularly identify new directions they had not considered or had prematurely dismissed… so I am a bit more useful than just a financier of tokens! But I can totally see how they could grinding ahead by themselves and brute force useful results, especially if given very concrete goals like a theorem to prove.
I don’t think we’re at Recursive Self Improvement yet, but I don’t doubt for a minute that this is behind the insane pace of development at Anthropic and OpenAI.
My perception is that most data collection, once designed (which does take considerable thought, as you point out), is a slog of executing rather mechanical processes over a long time. (Which is what RA’s and lab assistants are for!) For sure, regular analysis is needed to ensure the right data is being collected and no confounding factors are sneaking in, but seems to me the majority of time goes into the relatively rote work of setting up equipment, running experiments and recording the results.
I do think that with access to MCP-enabled equipment, AI would be ideal for this too, given its infinite patience. The problem, as with all other fields, is this would rob novices the opportunity of building the necessary intuition that comes from spending time in the weeds.
No, it is intellectual. Designing and implementing an experiment that eliminates confounding variables requires a great deal of creativity and ingenuity. Reality is messy and complicated.
Some notable examples include: Michelson-Morley, Millikan, Wrights, Cas-9 for CRISPR, CERN (one giant experiment).
Sadly, no. There are surely exceptions, but most data collection for experimental research is highly skilled and intellectually demanding.
The good RAs are the ones that could have aced a PhD if they'd wanted to (there's one in my lab, and they're worth their weight in gold) -- but they are rare. There's a reason that the majority of research effort in the sciences comes from PhD students and postdocs, not from RAs. This stuff is not simple, and definitely not rote.
Then I had friends studying biotech who seemed to spend hours in labs slicing up flies or preparing cultures for experimental assays.
They were all definitely very smart and hard-working, and the science was extremely deep, but setting up and carrying out the experiments seemed to be the most time-consuming aspect, which may have skewed my perceptions.
https://desystemize.substack.com/p/desystemize-1
Context is king.
"It's not much, but it's honest work."
For people not in the know, it's common - both now and historically - for physicists to use "mathematical techniques" that have not been proven valid.
I took grad level physics courses, and this drove me bonkers. I was accustomed to the comfort mathematics provided me. If something was unclear, I could always dig deeper and find a proof. But a lot of the stuff the physics professors would do - I couldn't, and nor could they. They just responded with "intuition". Some of them had dug deep earlier in their careers and stated that the technique was likely invalid mathematically, "but still useful in getting answers experiments can verify".
I won't even talk about relying on diverging series...
More mathematical rigour would be required for formal papers, though it wouldn’t need to be derived from first principles.
The Dirac delta function is a famous example of something mathematically non-rigorous at its inception, and mathematicians objected to it, but a few decades later Laurent Schwartz developed generalized function theory, that made it work.
Or going even further back, calculus was done in terms of ratios of infinitesimals (like dy/dx) or infinite sums of them (integrals). A century or two later the formulas of calculus were proven rigorously through classical analysis (delta-epsilon proofs) but that lost sight of the original motivations. A century or two after THAT, nonstandard analysis made a rigorous theory of infinitesimals, but reliant on relatively high powered mathematical logic that didn't exist til the 1950s or so.
Right now I think quantum field theory (renormalization, path integrals etc.) have been put on a sound mathematical basis but physicists mostly don't care. And very recently, an AI proved something about Yang-Mills theory that helped rigorize part of the Standard Model of particle physics. I don't understand that last part at all.
Besides physics, a lot of the procedures of machine learning and AI are mathematically suspicious but people use it anyway because it empirically works. People in the field have told me that yeah, they expect mathematicians will someday get around to backing it up, but for now, practitioners are moving forward with what they have.
If you attempt to formalize math involving such assumptions, one expects to encounter a proliferation of contradictions. So I’m curious to see how the contributions of AI authors is handled by the physics community. With pure math, you can ostensibly write a compact statement of a theorem in Lean, and if so, trust the process.
I just love to see this, particularly the response when AI then goes on to do it.
> https://news.ycombinator.com/item?id=50563604
Because they want you to think that you will be able to do the same, too.
Except, you don't have a dedicated cluster for a week with an unrestricted model.
Without the prompts, which seem like they might be needed to reproduce the results, I am not sure how we can ever know.
And then there is this cryptic bit on the company web page article: "Weeks after one of the agents reached a solution, a physicist, who had independently used Claude to tackle the same problem, shared his result, and a colleague realized that Hongbin’s agent had reached the same answer earlier. The two solutions used different methods, so they were a useful check on each other."
I have questions: Who was the other mystery physicist?
Does that mean there were at least 2 humans involved in the research?
Or were they not involved enough to even understand the abstract, in which case it seems like academic fraud to include any human as an author?
"Claude did it" doesn't cut it. What's Claude anyway? Some client app talking to Anthropic API's? That's just shuttling prompts and output back & forth.
Researchers x, y or z doesn't cover it either. In this case they'll have relied on Claude for some heavy lifting.
Suggestion: name all the parts involved. "This research was done by Alice, Bob & Carol, using Claude with compute provided by Anthropic" - or something along those lines.
Credits given, clear who did what, no implicit assumptions.
Is that marketing spin, lack of confidence, fear of blowback? IDK, but interesting to think about an academic world where that becomes the case
Even through after all my skepticism about AI in science, Anthropic proved they can search for and find non-trivial discoveries with their tech, they've done very little to dissuade my fear they are actively hurting the scientific community for PR.
We could treat AI as just a helpful fuzzer for open-problems that occasionally opens new interesting paths for the actual scientists, but we know the end-goal of these people is a future where there are no qualified people left to review, let alone make new discoveries independently from the machine.
:)
https://news.ycombinator.com/item?id=37123792
The hn comment guidelines say don’t complain hn is turning into Reddit, but if dang and co. still believe that’s an “illusion”, they’re not doing much to combat it.
There are numerous classes of things, such as reimplementations of utilities that already work quite well, on some convoluted protocol that doesn't seem to actually address any of the standing issues, which get rocketed to the FP every time, it's the same people spewing koolaid in the comments every time. You can't even ask what problems it actually addresses without getting attacked. Very strange
They actually encourage it with their authoritarian enforcement of decorum (you're not allowed to criticize people for being wrong and etc). The result is everything turns into exultant startup-y bullshit.
edit: To be clear I don't mean this as an insult, it's a site with millions of people, and the overlap between "working with software" and "understanding Cauchy surfaces" is pretty damned small.
And that’s being generous.
So perhaps you are seeing what you’re biased towards, or you didn’t scroll past the first root comment?
love the Aussies
The German software giant SAP was jokingly dubbed "Sammelstelle arbeitsloser Physiker": agency for unemployed physicists.
The article is still over my head though.
I'm not sure if that's sampling bias, but it's not true of the Dutch or Belgian or French people I have met.
As far as I can tell, the entire German economy is an agency for unemployed physicists.
Never heard that, but it explains some UI and UX choices.
nowadays i open the comments first and often dont open the article at all
i dont think this is a good thing, but i do find it interesting. I think a big part of it is how often i would read an article and develop a bunch of thougths about it and then come into the comments and see someone correct a fundamental premise of the article rendering all of my thoughts kind of useless as they are all based on misunderstanding and so i would start checking the comments to see if the article had been 'overruled' so to speak before i waste my time on it. this seems to have slowly shifted me to not knowing if the articles are even worth reading to begin with
For example, the metaphor they give about walking from Penn Station to the High Line comes off as somewhat out of touch.
This is a ridiculous comparison, because that wouldn't be talking about the paper either.
I would make a comparison to illuminate the phenomenon I'm seeing in the rest of this thread, but unfortunately there is a specific rule artificially preventing me from doing so.
It'd be cool if that WASN'T the case, but reality is often disappointing.
I think egos do exist and they probably do cause problems here and there but academia has much bigger problems that hold it back.
"Neither result describes our own universe directly, and neither has practical applications, but both contribute to a long-running theoretical effort to understand how quantum mechanics and relativity fit together."
What does it mean for Claude to solve it? What was the prompt? I didn't read it all, but the word "prompt" does not appear in the blog post, either abstract, or either paper.
When my accountant whips out a calculator, it is the accountant that defined the problem and set the solution in motion. Fantastic tools are still tools. Claude does not love me nor you no matter how its trained and encoded patterns express.
And you know the details how?
What does it mean for Jenny to spin it? Who provided the power to it? I didn't read it all, but the word "power" does not appear in the patent, or the newspaper articles about it.
To pick yet another example: you _can_ say "the car drove from California to Florida", but that'd an unusual stylistic choice and so would probably imply that you didn't know for sure who drove it or something. In ordinary conversation people would be much more likely to say "Jenny drove from California to Florida [in her car]".
Is the boiled/boiler thing related or just some other example? That isn't analogous either. The analog would be "the pressurized gases escaped the vessel" vs. "the vessel failed to contain the pressurized gases". These are different. If you want to emphasize how out-of-spec the internal conditions were, you say "the pressured gases escaped the vessel". If you want to emphasize that the chosen vessel was obviously wrong for the job, you say "the vessel failed to contain the pressurized gases". Either way though there's no risk of the reader anthropomorphizing the pressurized gases. That's different from "the agents escaped the sandbox" because that implies ... agency.
Sorry if I've misunderstood!
By the persons who trained the model.
By the persons who provided the data the model was trained on.
etc.
> What was the prompt?
Find open physics problems and solve them, basically.
This is a rather creative problem so I think it's a very good AI benchmark.
i feel solipsistic saying this, but the more i understand about physics, the less awed i am by papers? obviously i am impressed, i know firsthand the amount of thought and calculations that goes into these sorts of results. but i dunno im just not awed. awe implies an incomprehensible greater than thou incredulity, and some of the physics in this paper feels really normal and achievable to me. i think im just too young to properly understand the effort this takes, maybe i should be awed, but im not.
i think i am trying to get at something where, it feels when ai labs put out big press releases on topics, they are making a big show of something only truly interesting to a small portion of people. if a human came up with these same results independently it would yield conference talks and grants, but ai is getting more eyes and a bigger press release just because its ai that solved it and not a human. which is impressive from an ai development standpoint! but less so from a physics standpoint (insert deintensifiers of this statement here im just stupid), so the physics feels like a prop.
still really interesting results though, again, i wish i understood enough of the underlying theories theyre connecting to feel really moved by the results. sorry for being so self centered in my analysis lol, wish i could do better but i cant.
For any of the big things I've worked on, it isn't one "prompt", it's a goal that keeps getting updated and a bunch of back-and-forth steering of the model and sub agents, where what the human is saying to guide the model is heavily context dependent on what the model is currently doing or saying. Much of that guidance will ultimately be useless guidance on dead-end explorations. Also, the model will likely have written huge programs to solve things symbolically or simulate things numerically, and both the programs that were written, and the output of those programs are critical to understanding how the model came up with the things that led to a solution.
Now, I'd be all for publishing the full transcript, but the transcript is likely to be massive. Gigabytes of text, and reproducing the local computation (often sympy or numpy) may be hundreds or thousands of CPU-hours.
But it isn’t “just a prompt”, and sharing it doesn’t make the results reproducible (since you don’t get control over the model’s “seed” from run to run).
I wonder how this works out in other parts of physics codes. If you need a Monte Carlo integrator, is it common to guarantee a seed for reproducibility? How do these codes handle distributed systems for rng? Is it common to use a fully reproducible distributed rng and GPU kernel for scientific compute?
Also nice timing around same time as IPO
I think reality is these technologies just do not make money compared to the money they cost, at least not currently.
So they're just trying to shyster their way in as much as possible.
Really cringeworthy to see. I hope it all crumbles instead. They stole everyone's work.
Go fuck yourselves.
That talk did make enough sense.
I'm still waiting to see a "picture" of a black hole that didn't involve a "pre-trained model" developed with "synthetic data" or something like that.
I'm not feeling sufficiently persuaded that I can feel confident that gravitons are a thing.
Nonetheless, I am remaining open minded to this black hole stuff.
AI is good at pattern recognition but some problems require a different kind of inference
Representatives of AI labs and other organizations may email admin@hexagonmath.org to arrange a verified lab account.
https://hexagonmath.org/organizations/anthropic
Also: The main result was first obtained autonomously by an internal Claude research model. The publicly available Claude Opus 5.5 later obtained the same result independently. Claude then wrote this paper, following brief guidance from a physicist, who also checked every main result.
Wild.
https://hexagonmath.org/who
It was a comforting moment, for sure. But ephemeral, self-imposed ignorance is no bliss at all :(
Stop boosting products made by a scam company.
Next up, the Origins of the Universe... stay tuned for our Fable e^2. More on the 10 o'clock news...
/s
Answering myself: yes, one of the problems was solved using Lean. The other was undisclosed. Nothing new under the sun, huh.
Whilst there is no "world king" to make this decision, every nation will burn all their oil and coal, the world will warm, and people will adapt.
That has always been the future, and climate activists have mostly been wasting their time
https://elements.visualcapitalist.com/what-powered-the-world...
Efficient fuel creation from renewables could massively change this.
Stopped reading right here. It took one. Maybe you don't know which one it took but things can't be in two places at once.
If you don't, it could have taken either route.
If you observed route A you either saw it, or saw nothing meaning it took route B.
Of course it would have taken route A until you decided to observe that route. Then it took route B.
They do but we've seen enough solutions by now that surely one of these researchers would come forward to point it out?
OK, but it's still an unfounded claim and...
> a model or system [...] can solve hard problems across math, physics, biology
...is all that has been proven. That may one thing that intelligent humans can do, but it doesn't mean it requires human level intelligence. All computers are superhuman in some narrow sense.
Any minute now