I have not been following AI news all that closely, but it kind of seems like the hype is dying down a little? Like, three months ago everyone was acting like AI was going to take over the world soon, and now the attitude seems to be it's a kind of cool autocomplete.
For whatever it's worth, when I've used ChatGPT and Bing's AI chat it falls into the category of "kind of cool, but definitely not going to take anyone's job anytime soon."
If this is a representative summary of Asterisk's articles, it seems... very one-note. Are there any online rationalist/effective altruist hangout spots that aren't consumed with AI-related discussion?
"We should, as a civilization, operate under the assumption that transformative AI will arrive 10-40 years from now, with a wide range for error in either direction."
I feel like it's hard not to read this closing statement as a slightly more nuanced and somewhat less efficient way to say "we don't know." I don't mean that flippantly; I think there's a lot of value to thinking through what we do know and where our real confidence level should be. However, what really is the practical difference between 10-40 years with significant error bars and we just don't really know?
> I don’t think they emphasized enough the claim that the natural trajectory of growth is a hyperbola reaching infinity in the 2020s, we’ve only deviated from that natural curve since ~1960 or so, and that we’re just debating whether AI will restore the natural curve rather than whether it will do some bizarre unprecedented thing that we should have a high prior against.
Yeah, that's not how fitting curves to data works.
> the natural trajectory of growth is a hyperbola reaching infinity in the 2030s, we’ve only deviated from that natural curve since ~1960 or so
This is a very oddly stated claim. Most apparent exponential curves are actually S-curves that haven't yet hit the "S." My null hypothesis is that this is another example of such, not that there's some "natural" trajectory from which we've deviated. Why would one believe the latter?
Assuming the Angry Birds challenge is serious, what is it about that game that makes it so much more difficult for AIs to master? Surely if AIs can beat humans at Go and chess, not to mention master dozens of classic Atari games in a matter of hours, Angry Birds shouldn't be that much more difficult??
I asked ChatGPT how to avoid the problem of researchers always being depicted as chemical engineers wearing labcoats, then punched a summary into Dall-E. I think this is probably better representative: https://labs.openai.com/s/KJoq7h4Yfn5ukOnGmCQhQ3v8
China's behind on some things on AI and ahead on others. These advantages and disadvantages are actually probably fairly structural to the point where I'd feel semi-comfortable making predictions about who will lead where (or at least what will be close and where one side will be far ahead). For example, LLMs are handicapped by conditions in China and will be probably indefinitely. On the other hand, facial recognition or weapons guidance have structural advantages in the huge security state serves as an eager market and data provider. And China tends to play closer to the chest with everything but especially military technology.
I'm also highly suspicious of anyone writing about Chinese AI who doesn't have a tech background or who doesn't have access to things like AliCloud. Chinese reporters and lawyers and propagandists are usually about as technically literate as their American counterparts. Which is to say not very. You really need to scrub in to understand it.
Not to mention a lot of the bleeding edge is in dense papers in Chinese. Plus the Chinese produce a gigantic quantity of AI papers which means simply sorting through everything being done is extremely difficult. You need to BOTH understand Chinese and computer science language to even have a hope of understanding them. But Ding seems focused on the consumer and legal side of it.
Also fwiw: I tend to think AI will lead to lots of growth but not some post-scarcity utopia type scenario. It will speed growth, be deflationary as per usual, etc. And I think that we're actually going to see a move away from gigantic all encompassing models towards more minutely trained ones. For example, an AI model based on the entire internet might be less useful than an AI model trained on transcripts of all of your customer service calls and your employee handbook. Even if you wanted to make a "human-like" AI ingesting the entire internet is probably not the way to do it. Though the question of whether we can keep getting smaller/better chips (and on to quantum) is certainly open and important to computing overall.
And the idea of using chips unilaterally as a stranglehold is a bad idea. We only somewhat succeeded with Russia and only then because the other important players went along with it. Any number of nations, even individually, could have handicapped the effort. They just chose not too. And keep in mind China is in the "chose not to" camp. This is without getting into rogue actors.
All in all I enjoyed the magazine although I did not agree with much of it.
People would normally cite the AI survey as "Grace et al" because it was a collaboration with Katja Grace as the first author. (I was one of the authors).
In the Asterisk article you say, "As for Katja, her one-person AI forecasting project grew into an eight-person team, with its monthly dinners becoming a nexus of the Bay Area AI scene." But the second author on the 2016 paper (John Salvatier) was a researcher at AI impacts, and the paper had five authors in total.
The article about people underestimating the arrival of the Soviet atomic bomb is almost completely leaving out the fact that the Manhattan project was porous as swiss cheese to Soviet spies. Stalin had full blueprints for the bomb before Truman even knew what an atomic bomb was. The Germans failed after going down a research wrong alley (whether Heisenberg did it on purpose or not is perhaps an open question, but the fact remains that they didn't focus on the right path). It is a LOT easier to build an atomic bomb when you have the full manual, plus a list of dead ends to avoid, courtesy of Klaus Fuchs, Theodore Hall, David Greenglass, etc. Unless this article is suggesting that China has spies in OpenAI, I think the analogy needs more work. (And if he is claiming that, that's a major claim and he should be more specific about why he's claiming it.)
Also, can someone please explain to me the ending of the chatbot story? I liked it up until the end and then I was just confused. Ads on fAIth???? What's that mean?
Why they didn't compare it to GPT-4 on the MMLU (a famous benchmark that people actually care about)? A mystery of the Orient. It seems GPT-4 performs better on the hard version of C-Eval. It's only on the easier one that ChatGLM scores higher. So take from that what you will.
I like how the guy starts out by assuring people the result's not fake. Kind of says it all.
A lot of hints and suggestions by the prompter was needed, as well as relying on a fake (simplified) website, and also needed a human to engage with that website.
About the two surveys, I wonder whether the explosion of the field plays a role. "Expert" was defined as someone who has published at NeurIPS or ICML, probably in that year.
But for the bigger one, NeurIPS, in 2015 there were ~400 papers, in 2016 it was ~600, in 2021 it was ~2400 and in 2022 there were ~2700 papers.
I am not sure that this implies less expertise for each expert. It could be, because in 2016 there was a much larger fraction of experts with years or decades of experience in AI research, while the typical expert nowadays has 2-3 years of experience. But perhaps long-term experience on AI is useless, so who knows?
But it definitely implies that experts can no longer have an overview of what's going on in the conference. In 2015/16 experts could still know the essential developments in all of NeurIPS. Nowadays they are way more specialized and only know the stuff in their specific area of expertise within NeurIPS. So perhaps it's fair that 2022 experts don't know the state-of-the-art in all of AI, while 2016 experts still knew that?
Here is a chart for how NeurIPS has grown over the years:
I'm a bit angry about the Starcraft thing. Scott says that AI has never beat top humans players. But the reason it did not is that it was not allowed to.
The AI that played starcraft was seriously hobbled : it wasn't allowed to view the whole terrain at once by moving it's camera very fast. It wasn't allowed to micromanage each and every unit at once by selecting it individually. In general it's APM (action per minute) rate was limited to a rate comparable to a human.
That's like inviting a computer to a long-division competition, but only giving it a 100 Hz CPU.
The whole point of having AI instead of meat brains is that AI works on very fast CPUs with very large RAM. If you're gonna forgo that, you won't reap the benefits of having a silicon brain.
That article about AI taking tech jobs is really frustrating. He predicts by 2025, 55% of programmers will be using LLMs "for the majority of their professional programming work". He bases this on "the 1.2 million people who signed up for Github Copilot during its technical preview" (divided by a bit to restrict to U.S. programmers), but utterly elides the distinction between "signed up to try the new thing" and "have adopted the technology as a key part of programming from now on".
What percentage of those 1.2 million even got around to trying it at all after signing up? (I'd guess less than 50%.) What percentage of the remainder got frustrated and didn't put in the effort to learn how to work it? (I'd guess more than 50%.) What percentage of the remainder were able to find places to use it in a significant proportion of their work? (I'd guess less than 50%.) There's a huge, huge gulf between "is a tool someone's tried once" and "is an invaluable part of _the majority of_ someone's work".
In your article, you argue that experts are well-callibrated because they had as many underestimates as overestimates. But this doesn't account for sampling bias - overestimates are more likely to be in the sample (since they've already happened). That is, things predicted for 2026 that happened 4 years only are in green, but 2026 predictions that only happen in 2030 aren't in red in your table.
Adjusting for this, experts probably do have a bias towards optimism (though small sample size so hard to be sure).
Also, re the argument that "fusion is always 30 years away" - there's consistent models for that (e.g. if you think it'll come 10 years after a visible breakthrough but aren't sure when that breakthrough will happen, modeling it as a poisson process will give you 30 year timelines until they jump down to 10). AGI isn't quite as single-breakthrough-dependent, but I can see the case for a similar model being consistent (where it mostly remains X years out then jumps down whenever something like gpt comes out).
One crux I note that I have with "rationalists"/AI doomers, that I see here on display:
Most of the items that Scott resolves positively on the AI predictions, I would resolve negatively, with the further conclusion that AI predictors are way too "optimistic". A major example would be: "See a video, then construct a 3d model". We're nowhere close to this. And sure, you can come up with some sort technicality by which we can do this, but I counter that that sort of reasoning could be used to resolve the prediction positive on a date before it was even made.
I find this further makes me skeptical generally of prediction markets, metaculus, etc. The best "predictors" seem to be good not at reading important details regarding the future, but at understanding the resolution criteria, and in particular, understanding the biases of the people on whose judgement the prediction is resolved.
As much as it's a good concept that disputes between models can be resolved via predictions, in cases where different models produce different presents, and both models accurately predict the respective present environments that the people using the models find themselves in, the entire system of making predictions doesn't work.
I continue to be annoyed at people identifying the phrase "Moore's Law" exclusively with Moore's 1965 paper on the optimal number of transistors per IC. When the phrase was coined in 1975 many people, including Moore, had pointed to other things also improving exponentially and it was in the context the conference at which Dennard presented his scaling laws tying all the different exponential curves together. From 1975 to 2005, when Dennard Scaling broke down, it was always used as an all-encompassing term to include transistors size decreases, power use decreases, and transistor delay decreases. It's only when the last of these went away that people started arguing over what it really meant and some people substituted the easier question of "What was the first exponential Moore talked about" for "What does Moore's Law really mean." /rant
The bit about AI alignment in "Crash Testing GPT-4" was miles better than most of the AI alignment stuff I've read. Practical, realistic, fascinating. That's real adult science!
In my limited testing and subjective evaluation, when translating from Russian to English, ChatGPT 3.5 beats Google Translate by a wide margin and translates about as well as a fluent but non-expert human. I have a degree in translation between this pair of languages and it beats me at translating songs. I think the chart pegging it at 2023 is in fact spot on.
The rank stupidity of questioning whether animals think, have theory of mind, etc., gives good reason to mistrust experts of all stripes, but particularly those who think about intelligence, consciousness, etc. (Even in early 90s, time magazine had a cover story asking something like "can animals think?") Combine this with claims about qualia, etc., denying same to other things plays similar role in misguided philosophical arguments about thinking, etc., denying the obvious that it arises out of physical structure of brain, chemical interactions, etc., an emergent property without any magic or special sauce involved. Which suggests machines "feel" something, even if something we might not recognize, are "thinking" even if we just want to frame it as "mechanistic" or "without the magic spark," etc. We're playing the same game with AI now. "OK, fine, crows use tools, but they still don't really "think" because . . ." "OK, fine, GPT seems to "understand" X pretty well and respond appropriately, and can fool most fools most of the time, but still can't . . . "
Sarah Constantin is an ace at topic selection and she goes deep enough into it to just satisfy me too, although I would have liked two areas in particular to be explored more: memristors and neuromorphic computing. If there is to be a breakthrough to get us past the Moore's Law limit that is approaching around 2030 then I think these two technologies will contribute.
The economist and research scientist, in their debate, seem to have forgotten the basics of GDP and *economic* growth (distinguished from other "growth" such as the invention of collectibles, or changes in price of existing goods, that is not economic).
Statements about GDP and economic growth are statements about household income and expenditure.
Yes, there are government spending, business capital formation, and net exports, but to a first approximation, GDP is what households get, and what they spend. *Economic* growth is an increase in household income and expenditure--an increase in the material welfare of households.
Again to a first approximation, households get their income from jobs. The debaters, while talking at length about automating jobs, have not bothered to look at *where the jobs are.*
A quick search of the Bureau of Labor Statistics's web site will tell you that the high-employment occupations are in health and personal care aiding, retail sales, customer service, fast food, first line supervision of retail,office and logistics staff, general and operational management, and physically moving things around - warehouse staff and shelf re-stockers, and order fulfilment. And nursing.
(Just beyond the top 10 we have heavy truck driving, the paradigmatic automatable job, that will be gone in 2016... no, wait, 2018... er, 2021 for sure... well maybe 2025 ... umm, ah, now we've tried to do it, it's a little tricky: perhaps 2040?)
If you want to alter the economy fast, you have to augment the work of people in those high-employment occupations, so the value produced per employee becomes greater. That way, you can pay them more. You're not going to make a big impact if you only automate a few dozen niche occupations that employ a few thousand people each. That doesn't move the needle on aggregate household income and expenditure.
Augmenting work in high-employment occupations could result in either of two outcomes (or, of course, a blend):-
a) if price elasticity of demand is high, and the cost of providing the service falls, then demand explodes. Health and personal care aiding is probably like this--especially if we look at a full spectrum "home help" aide job: some combination of bathing and grooming (nail trimming and the like), medical care (wound dressing, and similar), meal preparation, cleaning (kitchen, laundry, house general) and supervision of repairs and maintenance, operating transport (for example to shops and medical appointments), and on and on.
Currently about 3 million people are employed in the US doing health-and-personal-care work. Probably this means three to five million recipients. There is a potential market of 300M--if the price is right. Who wouldn't make use of a housekeeper/babysitter? (That's a rhetorical question: "every working woman needs a wife", as the saying used to go, back in the '80s and '90s, before political correctness. Of course the existing workers in this occupation are nearly all working with elderly people who are no longer able to do some things for themselves. So "every adult needs a HAPC aide", perhaps.)
So, maybe a sixty-fold increase for an occupation that is about a sixtieth of the workforce: a doubling of household welfare, potentially. All we need is for every occupation to be like that: job done!
But...
b) if price elasticity of demand is low, then the service becomes a smaller part of the total economy. This was the case for food: home prepared and consumed food used to be about 40% of household expenditure; now, it's single-digit percentages. People ate more food and better food as its price dropped; they just didn't eat *enough* more and better to maintain food in its rightful place at the top of the budget. Heating (space-heating, water-heating) and clothing have also shrunk in percentage terms.
It seems likely that retail sales, order fulfilment, and restocking fall into this category. Yes, people will buy more stuff if the price falls, but the increase is probably less than one-to-one with the price reduction. Some of the saved money will be spent on extra tutoring for the child(ren), haircuts, spring breaks, or pilates classes instead.
The same applies to the "customer service representative" and "fast food restaurant worker" occupations.
So automating these occupations (about 12 million employees?)--which, by the way, is easier than automating health-and-personal-care-aiding: it's already happening--will, er, "release" those employees.
Observe that it will also "release" a fraction of the "first line supervisor" and "general and operational manager" occupations, because they supervise and manage workers in these occupations so the effect would be super-linear. There will be *a lot* of people wanting to tutor kids, style hair, facilitate spring breaks, and teach pilates classes.
On the gripping hand, automation may well cause some occupations to grow hugely, that are currently very niche. Personal trainers. Personal coaches/mentors. Wardrobe consultants. Dieticians. Personal "brand"/social media image managers. Ikebana teachers.
And new occupations, or sub-occupations will appear: tutoring people how to get the best out of conversational AI interfaces, for instance.
I have no clue what will take off or to what extent, but I think the process of getting to mass market will take years. (And in the mean time, the effect of automation will be to shrink household I&E.)
That brings me to the third omission from the debate: constraints on investment. If you look over the BLS's list of high-employment occupations you will notice that most of them do not have large amounts of capital associated with them. That is likely to be because businesses employing people in those occupations can't borrow to invest very easily. They are low margin businesses.
There is some discussion of constraints in the Asterisk piece, in the form of "the time it takes for a crop to grow", or "the time it takes to acquire information", but no discussion of the basic (to an economist) constraint: access to funds. Now that interest rates are no longer near zero, this looms as a major limiting factor.
So I think that Hofstadter's Law applies: it always takes longer than you expect, even when you allow for Hofstadter's Law.
Tamay and Matt's conversation seems a bit strange and off base, missing the mark on reality. They sound like those experts talking about how far out certain effects will be in the second survey in Scott's article where they gave reasonable sounding answers for future dates, but those events had already occurred.
One point jumped out at me a lot. Matt's point in his second to last reply talked about how we 'have not seen' impacts from AI in the economy to boost it beyond the 2%. I'd argue the 2% we have been seeing is already coming from AI as older value adds have faltered. They assume a static world and miss the economic skullduggery that'd been going on to hide the long term decline of western economic growth which has been going on for decades. What propelled us in the 1900s is no longer working.
I.
They are just dead wrong. Companies like Amazon and Nvidia have seen dramatic benefits from using AI over the past 10 years as it has scaled up and become useful. Not just LLM, but also in ML and such for Google and the social media companies as well. Truly we may look back at the role of compute power in the economy more so than some special types of compute called AI.
When Brexit and Trump benefits in razor thin technical victories using Cambridge Analytica...that was a win for AI as well in the zero-sum political games which are not progress focused. - which was a great point Matt made. AI and compute only models have already been a huge factor in the economy.
They are also wrong in fixating on AI doing things humans doing...but neglect the things AI can do which humans can't do! Such as write personalised political ads for a million of swing voters. The AGI and AGI vs human tasks is too narrow a framing when considering the impact of AI on the economy, as it already can do things humans cannot which create economic value. Their fixation on job/skill/task replacement vs grand scale macroenomic outcomes is just too narrow.
II.
I think this has been masked by the fact of economic decline. Something like 95% of the growth in the S&P can be attributed to just 5 or 6 companies, tech companies using AI. Most of the economy is worse than stagnating and has been for decades.
This is a hard pill to swallow due to false accounting of inflation and other manipulated economic metrics and conditions created in a decade of low interest rates decreed by central bankers, but the publicly traded economy has been hyper concentrated in FAANGT and such for a long time now with AI of various types being their leading edge for the past decade.
Amazon and Walmart and others have already greatly benefited from AI in their warehouse and supply chain systems. Tesla has greatly benefited from various forms of machine intelligence automation and much of their value is tied in speculation about how amazing their AI self driving will be and their Dojo compute system. Not to mention AI and compute based allocation and coordination of data for their charging network, which wasn't as simple as/equivalent to building a bunch of gas stations along major highways.
I think the AI revolution and forces which drive economic growth have already begun and these two are busy talking about needless highly specific and arcane definitions of full AGI and when this or that factor will lead to greater than 2% economic growth trend. We are already at 20-30% economic growth or greater for AI using companies and they are picking up for the falling value of things like big commercial banks which are down 90% over the past 10 years.
This has already been happening and it is a wrong assumption to think the old factors of 2% growth from the 1900s are still in effect. They are no longer happening in western economies and have been dying out for a long time, hidden by financial games and zombie companies.
This entire article is junk because it focuses on GDP.
GDP, particularly for FIRE (Finance, Insurance and Real Estate) focused Western economies, is utterly meaningless. The massive fees extracted from Western populations by junk credit card fees, for example, "grow GDP".
So does the enormous US overspend on health care.
Skyrocketing college costs.
The list goes on and on and on.
Graph something concrete like electricity usage, energy usage, steel consumption and you get dramatically different outcomes.
I have not been following AI news all that closely, but it kind of seems like the hype is dying down a little? Like, three months ago everyone was acting like AI was going to take over the world soon, and now the attitude seems to be it's a kind of cool autocomplete.
For whatever it's worth, when I've used ChatGPT and Bing's AI chat it falls into the category of "kind of cool, but definitely not going to take anyone's job anytime soon."
If this is a representative summary of Asterisk's articles, it seems... very one-note. Are there any online rationalist/effective altruist hangout spots that aren't consumed with AI-related discussion?
"We should, as a civilization, operate under the assumption that transformative AI will arrive 10-40 years from now, with a wide range for error in either direction."
I feel like it's hard not to read this closing statement as a slightly more nuanced and somewhat less efficient way to say "we don't know." I don't mean that flippantly; I think there's a lot of value to thinking through what we do know and where our real confidence level should be. However, what really is the practical difference between 10-40 years with significant error bars and we just don't really know?
> I don’t think they emphasized enough the claim that the natural trajectory of growth is a hyperbola reaching infinity in the 2020s, we’ve only deviated from that natural curve since ~1960 or so, and that we’re just debating whether AI will restore the natural curve rather than whether it will do some bizarre unprecedented thing that we should have a high prior against.
Yeah, that's not how fitting curves to data works.
> the natural trajectory of growth is a hyperbola reaching infinity in the 2030s, we’ve only deviated from that natural curve since ~1960 or so
This is a very oddly stated claim. Most apparent exponential curves are actually S-curves that haven't yet hit the "S." My null hypothesis is that this is another example of such, not that there's some "natural" trajectory from which we've deviated. Why would one believe the latter?
Assuming the Angry Birds challenge is serious, what is it about that game that makes it so much more difficult for AIs to master? Surely if AIs can beat humans at Go and chess, not to mention master dozens of classic Atari games in a matter of hours, Angry Birds shouldn't be that much more difficult??
I asked ChatGPT how to avoid the problem of researchers always being depicted as chemical engineers wearing labcoats, then punched a summary into Dall-E. I think this is probably better representative: https://labs.openai.com/s/KJoq7h4Yfn5ukOnGmCQhQ3v8
All technological progress follows an S-curve, and an S-curve always looks exponential until you hit the inflection point...
We could ask AI what she thinks her existence will have on us mortals.
China's behind on some things on AI and ahead on others. These advantages and disadvantages are actually probably fairly structural to the point where I'd feel semi-comfortable making predictions about who will lead where (or at least what will be close and where one side will be far ahead). For example, LLMs are handicapped by conditions in China and will be probably indefinitely. On the other hand, facial recognition or weapons guidance have structural advantages in the huge security state serves as an eager market and data provider. And China tends to play closer to the chest with everything but especially military technology.
I'm also highly suspicious of anyone writing about Chinese AI who doesn't have a tech background or who doesn't have access to things like AliCloud. Chinese reporters and lawyers and propagandists are usually about as technically literate as their American counterparts. Which is to say not very. You really need to scrub in to understand it.
Not to mention a lot of the bleeding edge is in dense papers in Chinese. Plus the Chinese produce a gigantic quantity of AI papers which means simply sorting through everything being done is extremely difficult. You need to BOTH understand Chinese and computer science language to even have a hope of understanding them. But Ding seems focused on the consumer and legal side of it.
Also fwiw: I tend to think AI will lead to lots of growth but not some post-scarcity utopia type scenario. It will speed growth, be deflationary as per usual, etc. And I think that we're actually going to see a move away from gigantic all encompassing models towards more minutely trained ones. For example, an AI model based on the entire internet might be less useful than an AI model trained on transcripts of all of your customer service calls and your employee handbook. Even if you wanted to make a "human-like" AI ingesting the entire internet is probably not the way to do it. Though the question of whether we can keep getting smaller/better chips (and on to quantum) is certainly open and important to computing overall.
And the idea of using chips unilaterally as a stranglehold is a bad idea. We only somewhat succeeded with Russia and only then because the other important players went along with it. Any number of nations, even individually, could have handicapped the effort. They just chose not too. And keep in mind China is in the "chose not to" camp. This is without getting into rogue actors.
All in all I enjoyed the magazine although I did not agree with much of it.
Re "The Transistor Cliff": We are out of compute, now we must think.
People would normally cite the AI survey as "Grace et al" because it was a collaboration with Katja Grace as the first author. (I was one of the authors).
In the Asterisk article you say, "As for Katja, her one-person AI forecasting project grew into an eight-person team, with its monthly dinners becoming a nexus of the Bay Area AI scene." But the second author on the 2016 paper (John Salvatier) was a researcher at AI impacts, and the paper had five authors in total.
The article about people underestimating the arrival of the Soviet atomic bomb is almost completely leaving out the fact that the Manhattan project was porous as swiss cheese to Soviet spies. Stalin had full blueprints for the bomb before Truman even knew what an atomic bomb was. The Germans failed after going down a research wrong alley (whether Heisenberg did it on purpose or not is perhaps an open question, but the fact remains that they didn't focus on the right path). It is a LOT easier to build an atomic bomb when you have the full manual, plus a list of dead ends to avoid, courtesy of Klaus Fuchs, Theodore Hall, David Greenglass, etc. Unless this article is suggesting that China has spies in OpenAI, I think the analogy needs more work. (And if he is claiming that, that's a major claim and he should be more specific about why he's claiming it.)
Also, can someone please explain to me the ending of the chatbot story? I liked it up until the end and then I was just confused. Ads on fAIth???? What's that mean?
>Today’s Marginal Revolution links included a claim that a new Chinese model beats GPT-4; I’m very skeptical and waiting to hear more.
I, too, am a bit skeptical.
The first question I'd ask is "beats GPT-4 at what?" Competitive figure skating? Tying cherry stems in knots with its tongue?
https://www.reddit.com/r/LocalLLaMA/comments/14iszrf/a_new_opensource_language_model_claims_to_have/
Apparently it's C-Eval, "a comprehensive Chinese evaluation suite for foundation models."
https://cevalbenchmark.com/static/leaderboard.html
Why they didn't compare it to GPT-4 on the MMLU (a famous benchmark that people actually care about)? A mystery of the Orient. It seems GPT-4 performs better on the hard version of C-Eval. It's only on the easier one that ChatGLM scores higher. So take from that what you will.
I like how the guy starts out by assuring people the result's not fake. Kind of says it all.
Another bit of AI news. Someone may have revealed GPT-4's architecture.
"GPT-4: 8 x 220B experts trained with different data/task distributions and 16-iter inference."
https://twitter.com/soumithchintala/status/1671267150101721090
I'd put it down as a rumor (like the 100 trillion parameter thing) but a couple of people on Twitter are corroborating it.
Thoughts, anyone?
The single mention of Ukraine in the non-proliferation article is
> Belarus, Kazakhstan, and Ukraine traded away the Soviet weapons stranded on their soil in return for economic and security assistance.
Was it written before --2022-- 2014?
No, GPT-4 did not just make a gig worker solve a captcha
https://aiguide.substack.com/p/did-gpt-4-hire-and-then-lie-to-a
A lot of hints and suggestions by the prompter was needed, as well as relying on a fake (simplified) website, and also needed a human to engage with that website.
About the two surveys, I wonder whether the explosion of the field plays a role. "Expert" was defined as someone who has published at NeurIPS or ICML, probably in that year.
But for the bigger one, NeurIPS, in 2015 there were ~400 papers, in 2016 it was ~600, in 2021 it was ~2400 and in 2022 there were ~2700 papers.
I am not sure that this implies less expertise for each expert. It could be, because in 2016 there was a much larger fraction of experts with years or decades of experience in AI research, while the typical expert nowadays has 2-3 years of experience. But perhaps long-term experience on AI is useless, so who knows?
But it definitely implies that experts can no longer have an overview of what's going on in the conference. In 2015/16 experts could still know the essential developments in all of NeurIPS. Nowadays they are way more specialized and only know the stuff in their specific area of expertise within NeurIPS. So perhaps it's fair that 2022 experts don't know the state-of-the-art in all of AI, while 2016 experts still knew that?
Here is a chart for how NeurIPS has grown over the years:
https://towardsdatascience.com/neurips-conference-historical-data-analysis-e45f7641d232
I'm a bit angry about the Starcraft thing. Scott says that AI has never beat top humans players. But the reason it did not is that it was not allowed to.
The AI that played starcraft was seriously hobbled : it wasn't allowed to view the whole terrain at once by moving it's camera very fast. It wasn't allowed to micromanage each and every unit at once by selecting it individually. In general it's APM (action per minute) rate was limited to a rate comparable to a human.
That's like inviting a computer to a long-division competition, but only giving it a 100 Hz CPU.
The whole point of having AI instead of meat brains is that AI works on very fast CPUs with very large RAM. If you're gonna forgo that, you won't reap the benefits of having a silicon brain.
That article about AI taking tech jobs is really frustrating. He predicts by 2025, 55% of programmers will be using LLMs "for the majority of their professional programming work". He bases this on "the 1.2 million people who signed up for Github Copilot during its technical preview" (divided by a bit to restrict to U.S. programmers), but utterly elides the distinction between "signed up to try the new thing" and "have adopted the technology as a key part of programming from now on".
What percentage of those 1.2 million even got around to trying it at all after signing up? (I'd guess less than 50%.) What percentage of the remainder got frustrated and didn't put in the effort to learn how to work it? (I'd guess more than 50%.) What percentage of the remainder were able to find places to use it in a significant proportion of their work? (I'd guess less than 50%.) There's a huge, huge gulf between "is a tool someone's tried once" and "is an invaluable part of _the majority of_ someone's work".
In your article, you argue that experts are well-callibrated because they had as many underestimates as overestimates. But this doesn't account for sampling bias - overestimates are more likely to be in the sample (since they've already happened). That is, things predicted for 2026 that happened 4 years only are in green, but 2026 predictions that only happen in 2030 aren't in red in your table.
Adjusting for this, experts probably do have a bias towards optimism (though small sample size so hard to be sure).
Also, re the argument that "fusion is always 30 years away" - there's consistent models for that (e.g. if you think it'll come 10 years after a visible breakthrough but aren't sure when that breakthrough will happen, modeling it as a poisson process will give you 30 year timelines until they jump down to 10). AGI isn't quite as single-breakthrough-dependent, but I can see the case for a similar model being consistent (where it mostly remains X years out then jumps down whenever something like gpt comes out).
One crux I note that I have with "rationalists"/AI doomers, that I see here on display:
Most of the items that Scott resolves positively on the AI predictions, I would resolve negatively, with the further conclusion that AI predictors are way too "optimistic". A major example would be: "See a video, then construct a 3d model". We're nowhere close to this. And sure, you can come up with some sort technicality by which we can do this, but I counter that that sort of reasoning could be used to resolve the prediction positive on a date before it was even made.
I find this further makes me skeptical generally of prediction markets, metaculus, etc. The best "predictors" seem to be good not at reading important details regarding the future, but at understanding the resolution criteria, and in particular, understanding the biases of the people on whose judgement the prediction is resolved.
As much as it's a good concept that disputes between models can be resolved via predictions, in cases where different models produce different presents, and both models accurately predict the respective present environments that the people using the models find themselves in, the entire system of making predictions doesn't work.
I continue to be annoyed at people identifying the phrase "Moore's Law" exclusively with Moore's 1965 paper on the optimal number of transistors per IC. When the phrase was coined in 1975 many people, including Moore, had pointed to other things also improving exponentially and it was in the context the conference at which Dennard presented his scaling laws tying all the different exponential curves together. From 1975 to 2005, when Dennard Scaling broke down, it was always used as an all-encompassing term to include transistors size decreases, power use decreases, and transistor delay decreases. It's only when the last of these went away that people started arguing over what it really meant and some people substituted the easier question of "What was the first exponential Moore talked about" for "What does Moore's Law really mean." /rant
The bit about AI alignment in "Crash Testing GPT-4" was miles better than most of the AI alignment stuff I've read. Practical, realistic, fascinating. That's real adult science!
In my limited testing and subjective evaluation, when translating from Russian to English, ChatGPT 3.5 beats Google Translate by a wide margin and translates about as well as a fluent but non-expert human. I have a degree in translation between this pair of languages and it beats me at translating songs. I think the chart pegging it at 2023 is in fact spot on.
The rank stupidity of questioning whether animals think, have theory of mind, etc., gives good reason to mistrust experts of all stripes, but particularly those who think about intelligence, consciousness, etc. (Even in early 90s, time magazine had a cover story asking something like "can animals think?") Combine this with claims about qualia, etc., denying same to other things plays similar role in misguided philosophical arguments about thinking, etc., denying the obvious that it arises out of physical structure of brain, chemical interactions, etc., an emergent property without any magic or special sauce involved. Which suggests machines "feel" something, even if something we might not recognize, are "thinking" even if we just want to frame it as "mechanistic" or "without the magic spark," etc. We're playing the same game with AI now. "OK, fine, crows use tools, but they still don't really "think" because . . ." "OK, fine, GPT seems to "understand" X pretty well and respond appropriately, and can fool most fools most of the time, but still can't . . . "
Sarah Constantin is an ace at topic selection and she goes deep enough into it to just satisfy me too, although I would have liked two areas in particular to be explored more: memristors and neuromorphic computing. If there is to be a breakthrough to get us past the Moore's Law limit that is approaching around 2030 then I think these two technologies will contribute.
The economist and research scientist, in their debate, seem to have forgotten the basics of GDP and *economic* growth (distinguished from other "growth" such as the invention of collectibles, or changes in price of existing goods, that is not economic).
Statements about GDP and economic growth are statements about household income and expenditure.
Yes, there are government spending, business capital formation, and net exports, but to a first approximation, GDP is what households get, and what they spend. *Economic* growth is an increase in household income and expenditure--an increase in the material welfare of households.
Again to a first approximation, households get their income from jobs. The debaters, while talking at length about automating jobs, have not bothered to look at *where the jobs are.*
A quick search of the Bureau of Labor Statistics's web site will tell you that the high-employment occupations are in health and personal care aiding, retail sales, customer service, fast food, first line supervision of retail,office and logistics staff, general and operational management, and physically moving things around - warehouse staff and shelf re-stockers, and order fulfilment. And nursing.
(Just beyond the top 10 we have heavy truck driving, the paradigmatic automatable job, that will be gone in 2016... no, wait, 2018... er, 2021 for sure... well maybe 2025 ... umm, ah, now we've tried to do it, it's a little tricky: perhaps 2040?)
If you want to alter the economy fast, you have to augment the work of people in those high-employment occupations, so the value produced per employee becomes greater. That way, you can pay them more. You're not going to make a big impact if you only automate a few dozen niche occupations that employ a few thousand people each. That doesn't move the needle on aggregate household income and expenditure.
Augmenting work in high-employment occupations could result in either of two outcomes (or, of course, a blend):-
a) if price elasticity of demand is high, and the cost of providing the service falls, then demand explodes. Health and personal care aiding is probably like this--especially if we look at a full spectrum "home help" aide job: some combination of bathing and grooming (nail trimming and the like), medical care (wound dressing, and similar), meal preparation, cleaning (kitchen, laundry, house general) and supervision of repairs and maintenance, operating transport (for example to shops and medical appointments), and on and on.
Currently about 3 million people are employed in the US doing health-and-personal-care work. Probably this means three to five million recipients. There is a potential market of 300M--if the price is right. Who wouldn't make use of a housekeeper/babysitter? (That's a rhetorical question: "every working woman needs a wife", as the saying used to go, back in the '80s and '90s, before political correctness. Of course the existing workers in this occupation are nearly all working with elderly people who are no longer able to do some things for themselves. So "every adult needs a HAPC aide", perhaps.)
So, maybe a sixty-fold increase for an occupation that is about a sixtieth of the workforce: a doubling of household welfare, potentially. All we need is for every occupation to be like that: job done!
But...
b) if price elasticity of demand is low, then the service becomes a smaller part of the total economy. This was the case for food: home prepared and consumed food used to be about 40% of household expenditure; now, it's single-digit percentages. People ate more food and better food as its price dropped; they just didn't eat *enough* more and better to maintain food in its rightful place at the top of the budget. Heating (space-heating, water-heating) and clothing have also shrunk in percentage terms.
It seems likely that retail sales, order fulfilment, and restocking fall into this category. Yes, people will buy more stuff if the price falls, but the increase is probably less than one-to-one with the price reduction. Some of the saved money will be spent on extra tutoring for the child(ren), haircuts, spring breaks, or pilates classes instead.
The same applies to the "customer service representative" and "fast food restaurant worker" occupations.
So automating these occupations (about 12 million employees?)--which, by the way, is easier than automating health-and-personal-care-aiding: it's already happening--will, er, "release" those employees.
Observe that it will also "release" a fraction of the "first line supervisor" and "general and operational manager" occupations, because they supervise and manage workers in these occupations so the effect would be super-linear. There will be *a lot* of people wanting to tutor kids, style hair, facilitate spring breaks, and teach pilates classes.
On the gripping hand, automation may well cause some occupations to grow hugely, that are currently very niche. Personal trainers. Personal coaches/mentors. Wardrobe consultants. Dieticians. Personal "brand"/social media image managers. Ikebana teachers.
And new occupations, or sub-occupations will appear: tutoring people how to get the best out of conversational AI interfaces, for instance.
I have no clue what will take off or to what extent, but I think the process of getting to mass market will take years. (And in the mean time, the effect of automation will be to shrink household I&E.)
That brings me to the third omission from the debate: constraints on investment. If you look over the BLS's list of high-employment occupations you will notice that most of them do not have large amounts of capital associated with them. That is likely to be because businesses employing people in those occupations can't borrow to invest very easily. They are low margin businesses.
There is some discussion of constraints in the Asterisk piece, in the form of "the time it takes for a crop to grow", or "the time it takes to acquire information", but no discussion of the basic (to an economist) constraint: access to funds. Now that interest rates are no longer near zero, this looms as a major limiting factor.
So I think that Hofstadter's Law applies: it always takes longer than you expect, even when you allow for Hofstadter's Law.
Tamay and Matt's conversation seems a bit strange and off base, missing the mark on reality. They sound like those experts talking about how far out certain effects will be in the second survey in Scott's article where they gave reasonable sounding answers for future dates, but those events had already occurred.
One point jumped out at me a lot. Matt's point in his second to last reply talked about how we 'have not seen' impacts from AI in the economy to boost it beyond the 2%. I'd argue the 2% we have been seeing is already coming from AI as older value adds have faltered. They assume a static world and miss the economic skullduggery that'd been going on to hide the long term decline of western economic growth which has been going on for decades. What propelled us in the 1900s is no longer working.
I.
They are just dead wrong. Companies like Amazon and Nvidia have seen dramatic benefits from using AI over the past 10 years as it has scaled up and become useful. Not just LLM, but also in ML and such for Google and the social media companies as well. Truly we may look back at the role of compute power in the economy more so than some special types of compute called AI.
When Brexit and Trump benefits in razor thin technical victories using Cambridge Analytica...that was a win for AI as well in the zero-sum political games which are not progress focused. - which was a great point Matt made. AI and compute only models have already been a huge factor in the economy.
They are also wrong in fixating on AI doing things humans doing...but neglect the things AI can do which humans can't do! Such as write personalised political ads for a million of swing voters. The AGI and AGI vs human tasks is too narrow a framing when considering the impact of AI on the economy, as it already can do things humans cannot which create economic value. Their fixation on job/skill/task replacement vs grand scale macroenomic outcomes is just too narrow.
II.
I think this has been masked by the fact of economic decline. Something like 95% of the growth in the S&P can be attributed to just 5 or 6 companies, tech companies using AI. Most of the economy is worse than stagnating and has been for decades.
This is a hard pill to swallow due to false accounting of inflation and other manipulated economic metrics and conditions created in a decade of low interest rates decreed by central bankers, but the publicly traded economy has been hyper concentrated in FAANGT and such for a long time now with AI of various types being their leading edge for the past decade.
Amazon and Walmart and others have already greatly benefited from AI in their warehouse and supply chain systems. Tesla has greatly benefited from various forms of machine intelligence automation and much of their value is tied in speculation about how amazing their AI self driving will be and their Dojo compute system. Not to mention AI and compute based allocation and coordination of data for their charging network, which wasn't as simple as/equivalent to building a bunch of gas stations along major highways.
I think the AI revolution and forces which drive economic growth have already begun and these two are busy talking about needless highly specific and arcane definitions of full AGI and when this or that factor will lead to greater than 2% economic growth trend. We are already at 20-30% economic growth or greater for AI using companies and they are picking up for the falling value of things like big commercial banks which are down 90% over the past 10 years.
This has already been happening and it is a wrong assumption to think the old factors of 2% growth from the 1900s are still in effect. They are no longer happening in western economies and have been dying out for a long time, hidden by financial games and zombie companies.
Re: Economic doubling times
This entire article is junk because it focuses on GDP.
GDP, particularly for FIRE (Finance, Insurance and Real Estate) focused Western economies, is utterly meaningless. The massive fees extracted from Western populations by junk credit card fees, for example, "grow GDP".
So does the enormous US overspend on health care.
Skyrocketing college costs.
The list goes on and on and on.
Graph something concrete like electricity usage, energy usage, steel consumption and you get dramatically different outcomes.