The child in the Mother and Child image, however, has an extra thumb or some kind of appendage, sticking out of the back of his thigh, which is partly why I rated that one as AI.
My immediate reaction was "Scott included a lot of AI images with less-ambiguous errors than that" but my guess is that Scott just missed the other errors; they were all fairly subtle.
For me, the "AI tell" is the fussy, overly complicated earring, which doesn't make visual sense and clashes with the peasant-style kerchief on the woman's head.
This was a fun exercise! And it is slightly instructive. But not more than that: the popular impressionist "AI" paintings are just altered copies of actual impressionist artists, e.g. Van Gogh or Renoir.
I'm not the person you asked, but I see a lot of Edouard Henri Leon Cortès in "44: Paris Scene", particularly "Porte Saint Denis" and some other unnamed paintings of his. I also see some Antoine Blanchard, although the "blur" is more evident in the AI painting than in any of these two artists' work. I was fooled!
"11: Green Hills" is basically Monet, you can see the 'inspiration' from his various Poppy Field and Meadow series.
I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here. It's more the style that is clearly ripped off, and applied to new scenes as described in the prompts.
Yeah, good examples! Thanks! It would be interesting to see the prompts on the AI imagery (or the process in general if it is more complicated than a single prompt)
You can have a lot of fun with Midjourney, I got this (https://imgur.com/a/nzqcom8) from the following prompt: "a defeated Jeanne d'Arc in a wheat field contemplates while a castle burns in the distance, Viktor Vasnetsov". If you're more descriptive and use more tags as well as artist names, you can concoct stuff that would fool.. well.. about half of the people in here apparently!
Note: I did not share that image to suggest it's a great one, but rather to show how easily you can generate something that would look pretty convincing when not zoomed in.
>I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here.
On a related note: I generated images of "kitten pouncing on owner's toes" with DALL-E, and then tried searching for the image with Google Lens aka image search, and I don't find anything with e.g. the kitten's paws and owner's toes in the same relative position. So, even in this case (there are a _lot_ of kitten pictures on the net), this doesn't look like retrieving a memorized picture.
I don't buy it TBH- the subject matter is very similar but the actual artwork is pretty different. For example if you zoom in on any little square inch of one painting and try to match it to the other it's not a very close match, which seems to me to make calling it a "copy" very suspect.
I'm no kind of expert, but those look pretty different to me. The AI one has no people, the sky is very different, the trees are very different, the lighting is quite different, the overall coloration is pretty different, the AI one seems to have a higher level of detail on everything except the street surface. No individual element of the scene looks very similar if I compare them.
They are both outdoor dining areas beneath an awning on a street at night, but surely there are many pictures that meet that description? I could see how somehow who was familiar with the human one might be reminded of it by the AI one, but what makes you consider this "an altered copy" instead of just "similar subject matter in a vaguely similar style"?
It looks so much like that painting that I voted it human on the assumption that I'd remembered the real painting as having more vivid colour than the above, or possibly that Van Gogh painted the same scene twice
They're certainly copying the style of those artists- though not so much the specific artist as sort of a blend of impressionist artists in general.
Unlike the human artists, the AI isn't really capable of inventing new visual styles to convey semantic information. If you brought Van Gogh into the present and asked him to paint a shopping mall, he would definitely come up with a new twist on his visual language to convey the vibe of the place- wheres an AI copying the style is going to shoehorn in stylizations that were intended to emphasize things Van Gogh found interesting about 19th century France, and it's going to look awkward and semantically incoherent.
So, the AI is copying the most important part of the impressionist images- repeating the same visual insights about the same subjects, rather than saying anything new about reality. That said, this is the sort of copying that human artists do pretty frequently- not often to the degree that these models do when prompted to replicate styles, but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
And that is also something that can be mitigated a bit with the right kind of prompting and careful selection- letting the human inject a bit of original semantic content into the images. Once we have really strong multimodal models that are able to use the same internal world model acquired from language and audio and video and so on to images, we may also see that lack of original semantic content start to fade.
>but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
My immediate thought when Scott commented that the image with the huge cats in the throne room represented painting skill a lot of human artists would envy, was precisely something like this: One of the main tells of AI art is that the images would have taken immense skill and effort to paint by hand, but the subject matter is horribly gauche. It is difficult to believe a human would be able to achieve that level of skill without also acquiring some taste, and that they would be willing to spend weeks creating a painting with nothing deeper to say than "wouldn't it be cool if some imaginary queen had gigantic cats as guardians?".
Incidentally, this is why warhammer illustrations are easy to mistake for AI. (Illustrations as a genre, really)
The opposite end of the spectrum are paintings made by someone with little to no technical skill, but a desperate need to convey some message. That's a very human thing. Hence Angry Crosses was the image I was most confidently wrong about.
On that note, I was surprised to read The Mourning of Christ was one of the images people tended to get wrong, as that was the one I most confidently marked as human, because the subject is clearly a corpse slumped in a way no living human would contort, and so I would not expect AI to depict that kind of pose.
Do you think the results of this test would be any different if some high quality printouts of the AI generated oil paintings were put in frames next to the real things?
Imagine it upscaled to enough pixels that the human eye wouldn't be able to tell then 😉
I guess the real thing I'm trying to get at is: what it is about the physical medium of an oil painting that's important, and is there something fundamental about it that current AI technology couldn't replicate in a way that would fool people?
I know this is obtuse by your standard—but FYI, even a very high-quality, high-resolution Giclée is easily distinguished from an oil painting in person. You can try this experiment at an art museum if you like.
I think you're probably right. But is the difference important, really? I find it hard to believe the texture and brushtrokes of an oil painting are significantly enjoyable to people, especially given you can't touch paintings in an art gallery. Perhaps there's value in those features to people in the sense that they are a mark of human hands - but is there more to it than that?
As you've observed, I know almost nothing about art, so I would be interested in the perspective of someone who does.
This self portrait is scanned at a resolution where you can see the brush strokes. It looks incredible on my massive work monitor. Wikipedia has more examples of painting scanned at pretty high res.
It's not just about the end result - a big part of art is the artistic process itself. It's most visible with unconventional mediums - matchstick sculptures, life-size lego structures, patterns in flower gardens, etc. In looks alone, they will virtually always look worse than traditional clay sculpture or a traditional painting with similar time spent on it, just because it's so much harder to work with. But it's because it's so hard to work with that people find it so impressive. To this day many people don't recognize digital art as real art - and it's almost entirely due to how much easier it is (try to Ctrl+Z a brush stroke). AI generators have brought down the effort required basically to zero - and so, following the rule of inverse proportion, most people's threshold to be impressed by AI art shot up pretty much to infinity.
Brushstrokes have a visceral effect on some observers. That's also why it can be so different to see a painting in person rather than reproduced in a book: the tangible qualities of it (including the scale, color, visible marks) are all a direct communication that can get lost in a printed or digital image of the work.
Oil paintings have 3D texture. Printouts do not. So the comparable would be a robot powered by AI painting with oil on a canvas. Comparing digital images is flattening the artistic expression of the paintings.
I’m not taking away from the power of gen AI but it’s a tool. The images are generated by talented people who can prompt the tool because not everyone can get gen AI to create such well composed images. Just as some oil painters create masterpieces while I am a step above finger painting.
Or possibly an AI making a 3D version with simulated brush strokes. Live it up, include those little cracks for what's supposed to be an older painting.
It seems like an obvious next step is to merge a powerful prompt driven image generator with robotically manipulated traditional brushes, paints, canvas, etc., to see what caliber of wall art (down to the textured dimension of the actual brushstrokes) it can produce.
While correct, that's only relevant if you think there's something important about the medium. Or if your main concern is detecting that it wasn't by an AI.
Is your concern with identification of artist or of quality? I didn't read this as a test of "fooling the eye", but rather of producing "quality work". If it's about "fooling the eye", then I consider it truly trivial, especially given the preprocessing. If your concern is with the quality of the work, then I don't see the medium as that important (though some media are better for some presentations).
I wonder about the version of this that doesn't require the test take place on a computer display? I believe in the transmission of art, the experience that comes from interacting with it with full attention. Context is important: seeing art in a gallery, often within a sterilized space and wedged between dozens of other pieces, is a particular kind of experience, and art doesn't often survive it. If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
In my experience, most people I know seem to 'take in information,' which is a very different act than experiencing. A different test (?) involves noting how individuals (or cultures?) are changed by artworks.
I notice I'm not writing in my voice. The couple years I spent in San Francisco I had to learn another way of speaking, which was oriented toward precision, a constant clarification. I'm defensively tending toward that voice in this post, when what I'd rather do is sing. Somewhere under all the apprehension in the world is an understanding, and under that is awe, and the art made with loving attention rises to meet me there.
> If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
That was how I felt too, until I saw a Rothko exhibit. Being able to get really close to the paintings, spending time with them, the quietness, dim lights, the sheer size of the paintings and Rothko's style itself gave me way more "art appreciation bandwidth". Or maybe I'm getting older.
I suspect that the rise of AI art will get a lot of people going to museums again to experience physicsl art in person. Over the past few decades, we had gotten used to seeing computer images of physical art, rather than the real thing. But the colors and textures just aren’t the same, even though the visual sensation is quite close.
Not that purely digital art isn’t good - it’s just a different thing.
I’m guessing that some portion of new physical art hanging in galleries will be based off of AI-generated images. The next evolution of paintings from sketches and photographs. I think it’s kind of exciting.
Yes. Seeing a painting in person is a totally different experience than on a screen or in a book. I knew Sargent's famous portrait of Henry James but viewing it at The National Gallery in London, I discovered the brush strokes and texture which are flattened by reproduction.
>Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
I'm not one of these people, but I *do* consider some of history's "greatest" human artists as mostly producing hideous slop, however well made and well executed and far beyond my complete lack of artistic ability. Not sure where that puts me.
Right. I did fairly well on the test, and one of the heuristics I used successfully was identifying hideous pieces as human, thinking that few people would willingly generate hideous AI art unless it was a deliberate curve ball to throw judges off in this contest, while humans have at least one clear motive to dabble in hideous styles: being bored with the aesthetically pleasing ones. See: architecture discussion on this blog.
Art I like is good, art I don't like is slop, Ai art is slop. I'm not surprised I had trouble distinguishing between human slop (that skull painting etc.) and Ai
There's plenty of art I don't like that I can respect as being real or good art, as demonstrating an impressive set of skills, or as producing something people in general judge as good or valuable in a way I value indirectly.
Human art one dislikes might look superficially like slop, but it usually isn’t slop. “Slop” is a bunch of stuff with no reason to it (see the discussion of the blue and red arch in the OP) while the human art is usually getting at something, even if you don’t get it.
Yeah, I think a lot of the human art here is aimed at a very specific audience with a lot of context and background knowledge and not remotely meant to appeal to the general public. That kind of expert-oriented art would seem to be especially hard for AI to match, but only experts will be able to tell that. Meanwhile a lot of AI art looks great to a mass audience.
Notably, this kind of self-referential appeal-only-to-novelty-starved-experts creation appears to be how architecture went from things that are nice to look at to things that are ugly to look at, basically because the field was allowed to judge itself and architects began catering to the tastes of other architects rather than anyone else.
It seems like a broadly unhealthy tendency to indulge.
(Thinking in particular of Item #46 here and the related "Whither Tartaria" post)
Also, many of these great artists are great because they invented a movement, or captured a historically important moment in time. Not necessarily because the paintings themselves are wonderful.
I think AI is very good at art with a medium amount of detail (such as impressionism), but not good at art with a high amount of detail, (such as a realistic battle scene), or a low amount of detail (such as pixel art). I think what the low and high detail art have in common is that you really need to know what you're looking at. With medium level detail you can kinda get away with not knowing what a hand is, whereas with high level detail (e.g. battle scene) you really need to know the number of fingers and in what order they are, and with low level detail (e.g. pixel art) you need to be able to represent the whole thing with a couple of well placed pixels while having it still readable as a hand.
If you had asked me before generative AI I would've guessed that it would be linear (e.g. best at pixels, then medium, then high detail), because I assumed we'd make AI focused on understanding things, with it becoming more difficult to understand things the more complex they became. Instead we focused on imitation, which succeeds more if it can fake understanding, and thus at mid-level detail.
I think we can probably do good AI pixel art, but the general purpose AI picture generators we have aren't the right tool for that particular job (yet?).
I think pixel art would actually fall under "high detail" here. To pass as real pixel art, every single pixel needs to be correctly aligned to the grid. Current AI image generators are not very good at that (but they are getting better.)
All art is selected/curated. If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles? To a teenager drawing poo on a bathroom stall? To one of the great human artists, but at the start of their journey when they still couldn't draw well?
If we take the curation efforts into account, I'd go as far as say that AI is even more impressive. Scott just took extremely famous paintings, basically the pinnacle of human art. And compared to mostly art generated by some dude that reads his blog.
>If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles?
That isn't quite right, no.
The AI model is trained on an enormous corpus of human art and then instructed on imitating it (without replicating it too closely). It is a tool designed from the ground up to be the best artist it can be, with the best education it could be given. A 3 year old with a crayon has been trained on nothing and has experience of nothing.
The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
I was arguing against OP's point that AI art "won" here because it was curated while human was not. (that's what I understood, correct me if I'm wrong)
> The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
This is still curation, just less granular. We're selecting on the artist instead of the artwork level. Is this the correct comparison? _Really_ depends on what are you trying to measure.
But just the fact that we're saying "known-skilled artists" is the bar you need to clear to fairly compare to AI is telling.
The thrust of my argument was meant to be that since the AI side of the test involved heavy human curation, it's not really fair to call it a Turing test.
But I think you make a good point, that both sides were curated by the same person. Considering that, I think it's a fair test. But I don't think it's a Turing test.
I haven't read Turing himself, but the idea of a Turing test is that you expose the AI to a person through some channel that obfuscates the source of the communication. Beyond that identity obfuscation, both sides interact naturally. (With "nature" in quotes for the AI.)
But this test introduces a skilled human curator between each side and the judges. So it's fair, and it's telling us something, but it's not telling us what "winning a Turing test" would tell us.
The real question is about what work the AI can do in the world. Can it replace an artist? And the answer is yes, as long as somebody like Scott puts time into curation. Since the human artists already curate their own art (at least to the extent of making sure there are the right number of fingers & the written words are real words) they still have a pretty good advantage.
The art was curated by the same person, but for backwards reasons. AI art was chosen to imitate human art and human art was chosen to imitate AI art. Scott was intentionally making it hard to tell which was which. Completely un-curated would have been too easy, as the AI art would have been much more obvious (like the few Scott put in that were very obvious).
The art was also actually double-curated, since Scott wasn't just putting prompts into AI engines but was accepting submissions for "good" AI art.
Exactly! If you were to take a bunch of uncurated human art and ask people, "is this human or AI?" you wouldn't expect them to have a difficult time determining that it was human - even if it was full of a bunch of crap.
If the question were which looks better when curated and stripped of context, this would be a good test. But the question is, "can AI art fool humans into believing it's not AI-generated?" and in curating the dataset to obfuscate which is which, Scott intentionally excluded all the tells an AI artist isn't able to replicate. At best, you get a narrow version of the question, "Can certain AI images pass for human when we heavily select for the kinds of things computers do well at?"
I think it's impressive that the AI side did as well as it did on even this heavily biased test. Just 5 years ago, nobody would have believed it. Given certain trends, I expect 5 years from now an AI will be able to drop many of the heavy caveats/redactions/biases built into this test and do well in a head-to-head competition with history's greatest painters.
This isn't the progression people expect, though. They expect AI art to do as well as a 5th grader first, then get as good as a college student, etc. But they're going through the same type of training, so instead of taking the traditional path to developing their skills, they'll go through the Uncanny Valley on their way to excellence.
He doesn’t actually claim that a 30% rate of fooling humans counts as passing - he just predicts that by the year 2000, there would be computers able to fool humans 30% of the time in a five minute conversation.
What he actually proposes is more sophisticated. It’s really much more like, if you could have extended interactions, and come away with the same sort of value that you do from equivalent human interactions, then we might as well say it’s as good as a human. In at least some parts of the paper, he takes seriously the idea that not being able to enjoy strawberries and cream is a potential problem, because shared enjoyment is part of what we look for in human friends.
But I think the really surprising parts are how well he predicts the capacities of computers decades in advance (a decade or so before Gordon Moore invents his eponymous law even), the fact that he considers ESP to be so well established that it might break his test, and the suggestion he makes at the end that some sort of machine learning is likely to be a more effective approach than directly programming a conversational AI.
Yes, I definitively would recommend reading Turing's original paper. It also anticipates and answers many of the objections people still bring up to this day.
Agree. I see a lot of stuff on the internet saying the Turing test has been passed and when you read the actual details they did not do an actual Turing test as he described it in his paper. I contend that no computer has yet passed an actual Turing test, mostly because no one runs an actual Turing test.
You should read Turing's original paper. It's quite short and easy to understand for the layman.
He explores some great nuances and also anticipates most of the objections people have later brought up. (Mostly people who vaguely heard of his test, but haven't read the paper.)
The problem is that if you're comparing human art to AI art, you don't want the AI art to benefit from human curation.
Imagine a chess-playing bot that has a human grandmaster correcting its moves whenever it fumbles: it'd be hard to claim that it's playing "AI chess" at that point.
There is a separate side question of "how does a human+AI system perform against an all-AI system or an all-human system".
To steal Matthew Talamini's analogy below, I think it's more like a blind taste-test than a game of chess.
If we decide to go with the chess metaphor, though, it's more like "we'll select the best AI player and the best human player to play a match", or perhaps "we will select the best 10 AI-vs-human games and the best 10 human-vs-human games to look at"—a human "correcting" *every move* is more like digital art, i.e. a computer doing what a human commands.
There is no input from the AI in such a scenario, which seems clearly distinct from this. The level at which it is curated isn't every move nor every pixel, but complete works / complete games.
From my point of view, a 3 year old's scribbles are more interesting because they have intention behind them. If you've ever asked a kid that age what they're drawing, they're often making a picture of something very specific – it's just that it's wildly abstract. I'm not saying that makes it delightful to look at, but it does make it interesting! Whereas AI can sometimes be exciting to look at, but fundamentally lacks intention. I mean the human prompt is there, but there's a disconnect between that prompt and what's visually created. The place where a conscious mind would intentionally organize the visual around the idea is just missing.
Also for what it's worth these paintings are mostly B Sides and I think it would be hard to argue that they really represent the pinnacle of human artistic achievement.
If "intention" was something intrinsic to the art itself, there should be no difficulty telling AI and human art apart.
I.e., it's fine to value it, but it is like valuing the state of mind a chef was in when cooking a meal: "it's not interesting to me to eat it" and "something is different about the meal itself" ought be distinguished.
Maybe intention is the wrong word. But human art does have consciousness behind it. AI art doesn't. I personally don't have much trouble distinguishing AI art and human art apart – for why, see Ilzo's comments in the original post. If you take the stance that these add up to "small inadequacies" then I guess you might think AI art carries as much or little meaning as human art, but what those inadequacies do is betray a lack of consciousness, which means that the whole project is meaningless (and on an aesthetic level, it just falls apart and is actively grating to look at).
I don't like the food comparison (cooking can be an art, but eating itself is a survival need, so not a 1:1 correlation). But if I were to hold with your metaphor, I'd be saying that something does taste different about the visual "meal" AI creates, and it tastes off.
It would be very easy to distinguish AI generated images from a child's scribbles, because AI can imitate the most skilled human artists but has no intention, while a child has very little skill but a clear intention.
I think what you're looking to account for here is the presence of taste. But a taste filter has been applied to art history (what survived, which are well known) as well as the AI competitors. It would be interesting to see this test designed and run again by Scott's artist friend.
What would be the right standard for a purely AI competitor though? Doesn't seem like a simple answer. You can argue the prompting process itself reflects human input, plus does the AI get a chance to review it's own work and decide itself what's most likely to pass the test?
Maybe giving a set of human artists and a set of AI artists identical prompts? - I think this has flaws, as well as being obviously challenging to set up.
Maybe an AI has to set the prompts and select the images itself with only some initial prompting about the nature of the test?
The test feels much more legitimate to me when I consider it as a kind of blind taste test.
If you want to prove that California can produce good wine, a blind taste test versus French wine is really useful. If a lot of people have the idea that only French wine is any good, a fair blind taste test between the curated best of California and the curated best of France is absolutely called for. It can't establish that California wine can replace French wine, or that people generally won't be able to tell the difference. But it's still useful.
Same with AI. Anybody who has the idea that only human-made art is any good should update based on the results of this test. It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful.
You can for sure demonstrate that AI is capable of being used to produce some kind of image that has some kind of broadly defined aesthetic value. The really hardline anti AI art crowd should dial it down a notch.
It's not an argument for AI maximalism though - human artists remain vastly superior to AI artists on almost every possible metric.
Yeah, I think we're agreement here. And it's probably the source of our original disagreement.
You seem to understand the "proper" Turing test better than me. So I guess that taking that view this is not a valid test is correct.
But the "popular" intepretation of the Turing test is more like a taste or capabilities test. Can AI generate art that's as good as a human?
> It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful
I think all of these statements need an explicit "for now" appended. AI art is advancing super quick. Most of these issues Scott curated out, like messed up hands and text won't be an issue soon. In fact, the flux models that came out recently basically solved text already.
> It doesn't show that AI artists can replace human artists
I'd expand on this. A _lot_ of work human artists do will be replaced. Most of corporate soulless art will be automated. Generating portraits of book characters, etc.
There will always be a market for human art, but will be linked to prestige. More than it is today.
It also will never replace the art that's more than "pretty picture". Art is more than the physical thing itself for many people. If who created it matters, then that kind of art is impossible to automate by definition.
Note that what they're comparing it against isn't human art, but human art plus human selection. I think if you're trying to find the more aesthetically pleasing images between AI slop and human slop, the AI slop would win handily.
> I think what people are talking about when they say they don’t like AI art is AI art plus unskilled human selection, which also was not tested here.
I think a lot of people just think AI art is inherently bad. But I agree that there's a big problem of people posting AI slop instead of going to the extra effort and just posting particularly interesting ones, or using all the methods they have at their disposal to add artistic control to the result.
I think one of the things I like about AI art is that in order to make it stand out you have to make more interesting stuff, whereas for a human you can basically paint a bowl of fruit and if you do a really good job people are impressed. Then again, that's exactly why I thought the Victorian Megaship was AI, so maybe I'm way off base with that and there's plenty of human artists that do the kind of thing I'm interested in.
I feel like the art chosen does skew the results fairly significantly. I am not an art expert, I expected it to be very hard, and I got about 60%. However, not only was the AI art chosen to avoid obvious tells, I think there was extra selection to avoid many subtle tells.
It felt like human-made computer generated had more detail than average, and the very simplest looking images were more likely to be AI. On impressionism, there were no actual images by Van Gogh or Monet, who people really like, and so it didn't surprise me that people rated the AI mimicry of those artists very highly. More people like Van Gogh's style than like Gauguin and the village of Osny (also good).
I actually left the exercise feeling that it might be easier to differentiate AI art than I thought, not harder. But my bar was very low.
re why cat court is AI, lots of detail = AI works there. I guess it failed us on the human drawn ship. But if you add in topic like silly and super detailed is surely AI
All the cats have the same expression, displaying a transitory emotion; for all of them to share it is both unnatural, and boring for a human to recreate.
It's a really weird composition, too. Why is the king cat's body hidden in some huge alcove while the queen cat's body sprawls across the floor? The small orange cat is sitting on nothing, next to a stool. The tabby cat at the right is standing on two legs and doesn't have front legs at all. The curtains are pulled very asymmetrically. Where are the lady's legs?
The painting also fails to tell a story, which you'd think the artist would have tried to do if they're going to put so much effort into such a silly painting. Who actually has the power here, the lady or the cat? They both have identical crowns. The lady is in a central more powerful position but she's clearly dominated by the cat behind her. Is this some kind of formal occasion in the throne room? Some characters seem to think it is, while others are just casually sprawled around.
I was surprised so many people missed the ship. No current AI model can render such complex, coherent rigging and ratlines without at least some ropes blurring together or bending. Seems like it's obviously a 3D render.
I think the main tell for me for Cat Court is that it seems too glossy, in a way that's very common in AI art but rare in human art of that style. Just about everything that's brightly-lit seems shiny, even if it clearly shouldn't be by virtue of having a rough or fluffy texture.
The same kind of glossy feel seems to happen by default in 3D-rendered artwork, unless you do a lot of worth with shading and textures to get a different effect. Come to think of it, that might be part of why human-made digital images were often incorrectly identified as AI.
Looking at it more closely, an additional tell is that the style of the candle flames are inconsistent with one another and with the overall style of the image. The ones on the left candelabra are fairly normal, but the ones on the center and right candelabras seem to have lens flare effects.
count me as an anti-AI person specifically _because_ I think the AI art is excellent. But that's just because I think artists are important parts of society and I don't want them to disappear
What's the point of an artist that can't produce anything of value? The same applies to any other worker. Humanity will be made obsolete in due time. You can't build something new without destroying what came before.
Your first reply said they were "important parts of society"; now it seems like you're arguing the problem is how they would feel about it, rather than how the rest of civilization would get on without them? Those seem to me like very different, almost diametrically-opposed arguments.
The normal way that I evaluate whether something is an "important part of a system" (as you suggested) is by asking what would happen to the system if the part were absent or non-functional. For example, if I want to know how important a doorknob is, I ask what would happen if there were no doorknob, or if the doorknob stopped working. This strikes me as the obvious comparison invited by your turn of phrase.
It doesn't particularly matter what I meant by the phrase, though, because I'm paraphrasing YOUR assertions back to you in order to check my understanding of them. Even if what I said is ambiguous, clarifying it wouldn't particularly help, because the entire point is to figure out what YOU meant; once your position is clarified, my attempts to paraphrase it are dispensable.
And you notably did not provide any clarification of your own position.
To me, your question looks suspiciously like you were trying to insinuate that maybe I'm in favor of killing people. Which would be totally false, and not remotely required by the counterfactual comparison (which just requires that society stops utilizing their art), and not my fault even if it WERE implied (since I'm just discussing the meaning of YOUR assertion), and a brazen evasion of my question about what YOU meant. If that was what you were doing.
I don't think AI art should replace human art (because I don't think AI art looks good or holds much value), but I also think that even if it did human artists would still have plenty of value to contribute. Art isn't just making pictures (or sculptures or novels or whatever), it's making meaning through every element of a picture. Unless it becomes conscious (instead of just being an algorithm stuffed into a fancy sweater), it won't be able to do that. Of course, a human artist could work with it to make something that makes meaning, but I tend to find that AI fundamental lack of meaning interferes with a meaningful output. Idk, I have yet to see an interesting collaboration between human and AI, but that doesn't mean that it couldn't happen.
The kid does have a thumb; it's just mostly-hidden by the rest of the hand.
Thought so as well.
I thought that, whereas the columns of Ominous Ruin are a defect that gives it away
Yeah, I'm surprised he excluded that one, but not String Doll (whose middle finger splits in 2).
The child in the Mother and Child image, however, has an extra thumb or some kind of appendage, sticking out of the back of his thigh, which is partly why I rated that one as AI.
My immediate reaction was "Scott included a lot of AI images with less-ambiguous errors than that" but my guess is that Scott just missed the other errors; they were all fairly subtle.
For me, the "AI tell" is the fussy, overly complicated earring, which doesn't make visual sense and clashes with the peasant-style kerchief on the woman's head.
That's a good description of a lot of AI: fussy and over complicated.
If not, I'm sure the subject of "Muscular Man" would be happy to lend the kid one of his.
This was a fun exercise! And it is slightly instructive. But not more than that: the popular impressionist "AI" paintings are just altered copies of actual impressionist artists, e.g. Van Gogh or Renoir.
Can you share links to the ones you think are originals?
I'm not the person you asked, but I see a lot of Edouard Henri Leon Cortès in "44: Paris Scene", particularly "Porte Saint Denis" and some other unnamed paintings of his. I also see some Antoine Blanchard, although the "blur" is more evident in the AI painting than in any of these two artists' work. I was fooled!
"11: Green Hills" is basically Monet, you can see the 'inspiration' from his various Poppy Field and Meadow series.
I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here. It's more the style that is clearly ripped off, and applied to new scenes as described in the prompts.
Yeah, good examples! Thanks! It would be interesting to see the prompts on the AI imagery (or the process in general if it is more complicated than a single prompt)
You can have a lot of fun with Midjourney, I got this (https://imgur.com/a/nzqcom8) from the following prompt: "a defeated Jeanne d'Arc in a wheat field contemplates while a castle burns in the distance, Viktor Vasnetsov". If you're more descriptive and use more tags as well as artist names, you can concoct stuff that would fool.. well.. about half of the people in here apparently!
Note: I did not share that image to suggest it's a great one, but rather to show how easily you can generate something that would look pretty convincing when not zoomed in.
>I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here.
On a related note: I generated images of "kitten pouncing on owner's toes" with DALL-E, and then tried searching for the image with Google Lens aka image search, and I don't find anything with e.g. the kitten's paws and owner's toes in the same relative position. So, even in this case (there are a _lot_ of kitten pictures on the net), this doesn't look like retrieving a memorized picture.
> in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here
44 is very very very similar to Renoir's The Grand Boulevards. I marked it as human during the test because I thought that was what I was seeing. https://en.wikipedia.org/wiki/The_Grands_Boulevards
"Riverside Cafe" looks to me like it is copying Van Gogh's "Cafe Terrace at Night" https://en.wikipedia.org/wiki/Caf%C3%A9_Terrace_at_Night
Yes, that's what I was thinking.
I don't buy it TBH- the subject matter is very similar but the actual artwork is pretty different. For example if you zoom in on any little square inch of one painting and try to match it to the other it's not a very close match, which seems to me to make calling it a "copy" very suspect.
I'm no kind of expert, but those look pretty different to me. The AI one has no people, the sky is very different, the trees are very different, the lighting is quite different, the overall coloration is pretty different, the AI one seems to have a higher level of detail on everything except the street surface. No individual element of the scene looks very similar if I compare them.
They are both outdoor dining areas beneath an awning on a street at night, but surely there are many pictures that meet that description? I could see how somehow who was familiar with the human one might be reminded of it by the AI one, but what makes you consider this "an altered copy" instead of just "similar subject matter in a vaguely similar style"?
It looks so much like that painting that I voted it human on the assumption that I'd remembered the real painting as having more vivid colour than the above, or possibly that Van Gogh painted the same scene twice
Beat me to it. I posted that before seeing your post.
They're certainly copying the style of those artists- though not so much the specific artist as sort of a blend of impressionist artists in general.
Unlike the human artists, the AI isn't really capable of inventing new visual styles to convey semantic information. If you brought Van Gogh into the present and asked him to paint a shopping mall, he would definitely come up with a new twist on his visual language to convey the vibe of the place- wheres an AI copying the style is going to shoehorn in stylizations that were intended to emphasize things Van Gogh found interesting about 19th century France, and it's going to look awkward and semantically incoherent.
So, the AI is copying the most important part of the impressionist images- repeating the same visual insights about the same subjects, rather than saying anything new about reality. That said, this is the sort of copying that human artists do pretty frequently- not often to the degree that these models do when prompted to replicate styles, but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
And that is also something that can be mitigated a bit with the right kind of prompting and careful selection- letting the human inject a bit of original semantic content into the images. Once we have really strong multimodal models that are able to use the same internal world model acquired from language and audio and video and so on to images, we may also see that lack of original semantic content start to fade.
It depends on the prompt. If you tell an AI to imitate van Gogh's old pictures and paint a shopping mall, it'll do that.
If you tell it to innovate and capture the vibe, it'll try that, too.
>but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
My immediate thought when Scott commented that the image with the huge cats in the throne room represented painting skill a lot of human artists would envy, was precisely something like this: One of the main tells of AI art is that the images would have taken immense skill and effort to paint by hand, but the subject matter is horribly gauche. It is difficult to believe a human would be able to achieve that level of skill without also acquiring some taste, and that they would be willing to spend weeks creating a painting with nothing deeper to say than "wouldn't it be cool if some imaginary queen had gigantic cats as guardians?".
Incidentally, this is why warhammer illustrations are easy to mistake for AI. (Illustrations as a genre, really)
The opposite end of the spectrum are paintings made by someone with little to no technical skill, but a desperate need to convey some message. That's a very human thing. Hence Angry Crosses was the image I was most confidently wrong about.
On that note, I was surprised to read The Mourning of Christ was one of the images people tended to get wrong, as that was the one I most confidently marked as human, because the subject is clearly a corpse slumped in a way no living human would contort, and so I would not expect AI to depict that kind of pose.
This was my thought... I found this painting which seems very similar to 44: https://www.1stdibs.com/art/paintings/figurative-paintings/marda-paris-rain-large-french-school-post-impressionist-oil/id-a_5671391/ and it also strongly reminds me of a painting I wrote about for a class in college but can't find at the moment.
I thought that cafe at night scene was AI "in the style of" Van Gogh. Specifically his "Terrace at Night."
A picture of an oil painting is not the oil painting. Ceci c’nest pas un pipe and all that shit. 🐸
Do you think the results of this test would be any different if some high quality printouts of the AI generated oil paintings were put in frames next to the real things?
Very obviously yes? Any real oil painting has a lot more than 1024x1536 pixels (or whatever).
Imagine it upscaled to enough pixels that the human eye wouldn't be able to tell then 😉
I guess the real thing I'm trying to get at is: what it is about the physical medium of an oil painting that's important, and is there something fundamental about it that current AI technology couldn't replicate in a way that would fool people?
I know this is obtuse by your standard—but FYI, even a very high-quality, high-resolution Giclée is easily distinguished from an oil painting in person. You can try this experiment at an art museum if you like.
I think you're probably right. But is the difference important, really? I find it hard to believe the texture and brushtrokes of an oil painting are significantly enjoyable to people, especially given you can't touch paintings in an art gallery. Perhaps there's value in those features to people in the sense that they are a mark of human hands - but is there more to it than that?
As you've observed, I know almost nothing about art, so I would be interested in the perspective of someone who does.
https://upload.wikimedia.org/wikipedia/commons/b/bd/Rembrandt_van_Rijn_-_Self-Portrait_-_Google_Art_Project.jpg
This self portrait is scanned at a resolution where you can see the brush strokes. It looks incredible on my massive work monitor. Wikipedia has more examples of painting scanned at pretty high res.
It's not just about the end result - a big part of art is the artistic process itself. It's most visible with unconventional mediums - matchstick sculptures, life-size lego structures, patterns in flower gardens, etc. In looks alone, they will virtually always look worse than traditional clay sculpture or a traditional painting with similar time spent on it, just because it's so much harder to work with. But it's because it's so hard to work with that people find it so impressive. To this day many people don't recognize digital art as real art - and it's almost entirely due to how much easier it is (try to Ctrl+Z a brush stroke). AI generators have brought down the effort required basically to zero - and so, following the rule of inverse proportion, most people's threshold to be impressed by AI art shot up pretty much to infinity.
Brushstrokes have a visceral effect on some observers. That's also why it can be so different to see a painting in person rather than reproduced in a book: the tangible qualities of it (including the scale, color, visible marks) are all a direct communication that can get lost in a printed or digital image of the work.
The absence of brush strokes in the AI art would be a dead giveaway.
If you look closely, wouldn't you be able to see the texture of the paint rising off the canvas?
An oil painting is a 3D object. It is easily distinguishable from any printout, regardless of pixel resolution.
You could produce an actual oil paining from the AI's directions.
There are services that you can send a picture to, and they pay some people to produce an actual oil painting.
Or if you want to be a purist: I'm fairly sure you could build a robot that produces oil paintings, too.
Happened lately. First robot painted AI-art sold on auction for a million: https://www.bbc.com/news/articles/cpqdvz4w45wo
Another option could be to train the ai to generate a normal map and/or depth map alongside the image for the 3D effect
Oil paintings have 3D texture. Printouts do not. So the comparable would be a robot powered by AI painting with oil on a canvas. Comparing digital images is flattening the artistic expression of the paintings.
I’m not taking away from the power of gen AI but it’s a tool. The images are generated by talented people who can prompt the tool because not everyone can get gen AI to create such well composed images. Just as some oil painters create masterpieces while I am a step above finger painting.
Or possibly an AI making a 3D version with simulated brush strokes. Live it up, include those little cracks for what's supposed to be an older painting.
You don't even need to simulate the brushstrokes: you could have a robot wield an actual brush and do the AI's bidding.
It seems like an obvious next step is to merge a powerful prompt driven image generator with robotically manipulated traditional brushes, paints, canvas, etc., to see what caliber of wall art (down to the textured dimension of the actual brushstrokes) it can produce.
While correct, that's only relevant if you think there's something important about the medium. Or if your main concern is detecting that it wasn't by an AI.
Medium is important, especially if we are talking Turing Test type conclusion on fooling the human eye.
Is your concern with identification of artist or of quality? I didn't read this as a test of "fooling the eye", but rather of producing "quality work". If it's about "fooling the eye", then I consider it truly trivial, especially given the preprocessing. If your concern is with the quality of the work, then I don't see the medium as that important (though some media are better for some presentations).
I wonder about the version of this that doesn't require the test take place on a computer display? I believe in the transmission of art, the experience that comes from interacting with it with full attention. Context is important: seeing art in a gallery, often within a sterilized space and wedged between dozens of other pieces, is a particular kind of experience, and art doesn't often survive it. If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
In my experience, most people I know seem to 'take in information,' which is a very different act than experiencing. A different test (?) involves noting how individuals (or cultures?) are changed by artworks.
I notice I'm not writing in my voice. The couple years I spent in San Francisco I had to learn another way of speaking, which was oriented toward precision, a constant clarification. I'm defensively tending toward that voice in this post, when what I'd rather do is sing. Somewhere under all the apprehension in the world is an understanding, and under that is awe, and the art made with loving attention rises to meet me there.
Bravo
> If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
That was how I felt too, until I saw a Rothko exhibit. Being able to get really close to the paintings, spending time with them, the quietness, dim lights, the sheer size of the paintings and Rothko's style itself gave me way more "art appreciation bandwidth". Or maybe I'm getting older.
I suspect that the rise of AI art will get a lot of people going to museums again to experience physicsl art in person. Over the past few decades, we had gotten used to seeing computer images of physical art, rather than the real thing. But the colors and textures just aren’t the same, even though the visual sensation is quite close.
Not that purely digital art isn’t good - it’s just a different thing.
The color gamuts of most people's displays aren't quite as wide as reality. This will be fixed soon.
I'm not sure about the 'soon' part. We had to wait an awful long time until resolutions got better, too.
I’m guessing that some portion of new physical art hanging in galleries will be based off of AI-generated images. The next evolution of paintings from sketches and photographs. I think it’s kind of exciting.
Yes. Seeing a painting in person is a totally different experience than on a screen or in a book. I knew Sargent's famous portrait of Henry James but viewing it at The National Gallery in London, I discovered the brush strokes and texture which are flattened by reproduction.
>Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
I'm not one of these people, but I *do* consider some of history's "greatest" human artists as mostly producing hideous slop, however well made and well executed and far beyond my complete lack of artistic ability. Not sure where that puts me.
Right. I did fairly well on the test, and one of the heuristics I used successfully was identifying hideous pieces as human, thinking that few people would willingly generate hideous AI art unless it was a deliberate curve ball to throw judges off in this contest, while humans have at least one clear motive to dabble in hideous styles: being bored with the aesthetically pleasing ones. See: architecture discussion on this blog.
Art I like is good, art I don't like is slop, Ai art is slop. I'm not surprised I had trouble distinguishing between human slop (that skull painting etc.) and Ai
There's plenty of art I don't like that I can respect as being real or good art, as demonstrating an impressive set of skills, or as producing something people in general judge as good or valuable in a way I value indirectly.
Human art one dislikes might look superficially like slop, but it usually isn’t slop. “Slop” is a bunch of stuff with no reason to it (see the discussion of the blue and red arch in the OP) while the human art is usually getting at something, even if you don’t get it.
Yeah, I think a lot of the human art here is aimed at a very specific audience with a lot of context and background knowledge and not remotely meant to appeal to the general public. That kind of expert-oriented art would seem to be especially hard for AI to match, but only experts will be able to tell that. Meanwhile a lot of AI art looks great to a mass audience.
Notably, this kind of self-referential appeal-only-to-novelty-starved-experts creation appears to be how architecture went from things that are nice to look at to things that are ugly to look at, basically because the field was allowed to judge itself and architects began catering to the tastes of other architects rather than anyone else.
It seems like a broadly unhealthy tendency to indulge.
(Thinking in particular of Item #46 here and the related "Whither Tartaria" post)
https://www.astralcodexten.com/p/links-for-november-2024
I think that's what AI art is optimized for -- looking great to a mass audience.
No, I'm pretty confident that million dollar skull painting is slop
Basquiat is basically an elaborate hoax played by the art world on everyone else to see what they could get away with.
Maybe. Though at least it seems to 'say' something?
Also, many of these great artists are great because they invented a movement, or captured a historically important moment in time. Not necessarily because the paintings themselves are wonderful.
I think AI is very good at art with a medium amount of detail (such as impressionism), but not good at art with a high amount of detail, (such as a realistic battle scene), or a low amount of detail (such as pixel art). I think what the low and high detail art have in common is that you really need to know what you're looking at. With medium level detail you can kinda get away with not knowing what a hand is, whereas with high level detail (e.g. battle scene) you really need to know the number of fingers and in what order they are, and with low level detail (e.g. pixel art) you need to be able to represent the whole thing with a couple of well placed pixels while having it still readable as a hand.
If you had asked me before generative AI I would've guessed that it would be linear (e.g. best at pixels, then medium, then high detail), because I assumed we'd make AI focused on understanding things, with it becoming more difficult to understand things the more complex they became. Instead we focused on imitation, which succeeds more if it can fake understanding, and thus at mid-level detail.
I think we can probably do good AI pixel art, but the general purpose AI picture generators we have aren't the right tool for that particular job (yet?).
>or a low amount of detail (such as pixel art)
I think pixel art would actually fall under "high detail" here. To pass as real pixel art, every single pixel needs to be correctly aligned to the grid. Current AI image generators are not very good at that (but they are getting better.)
It should be extremely easy (at least compared to training the AI in the first place) to add the pixelation as a post-processing step.
AI artists don’t pass the test, or if they do, this doesn’t show that. AI artists plus skilled human selection is what passed this test.
I think what people are talking about when they say they don’t like AI art is AI art plus unskilled human selection, which also was not tested here.
All art is selected/curated. If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles? To a teenager drawing poo on a bathroom stall? To one of the great human artists, but at the start of their journey when they still couldn't draw well?
If we take the curation efforts into account, I'd go as far as say that AI is even more impressive. Scott just took extremely famous paintings, basically the pinnacle of human art. And compared to mostly art generated by some dude that reads his blog.
>If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles?
That isn't quite right, no.
The AI model is trained on an enormous corpus of human art and then instructed on imitating it (without replicating it too closely). It is a tool designed from the ground up to be the best artist it can be, with the best education it could be given. A 3 year old with a crayon has been trained on nothing and has experience of nothing.
The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
I was arguing against OP's point that AI art "won" here because it was curated while human was not. (that's what I understood, correct me if I'm wrong)
> The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
This is still curation, just less granular. We're selecting on the artist instead of the artwork level. Is this the correct comparison? _Really_ depends on what are you trying to measure.
But just the fact that we're saying "known-skilled artists" is the bar you need to clear to fairly compare to AI is telling.
The thrust of my argument was meant to be that since the AI side of the test involved heavy human curation, it's not really fair to call it a Turing test.
But I think you make a good point, that both sides were curated by the same person. Considering that, I think it's a fair test. But I don't think it's a Turing test.
I haven't read Turing himself, but the idea of a Turing test is that you expose the AI to a person through some channel that obfuscates the source of the communication. Beyond that identity obfuscation, both sides interact naturally. (With "nature" in quotes for the AI.)
But this test introduces a skilled human curator between each side and the judges. So it's fair, and it's telling us something, but it's not telling us what "winning a Turing test" would tell us.
The real question is about what work the AI can do in the world. Can it replace an artist? And the answer is yes, as long as somebody like Scott puts time into curation. Since the human artists already curate their own art (at least to the extent of making sure there are the right number of fingers & the written words are real words) they still have a pretty good advantage.
Although that advantage is probably mostly undone by the AI's incredible speed...
The art was curated by the same person, but for backwards reasons. AI art was chosen to imitate human art and human art was chosen to imitate AI art. Scott was intentionally making it hard to tell which was which. Completely un-curated would have been too easy, as the AI art would have been much more obvious (like the few Scott put in that were very obvious).
The art was also actually double-curated, since Scott wasn't just putting prompts into AI engines but was accepting submissions for "good" AI art.
And I think it's important to remember people still got it right above coinflip rates.
Exactly! If you were to take a bunch of uncurated human art and ask people, "is this human or AI?" you wouldn't expect them to have a difficult time determining that it was human - even if it was full of a bunch of crap.
If the question were which looks better when curated and stripped of context, this would be a good test. But the question is, "can AI art fool humans into believing it's not AI-generated?" and in curating the dataset to obfuscate which is which, Scott intentionally excluded all the tells an AI artist isn't able to replicate. At best, you get a narrow version of the question, "Can certain AI images pass for human when we heavily select for the kinds of things computers do well at?"
I think it's impressive that the AI side did as well as it did on even this heavily biased test. Just 5 years ago, nobody would have believed it. Given certain trends, I expect 5 years from now an AI will be able to drop many of the heavy caveats/redactions/biases built into this test and do well in a head-to-head competition with history's greatest painters.
This isn't the progression people expect, though. They expect AI art to do as well as a 5th grader first, then get as good as a college student, etc. But they're going through the same type of training, so instead of taking the traditional path to developing their skills, they'll go through the Uncanny Valley on their way to excellence.
It’s worth reading the Turing paper: https://academic.oup.com/mind/article/LIX/236/433/986238
He doesn’t actually claim that a 30% rate of fooling humans counts as passing - he just predicts that by the year 2000, there would be computers able to fool humans 30% of the time in a five minute conversation.
What he actually proposes is more sophisticated. It’s really much more like, if you could have extended interactions, and come away with the same sort of value that you do from equivalent human interactions, then we might as well say it’s as good as a human. In at least some parts of the paper, he takes seriously the idea that not being able to enjoy strawberries and cream is a potential problem, because shared enjoyment is part of what we look for in human friends.
But I think the really surprising parts are how well he predicts the capacities of computers decades in advance (a decade or so before Gordon Moore invents his eponymous law even), the fact that he considers ESP to be so well established that it might break his test, and the suggestion he makes at the end that some sort of machine learning is likely to be a more effective approach than directly programming a conversational AI.
Yes, I definitively would recommend reading Turing's original paper. It also anticipates and answers many of the objections people still bring up to this day.
Yes, calling this a Turing test is, at best, a metaphor. I tend to think of it as abuse of the language, but then I tend to be picky in some areas.
Agree. I see a lot of stuff on the internet saying the Turing test has been passed and when you read the actual details they did not do an actual Turing test as he described it in his paper. I contend that no computer has yet passed an actual Turing test, mostly because no one runs an actual Turing test.
You should read Turing's original paper. It's quite short and easy to understand for the layman.
He explores some great nuances and also anticipates most of the objections people have later brought up. (Mostly people who vaguely heard of his test, but haven't read the paper.)
Eg Turing's original test is adversarial.
I think "average sloppy AI art" versus "3 year old's scribbles" would really, really not pass a Turing test. Would be super obvious which is which.
Sure. But the Turing test is trying to test capabilities. Being indistinguishable from a human implies being equally (or more) capable.
The assumption is that if AI can generate art that requires masterful skill, it also can generate anything below that.
If we prompted the AI for a "3 year old's scribbles" contest, would it be indistinguishable from a 3 year old?
I dunno! I don't think anyone ever tried this. Could be that today's models can't do this because it wasn't trained to.
But that would be because of lack of trying, not a fundamental limitation in the tech.
This is an interesting question! I tried it out on Gemini and canva.com, and doing so has confirmed my original opinion. You should try it, too.
>All art is selected/curated.
The problem is that if you're comparing human art to AI art, you don't want the AI art to benefit from human curation.
Imagine a chess-playing bot that has a human grandmaster correcting its moves whenever it fumbles: it'd be hard to claim that it's playing "AI chess" at that point.
There is a separate side question of "how does a human+AI system perform against an all-AI system or an all-human system".
Scott is great, but he's hardly a grandmaster at selecting art.
To steal Matthew Talamini's analogy below, I think it's more like a blind taste-test than a game of chess.
If we decide to go with the chess metaphor, though, it's more like "we'll select the best AI player and the best human player to play a match", or perhaps "we will select the best 10 AI-vs-human games and the best 10 human-vs-human games to look at"—a human "correcting" *every move* is more like digital art, i.e. a computer doing what a human commands.
There is no input from the AI in such a scenario, which seems clearly distinct from this. The level at which it is curated isn't every move nor every pixel, but complete works / complete games.
From my point of view, a 3 year old's scribbles are more interesting because they have intention behind them. If you've ever asked a kid that age what they're drawing, they're often making a picture of something very specific – it's just that it's wildly abstract. I'm not saying that makes it delightful to look at, but it does make it interesting! Whereas AI can sometimes be exciting to look at, but fundamentally lacks intention. I mean the human prompt is there, but there's a disconnect between that prompt and what's visually created. The place where a conscious mind would intentionally organize the visual around the idea is just missing.
Also for what it's worth these paintings are mostly B Sides and I think it would be hard to argue that they really represent the pinnacle of human artistic achievement.
If "intention" was something intrinsic to the art itself, there should be no difficulty telling AI and human art apart.
I.e., it's fine to value it, but it is like valuing the state of mind a chef was in when cooking a meal: "it's not interesting to me to eat it" and "something is different about the meal itself" ought be distinguished.
Maybe intention is the wrong word. But human art does have consciousness behind it. AI art doesn't. I personally don't have much trouble distinguishing AI art and human art apart – for why, see Ilzo's comments in the original post. If you take the stance that these add up to "small inadequacies" then I guess you might think AI art carries as much or little meaning as human art, but what those inadequacies do is betray a lack of consciousness, which means that the whole project is meaningless (and on an aesthetic level, it just falls apart and is actively grating to look at).
I don't like the food comparison (cooking can be an art, but eating itself is a survival need, so not a 1:1 correlation). But if I were to hold with your metaphor, I'd be saying that something does taste different about the visual "meal" AI creates, and it tastes off.
It would be very easy to distinguish AI generated images from a child's scribbles, because AI can imitate the most skilled human artists but has no intention, while a child has very little skill but a clear intention.
I think what you're looking to account for here is the presence of taste. But a taste filter has been applied to art history (what survived, which are well known) as well as the AI competitors. It would be interesting to see this test designed and run again by Scott's artist friend.
What would be the right standard for a purely AI competitor though? Doesn't seem like a simple answer. You can argue the prompting process itself reflects human input, plus does the AI get a chance to review it's own work and decide itself what's most likely to pass the test?
Maybe giving a set of human artists and a set of AI artists identical prompts? - I think this has flaws, as well as being obviously challenging to set up.
Maybe an AI has to set the prompts and select the images itself with only some initial prompting about the nature of the test?
The test feels much more legitimate to me when I consider it as a kind of blind taste test.
If you want to prove that California can produce good wine, a blind taste test versus French wine is really useful. If a lot of people have the idea that only French wine is any good, a fair blind taste test between the curated best of California and the curated best of France is absolutely called for. It can't establish that California wine can replace French wine, or that people generally won't be able to tell the difference. But it's still useful.
Same with AI. Anybody who has the idea that only human-made art is any good should update based on the results of this test. It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful.
Yeah this is a good way of thinking about it.
You can for sure demonstrate that AI is capable of being used to produce some kind of image that has some kind of broadly defined aesthetic value. The really hardline anti AI art crowd should dial it down a notch.
It's not an argument for AI maximalism though - human artists remain vastly superior to AI artists on almost every possible metric.
AI artists are much cheaper and faster and accessible.
Yeah, I think we're agreement here. And it's probably the source of our original disagreement.
You seem to understand the "proper" Turing test better than me. So I guess that taking that view this is not a valid test is correct.
But the "popular" intepretation of the Turing test is more like a taste or capabilities test. Can AI generate art that's as good as a human?
> It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful
I think all of these statements need an explicit "for now" appended. AI art is advancing super quick. Most of these issues Scott curated out, like messed up hands and text won't be an issue soon. In fact, the flux models that came out recently basically solved text already.
> It doesn't show that AI artists can replace human artists
I'd expand on this. A _lot_ of work human artists do will be replaced. Most of corporate soulless art will be automated. Generating portraits of book characters, etc.
There will always be a market for human art, but will be linked to prestige. More than it is today.
It also will never replace the art that's more than "pretty picture". Art is more than the physical thing itself for many people. If who created it matters, then that kind of art is impossible to automate by definition.
Note that what they're comparing it against isn't human art, but human art plus human selection. I think if you're trying to find the more aesthetically pleasing images between AI slop and human slop, the AI slop would win handily.
> I think what people are talking about when they say they don’t like AI art is AI art plus unskilled human selection, which also was not tested here.
I think a lot of people just think AI art is inherently bad. But I agree that there's a big problem of people posting AI slop instead of going to the extra effort and just posting particularly interesting ones, or using all the methods they have at their disposal to add artistic control to the result.
I think one of the things I like about AI art is that in order to make it stand out you have to make more interesting stuff, whereas for a human you can basically paint a bowl of fruit and if you do a really good job people are impressed. Then again, that's exactly why I thought the Victorian Megaship was AI, so maybe I'm way off base with that and there's plenty of human artists that do the kind of thing I'm interested in.
I feel like the art chosen does skew the results fairly significantly. I am not an art expert, I expected it to be very hard, and I got about 60%. However, not only was the AI art chosen to avoid obvious tells, I think there was extra selection to avoid many subtle tells.
It felt like human-made computer generated had more detail than average, and the very simplest looking images were more likely to be AI. On impressionism, there were no actual images by Van Gogh or Monet, who people really like, and so it didn't surprise me that people rated the AI mimicry of those artists very highly. More people like Van Gogh's style than like Gauguin and the village of Osny (also good).
I actually left the exercise feeling that it might be easier to differentiate AI art than I thought, not harder. But my bar was very low.
no dragon lady attribution
re why cat court is AI, lots of detail = AI works there. I guess it failed us on the human drawn ship. But if you add in topic like silly and super detailed is surely AI
All the cats have the same expression, displaying a transitory emotion; for all of them to share it is both unnatural, and boring for a human to recreate.
It's a really weird composition, too. Why is the king cat's body hidden in some huge alcove while the queen cat's body sprawls across the floor? The small orange cat is sitting on nothing, next to a stool. The tabby cat at the right is standing on two legs and doesn't have front legs at all. The curtains are pulled very asymmetrically. Where are the lady's legs?
The painting also fails to tell a story, which you'd think the artist would have tried to do if they're going to put so much effort into such a silly painting. Who actually has the power here, the lady or the cat? They both have identical crowns. The lady is in a central more powerful position but she's clearly dominated by the cat behind her. Is this some kind of formal occasion in the throne room? Some characters seem to think it is, while others are just casually sprawled around.
>I guess it failed us on the human drawn ship
I was surprised so many people missed the ship. No current AI model can render such complex, coherent rigging and ratlines without at least some ropes blurring together or bending. Seems like it's obviously a 3D render.
I think the main tell for me for Cat Court is that it seems too glossy, in a way that's very common in AI art but rare in human art of that style. Just about everything that's brightly-lit seems shiny, even if it clearly shouldn't be by virtue of having a rough or fluffy texture.
The same kind of glossy feel seems to happen by default in 3D-rendered artwork, unless you do a lot of worth with shading and textures to get a different effect. Come to think of it, that might be part of why human-made digital images were often incorrectly identified as AI.
Looking at it more closely, an additional tell is that the style of the candle flames are inconsistent with one another and with the overall style of the image. The ones on the left candelabra are fairly normal, but the ones on the center and right candelabras seem to have lens flare effects.
oop the attribution was there. I was just waiting for headpats for putting it as most confident ai
The Giant Ship appears to be a 3D render. I guessed correctly that it was human because the details looked too coherent to be AI.
I've been looking forward to this. It was interesting to hear how many anti-AI people liked the AI Art.
count me as an anti-AI person specifically _because_ I think the AI art is excellent. But that's just because I think artists are important parts of society and I don't want them to disappear
What's the point of an artist that can't produce anything of value? The same applies to any other worker. Humanity will be made obsolete in due time. You can't build something new without destroying what came before.
Call me naïve, but I prefer a society where talented people don’t feel the crippling despair of having nothing of value to contribute anymore.
Well, the good news is that those feelings won't last for very long!
LOL! Could be true...
Your first reply said they were "important parts of society"; now it seems like you're arguing the problem is how they would feel about it, rather than how the rest of civilization would get on without them? Those seem to me like very different, almost diametrically-opposed arguments.
What do you mean by “without them” exactly?
The normal way that I evaluate whether something is an "important part of a system" (as you suggested) is by asking what would happen to the system if the part were absent or non-functional. For example, if I want to know how important a doorknob is, I ask what would happen if there were no doorknob, or if the doorknob stopped working. This strikes me as the obvious comparison invited by your turn of phrase.
It doesn't particularly matter what I meant by the phrase, though, because I'm paraphrasing YOUR assertions back to you in order to check my understanding of them. Even if what I said is ambiguous, clarifying it wouldn't particularly help, because the entire point is to figure out what YOU meant; once your position is clarified, my attempts to paraphrase it are dispensable.
And you notably did not provide any clarification of your own position.
To me, your question looks suspiciously like you were trying to insinuate that maybe I'm in favor of killing people. Which would be totally false, and not remotely required by the counterfactual comparison (which just requires that society stops utilizing their art), and not my fault even if it WERE implied (since I'm just discussing the meaning of YOUR assertion), and a brazen evasion of my question about what YOU meant. If that was what you were doing.
I don't think AI art should replace human art (because I don't think AI art looks good or holds much value), but I also think that even if it did human artists would still have plenty of value to contribute. Art isn't just making pictures (or sculptures or novels or whatever), it's making meaning through every element of a picture. Unless it becomes conscious (instead of just being an algorithm stuffed into a fancy sweater), it won't be able to do that. Of course, a human artist could work with it to make something that makes meaning, but I tend to find that AI fundamental lack of meaning interferes with a meaningful output. Idk, I have yet to see an interesting collaboration between human and AI, but that doesn't mean that it couldn't happen.
As a human, I don't want humanity to be made obsolete