The bigger tell on the Ominous Ruin painting is that the perspective on the right-side cliff is all wrong, and it looks like a foreground when it is supposed to be background.
If BOTH cliffs were trippy like that, we could have assumed that it was done on purpose to give the gate a trippy Lovecraftian feel, but when it's just one it just looks like a weird perspective mistake that no human would make.
The child in the Mother and Child image, however, has an extra thumb or some kind of appendage, sticking out of the back of his thigh, which is partly why I rated that one as AI.
My immediate reaction was "Scott included a lot of AI images with less-ambiguous errors than that" but my guess is that Scott just missed the other errors; they were all fairly subtle.
For me, the "AI tell" is the fussy, overly complicated earring, which doesn't make visual sense and clashes with the peasant-style kerchief on the woman's head.
its usually overcomplicated in a way that a human artist would have no patience for. Like say, if you prompted both an AI and a human artist to draw a building with 1000 windows, AI would literally draw 1000 windows, but they would make no sense in any kind of architecture or even geometry, while a human artist would simply draw 200 windows correctly, and bluringly imply the rest. AI never bullshits the viewer, but senselessly tries to fulfill the prompt against common sense.
This was a fun exercise! And it is slightly instructive. But not more than that: the popular impressionist "AI" paintings are just altered copies of actual impressionist artists, e.g. Van Gogh or Renoir.
I'm not the person you asked, but I see a lot of Edouard Henri Leon Cortès in "44: Paris Scene", particularly "Porte Saint Denis" and some other unnamed paintings of his. I also see some Antoine Blanchard, although the "blur" is more evident in the AI painting than in any of these two artists' work. I was fooled!
"11: Green Hills" is basically Monet, you can see the 'inspiration' from his various Poppy Field and Meadow series.
I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here. It's more the style that is clearly ripped off, and applied to new scenes as described in the prompts.
Yeah, good examples! Thanks! It would be interesting to see the prompts on the AI imagery (or the process in general if it is more complicated than a single prompt)
You can have a lot of fun with Midjourney, I got this (https://imgur.com/a/nzqcom8) from the following prompt: "a defeated Jeanne d'Arc in a wheat field contemplates while a castle burns in the distance, Viktor Vasnetsov". If you're more descriptive and use more tags as well as artist names, you can concoct stuff that would fool.. well.. about half of the people in here apparently!
Note: I did not share that image to suggest it's a great one, but rather to show how easily you can generate something that would look pretty convincing when not zoomed in.
>I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here.
On a related note: I generated images of "kitten pouncing on owner's toes" with DALL-E, and then tried searching for the image with Google Lens aka image search, and I don't find anything with e.g. the kitten's paws and owner's toes in the same relative position. So, even in this case (there are a _lot_ of kitten pictures on the net), this doesn't look like retrieving a memorized picture.
The AI version made a mistake of giving Paris left-sided traffic and a London-style red bus. Plus the cyclist rides a contraption that looks definitely more like a modern rickshaw than a period-appropriate bicycle.
AI got amazingly good at fixing common stylistic or logical errors in its art, but it does not understand the context, so most "historical" AI art is an anachronistic mishmash spiked with weird pseudohistorical gizmos.
I tried to produce several dozen of Classic style painting of battles, and while the AI got the style right, it make absolute hash out of weapons and armor, because it did not understand what they were for, so the warring knights end up looking like LOTR extras doing mutual surgery with farmtools rather than fighting with weapons.
I don't buy it TBH- the subject matter is very similar but the actual artwork is pretty different. For example if you zoom in on any little square inch of one painting and try to match it to the other it's not a very close match, which seems to me to make calling it a "copy" very suspect.
I'm no kind of expert, but those look pretty different to me. The AI one has no people, the sky is very different, the trees are very different, the lighting is quite different, the overall coloration is pretty different, the AI one seems to have a higher level of detail on everything except the street surface. No individual element of the scene looks very similar if I compare them.
They are both outdoor dining areas beneath an awning on a street at night, but surely there are many pictures that meet that description? I could see how somehow who was familiar with the human one might be reminded of it by the AI one, but what makes you consider this "an altered copy" instead of just "similar subject matter in a vaguely similar style"?
It looks so much like that painting that I voted it human on the assumption that I'd remembered the real painting as having more vivid colour than the above, or possibly that Van Gogh painted the same scene twice
They're certainly copying the style of those artists- though not so much the specific artist as sort of a blend of impressionist artists in general.
Unlike the human artists, the AI isn't really capable of inventing new visual styles to convey semantic information. If you brought Van Gogh into the present and asked him to paint a shopping mall, he would definitely come up with a new twist on his visual language to convey the vibe of the place- wheres an AI copying the style is going to shoehorn in stylizations that were intended to emphasize things Van Gogh found interesting about 19th century France, and it's going to look awkward and semantically incoherent.
So, the AI is copying the most important part of the impressionist images- repeating the same visual insights about the same subjects, rather than saying anything new about reality. That said, this is the sort of copying that human artists do pretty frequently- not often to the degree that these models do when prompted to replicate styles, but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
And that is also something that can be mitigated a bit with the right kind of prompting and careful selection- letting the human inject a bit of original semantic content into the images. Once we have really strong multimodal models that are able to use the same internal world model acquired from language and audio and video and so on to images, we may also see that lack of original semantic content start to fade.
>but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
My immediate thought when Scott commented that the image with the huge cats in the throne room represented painting skill a lot of human artists would envy, was precisely something like this: One of the main tells of AI art is that the images would have taken immense skill and effort to paint by hand, but the subject matter is horribly gauche. It is difficult to believe a human would be able to achieve that level of skill without also acquiring some taste, and that they would be willing to spend weeks creating a painting with nothing deeper to say than "wouldn't it be cool if some imaginary queen had gigantic cats as guardians?".
Incidentally, this is why warhammer illustrations are easy to mistake for AI. (Illustrations as a genre, really)
The opposite end of the spectrum are paintings made by someone with little to no technical skill, but a desperate need to convey some message. That's a very human thing. Hence Angry Crosses was the image I was most confidently wrong about.
On that note, I was surprised to read The Mourning of Christ was one of the images people tended to get wrong, as that was the one I most confidently marked as human, because the subject is clearly a corpse slumped in a way no living human would contort, and so I would not expect AI to depict that kind of pose.
Do you think the results of this test would be any different if some high quality printouts of the AI generated oil paintings were put in frames next to the real things?
Imagine it upscaled to enough pixels that the human eye wouldn't be able to tell then 😉
I guess the real thing I'm trying to get at is: what it is about the physical medium of an oil painting that's important, and is there something fundamental about it that current AI technology couldn't replicate in a way that would fool people?
I know this is obtuse by your standard—but FYI, even a very high-quality, high-resolution Giclée is easily distinguished from an oil painting in person. You can try this experiment at an art museum if you like.
I think you're probably right. But is the difference important, really? I find it hard to believe the texture and brushtrokes of an oil painting are significantly enjoyable to people, especially given you can't touch paintings in an art gallery. Perhaps there's value in those features to people in the sense that they are a mark of human hands - but is there more to it than that?
As you've observed, I know almost nothing about art, so I would be interested in the perspective of someone who does.
This self portrait is scanned at a resolution where you can see the brush strokes. It looks incredible on my massive work monitor. Wikipedia has more examples of painting scanned at pretty high res.
It's not just about the end result - a big part of art is the artistic process itself. It's most visible with unconventional mediums - matchstick sculptures, life-size lego structures, patterns in flower gardens, etc. In looks alone, they will virtually always look worse than traditional clay sculpture or a traditional painting with similar time spent on it, just because it's so much harder to work with. But it's because it's so hard to work with that people find it so impressive. To this day many people don't recognize digital art as real art - and it's almost entirely due to how much easier it is (try to Ctrl+Z a brush stroke). AI generators have brought down the effort required basically to zero - and so, following the rule of inverse proportion, most people's threshold to be impressed by AI art shot up pretty much to infinity.
Brushstrokes have a visceral effect on some observers. That's also why it can be so different to see a painting in person rather than reproduced in a book: the tangible qualities of it (including the scale, color, visible marks) are all a direct communication that can get lost in a printed or digital image of the work.
Oil paintings have 3D texture. Printouts do not. So the comparable would be a robot powered by AI painting with oil on a canvas. Comparing digital images is flattening the artistic expression of the paintings.
I’m not taking away from the power of gen AI but it’s a tool. The images are generated by talented people who can prompt the tool because not everyone can get gen AI to create such well composed images. Just as some oil painters create masterpieces while I am a step above finger painting.
Or possibly an AI making a 3D version with simulated brush strokes. Live it up, include those little cracks for what's supposed to be an older painting.
It seems like an obvious next step is to merge a powerful prompt driven image generator with robotically manipulated traditional brushes, paints, canvas, etc., to see what caliber of wall art (down to the textured dimension of the actual brushstrokes) it can produce.
While correct, that's only relevant if you think there's something important about the medium. Or if your main concern is detecting that it wasn't by an AI.
Is your concern with identification of artist or of quality? I didn't read this as a test of "fooling the eye", but rather of producing "quality work". If it's about "fooling the eye", then I consider it truly trivial, especially given the preprocessing. If your concern is with the quality of the work, then I don't see the medium as that important (though some media are better for some presentations).
wait until we teach AI to actually paint a painting using a robotic hand. I'm pretty sure it's already possible, just not particularly important or profitable to do, except as a publicity stunt.
I wonder about the version of this that doesn't require the test take place on a computer display? I believe in the transmission of art, the experience that comes from interacting with it with full attention. Context is important: seeing art in a gallery, often within a sterilized space and wedged between dozens of other pieces, is a particular kind of experience, and art doesn't often survive it. If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
In my experience, most people I know seem to 'take in information,' which is a very different act than experiencing. A different test (?) involves noting how individuals (or cultures?) are changed by artworks.
I notice I'm not writing in my voice. The couple years I spent in San Francisco I had to learn another way of speaking, which was oriented toward precision, a constant clarification. I'm defensively tending toward that voice in this post, when what I'd rather do is sing. Somewhere under all the apprehension in the world is an understanding, and under that is awe, and the art made with loving attention rises to meet me there.
> If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
That was how I felt too, until I saw a Rothko exhibit. Being able to get really close to the paintings, spending time with them, the quietness, dim lights, the sheer size of the paintings and Rothko's style itself gave me way more "art appreciation bandwidth". Or maybe I'm getting older.
I suspect that the rise of AI art will get a lot of people going to museums again to experience physicsl art in person. Over the past few decades, we had gotten used to seeing computer images of physical art, rather than the real thing. But the colors and textures just aren’t the same, even though the visual sensation is quite close.
Not that purely digital art isn’t good - it’s just a different thing.
I’m guessing that some portion of new physical art hanging in galleries will be based off of AI-generated images. The next evolution of paintings from sketches and photographs. I think it’s kind of exciting.
Yes. Seeing a painting in person is a totally different experience than on a screen or in a book. I knew Sargent's famous portrait of Henry James but viewing it at The National Gallery in London, I discovered the brush strokes and texture which are flattened by reproduction.
>Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
I'm not one of these people, but I *do* consider some of history's "greatest" human artists as mostly producing hideous slop, however well made and well executed and far beyond my complete lack of artistic ability. Not sure where that puts me.
Right. I did fairly well on the test, and one of the heuristics I used successfully was identifying hideous pieces as human, thinking that few people would willingly generate hideous AI art unless it was a deliberate curve ball to throw judges off in this contest, while humans have at least one clear motive to dabble in hideous styles: being bored with the aesthetically pleasing ones. See: architecture discussion on this blog.
Art I like is good, art I don't like is slop, Ai art is slop. I'm not surprised I had trouble distinguishing between human slop (that skull painting etc.) and Ai
There's plenty of art I don't like that I can respect as being real or good art, as demonstrating an impressive set of skills, or as producing something people in general judge as good or valuable in a way I value indirectly.
Human art one dislikes might look superficially like slop, but it usually isn’t slop. “Slop” is a bunch of stuff with no reason to it (see the discussion of the blue and red arch in the OP) while the human art is usually getting at something, even if you don’t get it.
Yeah, I think a lot of the human art here is aimed at a very specific audience with a lot of context and background knowledge and not remotely meant to appeal to the general public. That kind of expert-oriented art would seem to be especially hard for AI to match, but only experts will be able to tell that. Meanwhile a lot of AI art looks great to a mass audience.
Notably, this kind of self-referential appeal-only-to-novelty-starved-experts creation appears to be how architecture went from things that are nice to look at to things that are ugly to look at, basically because the field was allowed to judge itself and architects began catering to the tastes of other architects rather than anyone else.
It seems like a broadly unhealthy tendency to indulge.
(Thinking in particular of Item #46 here and the related "Whither Tartaria" post)
Also, many of these great artists are great because they invented a movement, or captured a historically important moment in time. Not necessarily because the paintings themselves are wonderful.
I think AI is very good at art with a medium amount of detail (such as impressionism), but not good at art with a high amount of detail, (such as a realistic battle scene), or a low amount of detail (such as pixel art). I think what the low and high detail art have in common is that you really need to know what you're looking at. With medium level detail you can kinda get away with not knowing what a hand is, whereas with high level detail (e.g. battle scene) you really need to know the number of fingers and in what order they are, and with low level detail (e.g. pixel art) you need to be able to represent the whole thing with a couple of well placed pixels while having it still readable as a hand.
If you had asked me before generative AI I would've guessed that it would be linear (e.g. best at pixels, then medium, then high detail), because I assumed we'd make AI focused on understanding things, with it becoming more difficult to understand things the more complex they became. Instead we focused on imitation, which succeeds more if it can fake understanding, and thus at mid-level detail.
I think we can probably do good AI pixel art, but the general purpose AI picture generators we have aren't the right tool for that particular job (yet?).
I think pixel art would actually fall under "high detail" here. To pass as real pixel art, every single pixel needs to be correctly aligned to the grid. Current AI image generators are not very good at that (but they are getting better.)
All art is selected/curated. If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles? To a teenager drawing poo on a bathroom stall? To one of the great human artists, but at the start of their journey when they still couldn't draw well?
If we take the curation efforts into account, I'd go as far as say that AI is even more impressive. Scott just took extremely famous paintings, basically the pinnacle of human art. And compared to mostly art generated by some dude that reads his blog.
>If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles?
That isn't quite right, no.
The AI model is trained on an enormous corpus of human art and then instructed on imitating it (without replicating it too closely). It is a tool designed from the ground up to be the best artist it can be, with the best education it could be given. A 3 year old with a crayon has been trained on nothing and has experience of nothing.
The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
I was arguing against OP's point that AI art "won" here because it was curated while human was not. (that's what I understood, correct me if I'm wrong)
> The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
This is still curation, just less granular. We're selecting on the artist instead of the artwork level. Is this the correct comparison? _Really_ depends on what are you trying to measure.
But just the fact that we're saying "known-skilled artists" is the bar you need to clear to fairly compare to AI is telling.
The thrust of my argument was meant to be that since the AI side of the test involved heavy human curation, it's not really fair to call it a Turing test.
But I think you make a good point, that both sides were curated by the same person. Considering that, I think it's a fair test. But I don't think it's a Turing test.
I haven't read Turing himself, but the idea of a Turing test is that you expose the AI to a person through some channel that obfuscates the source of the communication. Beyond that identity obfuscation, both sides interact naturally. (With "nature" in quotes for the AI.)
But this test introduces a skilled human curator between each side and the judges. So it's fair, and it's telling us something, but it's not telling us what "winning a Turing test" would tell us.
The real question is about what work the AI can do in the world. Can it replace an artist? And the answer is yes, as long as somebody like Scott puts time into curation. Since the human artists already curate their own art (at least to the extent of making sure there are the right number of fingers & the written words are real words) they still have a pretty good advantage.
The art was curated by the same person, but for backwards reasons. AI art was chosen to imitate human art and human art was chosen to imitate AI art. Scott was intentionally making it hard to tell which was which. Completely un-curated would have been too easy, as the AI art would have been much more obvious (like the few Scott put in that were very obvious).
The art was also actually double-curated, since Scott wasn't just putting prompts into AI engines but was accepting submissions for "good" AI art.
Exactly! If you were to take a bunch of uncurated human art and ask people, "is this human or AI?" you wouldn't expect them to have a difficult time determining that it was human - even if it was full of a bunch of crap.
If the question were which looks better when curated and stripped of context, this would be a good test. But the question is, "can AI art fool humans into believing it's not AI-generated?" and in curating the dataset to obfuscate which is which, Scott intentionally excluded all the tells an AI artist isn't able to replicate. At best, you get a narrow version of the question, "Can certain AI images pass for human when we heavily select for the kinds of things computers do well at?"
I think it's impressive that the AI side did as well as it did on even this heavily biased test. Just 5 years ago, nobody would have believed it. Given certain trends, I expect 5 years from now an AI will be able to drop many of the heavy caveats/redactions/biases built into this test and do well in a head-to-head competition with history's greatest painters.
This isn't the progression people expect, though. They expect AI art to do as well as a 5th grader first, then get as good as a college student, etc. But they're going through the same type of training, so instead of taking the traditional path to developing their skills, they'll go through the Uncanny Valley on their way to excellence.
He doesn’t actually claim that a 30% rate of fooling humans counts as passing - he just predicts that by the year 2000, there would be computers able to fool humans 30% of the time in a five minute conversation.
What he actually proposes is more sophisticated. It’s really much more like, if you could have extended interactions, and come away with the same sort of value that you do from equivalent human interactions, then we might as well say it’s as good as a human. In at least some parts of the paper, he takes seriously the idea that not being able to enjoy strawberries and cream is a potential problem, because shared enjoyment is part of what we look for in human friends.
But I think the really surprising parts are how well he predicts the capacities of computers decades in advance (a decade or so before Gordon Moore invents his eponymous law even), the fact that he considers ESP to be so well established that it might break his test, and the suggestion he makes at the end that some sort of machine learning is likely to be a more effective approach than directly programming a conversational AI.
Yes, I definitively would recommend reading Turing's original paper. It also anticipates and answers many of the objections people still bring up to this day.
Agree. I see a lot of stuff on the internet saying the Turing test has been passed and when you read the actual details they did not do an actual Turing test as he described it in his paper. I contend that no computer has yet passed an actual Turing test, mostly because no one runs an actual Turing test.
You should read Turing's original paper. It's quite short and easy to understand for the layman.
He explores some great nuances and also anticipates most of the objections people have later brought up. (Mostly people who vaguely heard of his test, but haven't read the paper.)
The problem is that if you're comparing human art to AI art, you don't want the AI art to benefit from human curation.
Imagine a chess-playing bot that has a human grandmaster correcting its moves whenever it fumbles: it'd be hard to claim that it's playing "AI chess" at that point.
There is a separate side question of "how does a human+AI system perform against an all-AI system or an all-human system".
To steal Matthew Talamini's analogy below, I think it's more like a blind taste-test than a game of chess.
If we decide to go with the chess metaphor, though, it's more like "we'll select the best AI player and the best human player to play a match", or perhaps "we will select the best 10 AI-vs-human games and the best 10 human-vs-human games to look at"—a human "correcting" *every move* is more like digital art, i.e. a computer doing what a human commands.
There is no input from the AI in such a scenario, which seems clearly distinct from this. The level at which it is curated isn't every move nor every pixel, but complete works / complete games.
From my point of view, a 3 year old's scribbles are more interesting because they have intention behind them. If you've ever asked a kid that age what they're drawing, they're often making a picture of something very specific – it's just that it's wildly abstract. I'm not saying that makes it delightful to look at, but it does make it interesting! Whereas AI can sometimes be exciting to look at, but fundamentally lacks intention. I mean the human prompt is there, but there's a disconnect between that prompt and what's visually created. The place where a conscious mind would intentionally organize the visual around the idea is just missing.
Also for what it's worth these paintings are mostly B Sides and I think it would be hard to argue that they really represent the pinnacle of human artistic achievement.
If "intention" was something intrinsic to the art itself, there should be no difficulty telling AI and human art apart.
I.e., it's fine to value it, but it is like valuing the state of mind a chef was in when cooking a meal: "it's not interesting to me to eat it" and "something is different about the meal itself" ought be distinguished.
Maybe intention is the wrong word. But human art does have consciousness behind it. AI art doesn't. I personally don't have much trouble distinguishing AI art and human art apart – for why, see Ilzo's comments in the original post. If you take the stance that these add up to "small inadequacies" then I guess you might think AI art carries as much or little meaning as human art, but what those inadequacies do is betray a lack of consciousness, which means that the whole project is meaningless (and on an aesthetic level, it just falls apart and is actively grating to look at).
I don't like the food comparison (cooking can be an art, but eating itself is a survival need, so not a 1:1 correlation). But if I were to hold with your metaphor, I'd be saying that something does taste different about the visual "meal" AI creates, and it tastes off.
It would be very easy to distinguish AI generated images from a child's scribbles, because AI can imitate the most skilled human artists but has no intention, while a child has very little skill but a clear intention.
I think what you're looking to account for here is the presence of taste. But a taste filter has been applied to art history (what survived, which are well known) as well as the AI competitors. It would be interesting to see this test designed and run again by Scott's artist friend.
What would be the right standard for a purely AI competitor though? Doesn't seem like a simple answer. You can argue the prompting process itself reflects human input, plus does the AI get a chance to review it's own work and decide itself what's most likely to pass the test?
Maybe giving a set of human artists and a set of AI artists identical prompts? - I think this has flaws, as well as being obviously challenging to set up.
Maybe an AI has to set the prompts and select the images itself with only some initial prompting about the nature of the test?
The test feels much more legitimate to me when I consider it as a kind of blind taste test.
If you want to prove that California can produce good wine, a blind taste test versus French wine is really useful. If a lot of people have the idea that only French wine is any good, a fair blind taste test between the curated best of California and the curated best of France is absolutely called for. It can't establish that California wine can replace French wine, or that people generally won't be able to tell the difference. But it's still useful.
Same with AI. Anybody who has the idea that only human-made art is any good should update based on the results of this test. It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful.
You can for sure demonstrate that AI is capable of being used to produce some kind of image that has some kind of broadly defined aesthetic value. The really hardline anti AI art crowd should dial it down a notch.
It's not an argument for AI maximalism though - human artists remain vastly superior to AI artists on almost every possible metric.
Yeah, I think we're agreement here. And it's probably the source of our original disagreement.
You seem to understand the "proper" Turing test better than me. So I guess that taking that view this is not a valid test is correct.
But the "popular" intepretation of the Turing test is more like a taste or capabilities test. Can AI generate art that's as good as a human?
> It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful
I think all of these statements need an explicit "for now" appended. AI art is advancing super quick. Most of these issues Scott curated out, like messed up hands and text won't be an issue soon. In fact, the flux models that came out recently basically solved text already.
> It doesn't show that AI artists can replace human artists
I'd expand on this. A _lot_ of work human artists do will be replaced. Most of corporate soulless art will be automated. Generating portraits of book characters, etc.
There will always be a market for human art, but will be linked to prestige. More than it is today.
It also will never replace the art that's more than "pretty picture". Art is more than the physical thing itself for many people. If who created it matters, then that kind of art is impossible to automate by definition.
Note that what they're comparing it against isn't human art, but human art plus human selection. I think if you're trying to find the more aesthetically pleasing images between AI slop and human slop, the AI slop would win handily.
> I think what people are talking about when they say they don’t like AI art is AI art plus unskilled human selection, which also was not tested here.
I think a lot of people just think AI art is inherently bad. But I agree that there's a big problem of people posting AI slop instead of going to the extra effort and just posting particularly interesting ones, or using all the methods they have at their disposal to add artistic control to the result.
I think one of the things I like about AI art is that in order to make it stand out you have to make more interesting stuff, whereas for a human you can basically paint a bowl of fruit and if you do a really good job people are impressed. Then again, that's exactly why I thought the Victorian Megaship was AI, so maybe I'm way off base with that and there's plenty of human artists that do the kind of thing I'm interested in.
Thinking back on this more, maybe I missed the point. What we should be comparing AI artists plus human selection to is human artists plus AI selection. Anyone up for training AI to recognize good art?
I feel like the art chosen does skew the results fairly significantly. I am not an art expert, I expected it to be very hard, and I got about 60%. However, not only was the AI art chosen to avoid obvious tells, I think there was extra selection to avoid many subtle tells.
It felt like human-made computer generated had more detail than average, and the very simplest looking images were more likely to be AI. On impressionism, there were no actual images by Van Gogh or Monet, who people really like, and so it didn't surprise me that people rated the AI mimicry of those artists very highly. More people like Van Gogh's style than like Gauguin and the village of Osny (also good).
I actually left the exercise feeling that it might be easier to differentiate AI art than I thought, not harder. But my bar was very low.
re why cat court is AI, lots of detail = AI works there. I guess it failed us on the human drawn ship. But if you add in topic like silly and super detailed is surely AI
All the cats have the same expression, displaying a transitory emotion; for all of them to share it is both unnatural, and boring for a human to recreate.
It's a really weird composition, too. Why is the king cat's body hidden in some huge alcove while the queen cat's body sprawls across the floor? The small orange cat is sitting on nothing, next to a stool. The tabby cat at the right is standing on two legs and doesn't have front legs at all. The curtains are pulled very asymmetrically. Where are the lady's legs?
The painting also fails to tell a story, which you'd think the artist would have tried to do if they're going to put so much effort into such a silly painting. Who actually has the power here, the lady or the cat? They both have identical crowns. The lady is in a central more powerful position but she's clearly dominated by the cat behind her. Is this some kind of formal occasion in the throne room? Some characters seem to think it is, while others are just casually sprawled around.
I was surprised so many people missed the ship. No current AI model can render such complex, coherent rigging and ratlines without at least some ropes blurring together or bending. Seems like it's obviously a 3D render.
I think the main tell for me for Cat Court is that it seems too glossy, in a way that's very common in AI art but rare in human art of that style. Just about everything that's brightly-lit seems shiny, even if it clearly shouldn't be by virtue of having a rough or fluffy texture.
The same kind of glossy feel seems to happen by default in 3D-rendered artwork, unless you do a lot of worth with shading and textures to get a different effect. Come to think of it, that might be part of why human-made digital images were often incorrectly identified as AI.
Looking at it more closely, an additional tell is that the style of the candle flames are inconsistent with one another and with the overall style of the image. The ones on the left candelabra are fairly normal, but the ones on the center and right candelabras seem to have lens flare effects.
count me as an anti-AI person specifically _because_ I think the AI art is excellent. But that's just because I think artists are important parts of society and I don't want them to disappear
What's the point of an artist that can't produce anything of value? The same applies to any other worker. Humanity will be made obsolete in due time. You can't build something new without destroying what came before.
Your first reply said they were "important parts of society"; now it seems like you're arguing the problem is how they would feel about it, rather than how the rest of civilization would get on without them? Those seem to me like very different, almost diametrically-opposed arguments.
The normal way that I evaluate whether something is an "important part of a system" (as you suggested) is by asking what would happen to the system if the part were absent or non-functional. For example, if I want to know how important a doorknob is, I ask what would happen if there were no doorknob, or if the doorknob stopped working. This strikes me as the obvious comparison invited by your turn of phrase.
It doesn't particularly matter what I meant by the phrase, though, because I'm paraphrasing YOUR assertions back to you in order to check my understanding of them. Even if what I said is ambiguous, clarifying it wouldn't particularly help, because the entire point is to figure out what YOU meant; once your position is clarified, my attempts to paraphrase it are dispensable.
And you notably did not provide any clarification of your own position.
To me, your question looks suspiciously like you were trying to insinuate that maybe I'm in favor of killing people. Which would be totally false, and not remotely required by the counterfactual comparison (which just requires that society stops utilizing their art), and not my fault even if it WERE implied (since I'm just discussing the meaning of YOUR assertion), and a brazen evasion of my question about what YOU meant. If that was what you were doing.
I don't think AI art should replace human art (because I don't think AI art looks good or holds much value), but I also think that even if it did human artists would still have plenty of value to contribute. Art isn't just making pictures (or sculptures or novels or whatever), it's making meaning through every element of a picture. Unless it becomes conscious (instead of just being an algorithm stuffed into a fancy sweater), it won't be able to do that. Of course, a human artist could work with it to make something that makes meaning, but I tend to find that AI fundamental lack of meaning interferes with a meaningful output. Idk, I have yet to see an interesting collaboration between human and AI, but that doesn't mean that it couldn't happen.
The problem is getting stuck in a local maxima where AI is good enough to outcompete humans at entry level art, but not good enough to come up with anything original or drive the medium forward.
I don't particularly care about fine art, but I care a lot about writing, and I'm worried about what will happen when an LLM can show up with superficially good-enough prose and plot that it scratches an itch for the first few books, but then by book ten you notice some patterns, and by book fifty everything is familiar, and by book two hundred you've stopped reading anything written after 2020 just in case it was touched by a machine. Imagine a Marvel movie projected into the human retina-- forever. Imagine going to a used bookshop and finding the contents are all at the same level of excellent mediocrity and only one book in ten thousand delights you.
> Imagine a Marvel movie projected into the human retina-- forever. Imagine going to a used bookshop and finding the contents are all at the same level of excellent mediocrity and only one book in ten thousand delights you.
You realize you're literally describing the pre-AI state of "arts" today, at least as it pertains to movies, shows, and books?
Nobody makes anything but hack sequels a child could write. Writing quality has plummeted in every single streaming medium, as the void of twenty thousand crappy shows each service needed to spin up to be "unique" creates a vacuum sucking in ever-crappier writers to fill the yawning blank page need.
Books have ALWAYS been 99%+ schlock - publishers recently confirmed it. They only ever back and give shelf space to the handful of "established" writers that churn out quantity schlock, like romance novelists, Danielle Steele, and the like.
I personally think AI will be an IMPROVEMENT on this, because then at least we'll be able to get books in different genres of a given quality.
Or imagine being able to drag and drop a couple of artists and assign percentages - 10% this one, 20% this one, 40% this one - and then the AI creates a new radio station with original compositions inspired by that allocation?
Same deal for shows and movies. That sounds like a strict improvement to ME over the endless "sequels and schlock" we get today.
> You realize you're literally describing the pre-AI state of "arts" today, at least as it pertains to movies, shows, and books?
Yes, and this is the result of automation and streamlining, but AI is the ultimate expression of automation. Imagine it gets worse to the point where it's no longer feasible to find anything good, at all. Writing a book is costly signalling. An AI writing a book might technically be costly, but if the cost is in the cents or dollars range, then it no longer functions as a barrier.
For me, what I hate is not that bad content exists. It's that good content exists, unable to be found, even though the writer and I are both desperate to connect. It's difficult to find such content today, but in the future it may pass the point of no return. AIs reading AI-written works to try and assess their humanity and skill level.
Culture feels like nuclear fission, and AI feels like it's padding out the fissile material until nothing can collide.
> Imagine it gets worse to the point where it's no longer feasible to find anything good, at all.
But AI should help with this, too! Above and beyond the "mixing and aggregating" things I mentioned.
Literally the only thing I use Gemini for is summarizing Youtube links so I don't have to watch them - this is particularly great when it's longer than a few minutes. You can also ask questions about the video, and it will answer. Sometimes, that process will inspire me to actually watch the thing.
Your personal AI assistant can consume and filter impossibly more media than you ever can, and you can train it on your specific tastes. Then you can send it out to the schlock mines and have it come back with gold, or at least denser-than-usual-ore.
Everyone's going to have one of those assistants - there'll probably be some free ones (like facebook is free), mass market will have $10-20 a month subscription ones, and richer people will have more expensive ones with humans in the loop to greater or lesser degress. But all of them will be performing this function. Arguably, we could all use and benefit from these NOW, because human-generated schlock is quite deep enough to need it already.
I like the sausage. But if I find out and don't like how the sausage is made, I would advocate for making that way of sausage-making illegal, because I know that a free-market solution is not going to help if that sausage has no competitors or has a competitive advantage over the less harmful ways, and/or consumers don't know the harmful ways of the sausage-making to make an informed decision.
A majority of the source images used by AI were generated by humans. AI is basically painting with human stuff. On the other hand, so-called human-credited art uses computer-assisted tools and techniques, sometimes to the extreme (the contemporary work at least). Using a 3D art program is an example. The bulk of the final image is the work of the computer program, but the human gets credit. In lieu of "prompts" are procedures the human artist follows.
There's not much difference between AI image generation which requires human prompts, and human-created art that requires sophisticated computer programming.
Yes, lots of it. Computer programs do the heavy lifting. There are decisions to be made, but rendering a flock of birds using a 3D program might takes 5 minutes. Doing it by hand is a different matter altogether. Human gets credit for both.
I'm not following. Once you're at the rendering stage in 3D graphics, all the human effort is done. The human effort is in the modeling, texturing, and building the scene, which definitely isn't something that can be done in 5 minutes for a complicated scene like a flock of birds. Why are we describing the rendering part as the heavy lifting of digital art? And how is this process at all similar to prompting an AI image generator?
Example 1: There was a landscape and environment program called E-On Vue which supplied ready-made environmental elements -- trees, flowers, rocks, water, sky, clouds, planets, soil, etc. I used it for a few years. Yes, you need to tweak settings to get exactly what you want, but even if you don't make adjustments, you still would get an image resembling something from an AI image generator. Using Vue was akin to being a photographer -- positioning the camera, and determining how much light was needed in the scene. You can make a simple scene photographically realistic in 5 minutes. Look for image examples on the web. (Here's one that I'm sure took around 5 minutes to set up. c. 2010 or so. Rendering the animation took longer. There are 5 ready-made objects : sky, ocean, tree, sand, girl.
Example 2: Download a simple bird object, or purchase one, then import it into a 3D program. Use the "sprayer tool" to add 100,000 random points into a scene and have the program link your bird to each point. Select a sky backdrop from the program's assets. Finally, have the program generate an image of the flock of birds in the sky. The final image is the work of the computer program, but the human gets credit. Obviously it would be beneficial to have 3 or 4 bird objects with wings in difference positions. There are global rotation settings to get more variety, too. Yes, you have to learn how to do it and decide if you like it or not. It's not much different from prompting an AI bot, in that anyone can come up with an image without having any drawing skills. I would agree that 3D scenes can take many hours to set-up and render, but the machine is doing the work. It's similar to when I spent an hour of trial and error trying to get Midjourney to make an image I wanted -- and it still wasn't right.
In those cases I would say it's not the computer that does the heavy lifting, rather it's the human artists who made the assets. I suppose I can see the similarity between using pre-made assets to construct a scene and prompting an AI. But if we consider the digital art creation process as a whole, including asset creation, the effort is mostly human and not particularly similar to the AI process, in which the computer really does do the heavy lifting.
In the limiting case where AI generates "good" images from minimal prompts with no or very little correction, I'd call the human role closer to commissioning a work of art than to creating it.
My comment is only about categorizing human vs AI in terms of Scott's survey questions. Visual artists deserve credit for everything they do. I've spent many hours wrangling with illustration programs -- it is work! and one needs an artist's brain/training to make meaningful decisions along the way.
Biggest update for me is that I'm completely bamboozled by Impressionist and other abstract art styles; they already look so "inhuman" that I can't find any handholds to discriminate on. Which is personally fine, those genres are a turn-off anyway so I don't mind if AI ends up dominating them...I also notice that, despite intentional curation away from "DALL-E house style", some of the AI images in the test have that exact vibe, which is what made them really obvious targets for me. It's easy to overcorrect - I'm pretty sure I waffled on Rainbow Hair - but for now, it feels like a useful heuristic still. Will be interesting to see if human production of AI-style art increases to follow demand. I assume someone wants images in that fantastical-veneer style, anyway...
I don't know whether AI will really dominate in the fields of Impressionism or Modernist abstraction – those are both historical movements so I'm not sure what there is for AI to really "win" in those cases.
Rainbow Hair looked human-created to me because it lacked the flaws in and around hair that the AI-created images had. In the AI art there were inexplicable gaps between sections of hair that should be solid and there was also hair wicking off into the background that wasn't connected to anything. This is exactly what Ilzo meant by "bad and incoherent" details.
Both of those art styles are commonly sought by people as prints for room decor. It is possible that decorative art (as opposed to "high" art or illustrative art) will be dominated by AI generated images in the future.
It prompted me to Google Hatsune Miku to verify my memory. Apparently she's typically teal-haired, and drawings of her drift from teal to both blue and green.
That said, I'm sure I've seen that image as, like, a videogame cover on Steam, or something. I don't think it's Hatsune Miku.
Yeah, my initial reaction was "no way that's Miku, I've been at her concert".
But maybe that's what makes it clearly human - no AI would take "Hatsune Miku" as instructions and output that, you need human creative freedom or significant prompting.
I managed to narrow down my answer set in the .xlsx, I got a score of 74% which I suppose is pretty good. Got fooled by the tough ones, but looking at the spreadsheet I'm surprised by how many folks were fooled by ones I thought were EXTREMELY obvious like Minaret Boat, Turtle House, and Landing Craft. Would love to see more in-depth research on people's abilities/strategies to distinguish between human and AI.
I got turtle House wrong because I thought I knew the name of the artist who made it, but I had confused it with another "giant animal carrying a city" work by Serena Malyon (https://www.serenamalyon.com/).
I could see that it looked very AI but I thought I had specific information that it wasn't.
I got 61% and got Turtle House wrong! I guessed that at least some of the more "Dall-E House Style" examples would be human-generated (Scott did say he was trying to match them), so there was a lot of second-guessing going on.
I thought there were more obvious ones, especially ones with weird anatomic detail. Snow Princess has a peculiar left eye (her left), Strings Come Alive has odd fingers, Girl in Field has odd eyes, Muscular Man check out the reclining man's left hand. Minaret Boat and Turtle House I got right, but I thought Landing Craft kind of nailed the shitty sci fi covers of my youth.
Underrated comment. I couldn't complete the survey. It was fun for the first ten, but then yeah, I started feeling like a robot. It's like with my casual perusing of the subreddits for spotting AI art. I think, "Ooh, this is strength-training," then I think, "Eh, we can just downsignal AI art and tell people not to submit it." Which is where we've landed as a society.
When someone says an artwork of theirs was made with AI, or is AI art, what does that mean exactly?
At one end of the scale, it may mean it is the unmodified output of a (fine-tuned) model given some (human-optimized) prompt. At the other end of the scale, there was some AI output used, but it has been heavily modified, combined or otherwise transformed by the human using various tools, such that the work is overwhelmingly human-created.
I presume Jack can tell you what his AI art process is, but what about the other artists you have chosen to feature? Were they asked what they meant? I find that different AI artists have very different definitions.
The prompter appears to have cleverly chosen a year not too long after Caillebotte's big "Paris Street: Rainy Day" breakthrough when talented but less original painters were hopping on the bandwagon, so there are many decent paintings to imitate.
I would imagine a prompt of "Impressionist painting of a bustling Parisian street under the soft rain, 1781" would be more of a challenge for AI.
This is Jack - I'm pretty proud that I fooled a lot of people. Going in I had the goal of making the art that fooled the most people, thrilled I achieved it.
I thought your pieces were unusually good, easily the best ones in the mix. The lesson for me is that human discernment still matters - you still have to have an eye for what works, even if you're just creating a million images and sorting through the best ones. Curation is still part of the artistic process.
I think it's easy to overlook the fact that AI art is still created by humans and still involves human aesthetic judgment. The computer is a tool just like a drawing tablet or a paintbrush - you're still responsible for the decisions you make with it.
What do you get when you give Midjourney a date before the existence of the style? Such as "Impressionist painting of a bustling Parisian street under the soft rain, 1781"
I tried it in Ideogram (because it's free to use), and the year didn't seem to make much of a difference, though with 2081 I got cars in 2/4 renderings. Overall I was impressed by the output. I'd suggest you give it a try.
I called #11 because of the gibberish pseudo-autograph in the bottom left and #20 because of the different-sized windows. Didn't see anything obvious in the other two.
Fun exercise and good article! I'll say again that to my eyes, the blue haired anime girl and black hair anime girls are absolutely obvious. The black haired anime girl looks like most AI-generated anime girl (I think the model was orange or something? It was when I tried it), this kind of blend of photorealistic but also anime. I would have expected Miku to be drawn way earlier than that, as the style is more 2000-2010 than post 2010. I guess what I'm trying to say here is the more experience you have with a particular subset, the easier it gets to distinguish it.
There's also something important here: with AI we're all seeing artists as they grow. This is something we aren't used to seeing, especially with old painting. I really really like Rothko, but I like his """newer""" paintings more than the ones he did before he really found his style.
I have never seen an anime character with hair like Anime Girl in Black, there were far too many strands coming off her head. It would be annoying to draw and even worse to animate.
I got both wrong, weeb credentials in danger. It's also amusing that the blue girl has something seriously weird going on with her left hand, which I didn't notice in time, but would've made me even more sure.
Ice Princess was the other obvious anime one. String Doll also basically adjacent to those two. All of the ones I was most confident on were these anime pictures, because I've only ever seen AI-generated anime images look like that.
Yeah true. Personally I wouldn't even consider them exactly "anime", it's kind of like between anime and 3D render if that makes sense? Or maybe a new art form that is mostly possible thanks to AI.
This is irrelevant, but I quite recently learned it is incorrect to refer to Leonardo da Vinci as ”da Vinci” because it isn’t his name. Correct is ”Leonardo”. Same goes for other people from the era before real last names.
(It is pretty snobbish to care about this kind of thing but I thought it was interesting so am mentioning it. )
There as some really good fake Leonardo da Vinci painting out there, like the notorious six copies of the Mona Lisa featured in the 1979 BBC Documentary "City of Death" which were indistinguishable from the original except for the words "THIS IS A FAKE" written on the panels with a marker underneath the paint so it would show up on an x-ray.
Not surprised Victorian Megaship has such a skewed rate if people are mostly thinking about painting. Since it's a 3D render, it's not AI, but it is also not rendered by human hand. Would be a bit like throwing a couple photographs into the human art mix. What did the breakdown of guesses on the other 3D renders look like?
A big part of the confusion is from the topic itself. AI Art got a tendency to generate this kind of super-proportioned megastructure with a ton of repetitive details.
I like Ideogram the most, it even handled such a difficult case as my description of six humans without confusing the details (e.g., giving one person hair color of another). But it usually requires effort in prompting and dozens of generations for the result to be good. I once generated a total of a thousand pictures in one day because I was optimizing that hard.
I did very badly on the first part of the test. I took the "please don't spend a lot of time looking at the details" directive to heart, and my gestalt impressions were largely wrong. When "allowed" to look at the details, I got all 5 right. But there's a difference between "this looks good on initial viewing" and "this is a piece worth coming back to again and again."
I'm not super interested in visual art, but love me some stories, whether they're books, movies, games, whatever. I have some background as both a writer and a critic, though I'm definitely not a professional at either. And I'm constantly frustrated by the quality of narrative I'm expected to enjoy - stories that all my friends act like are fantastic but that seem to me to be kids bashing action figures together.
There is a place for that kind of story. In fact most folks would call some story of that kind their favorite. But that doesn't make it good, durable, lasting art.
To get to the point, if 95% of people say "IDK, looks fine to me" and the 5% best-trained critics are calling it slop, I'd expect that it's slop. I don't know what makes an impressionist painting "good" because I'm not into impressionism anyway. But presumably it filled a specific artistic need at a specific time - maybe art critics had seen 80000 hyper-realistic renaissance portraits and were sick of looking at hyper-realistic renaissance portraits. I don't know if showing random internet folks decontextualized versions and specifically instructing them not to look at them very closely can capture that kind of nuance.
Art is ultimately just a means to an end -- an ointment to make our existence just a little less painful. Just because you're harder to please doesn't mean your opinions are more correct, nor does it make you a better person. But as long as we are talking about good stories, ahem...
Have you heard of the critically acclaimed MMORPG Final Fantasy XIV? With an expanded free trial which you can play through the entirety of A Realm Reborn and the award-winning Heavensward and Stormblood expansions up to level 70 for free with no restrictions on playtime.
So addressing the meme first - Final Fantasy XIV is easily the best story I've ever experienced in video games and possibly in any medium - though I'm wary of my own recency bias and Arcane is giving it a run for its money in terms of time spent bawling like a baby :D
I'll push back a little on the other bit - I think really good art has the ability to change perspectives and challenge people. I wouldn't go so far as to say that AI art isn't art. I don't think that's a reasonable position to take about...basically anything. I think you can probably do plumbing artistically, though I'd prefer you not do it in my house.
But I also think there's a connection between being willing to treat art as more than just something to pass time until you die and personal growth. Maybe I'm a snob but all the people I know with the least interest in art and narrative are also the people who are least likely to consider the feelings and perspectives of others. And all the people I know who are defensive about comforting art that isn't challenging are also immature in other ways.
I don't think that being harder to please makes your opinions more correct or makes you a better person. But I do think consuming a lot of art, and seeking out art that's challenging will both make you harder to please and make you a better person.
Meh, my dad is an art snob, and he's also basically a psychopath. Though, I do have a feeling he's appreciating art for very different reasons than you and I... The point is, there's more to art than empathy, and there's no reason why it would necessarily make you a better person.
...It sure as hell hasn't helped me. I mean, I probably would have killed myself a long time ago if I didn't have access to this much entertainment, but in retrospect, maybe that wasn't a good thing for anyone involved...
Anyways, thank you for recommending Arcane, I was holding off on checking it out but you're finally giving me a reason to watch it.
Like all status stuff people can definitely use art criticism as a cudgel to bully people, so very fair point.
I don't know you or your situation but from my experience the main thing to remember is that time continues and things change - it's not just this forever however much it feels that way now.
And yeah - Arcane surprised me. It takes a lot of tropey characters and makes them act very human and form very believable, sometimes complicated relationships
> And I'm constantly frustrated by the quality of narrative I'm expected to enjoy - stories that all my friends act like are fantastic but that seem to me to be kids bashing action figures together.
Thank you! This is exactly how I describe "writing" now in the age of crap streaming and "everything is a sequel or reboot."
Obviously their method is putting a bunch of sets and action figures pertaining to the show in a room with a 5 year old, and then, fingers eagerly poised, they breathlessly transcribe whatever said kid comes up with over the next hour. That's E3 of season 2, post it!
But when I say this to people, they look at me like I'M crazy, when writing quality has obviously plummeted nearly universally to "below schlock" depths.
I generally avoid known AI output (including art), but this isn't because I think it's bad in the "ugly" sense; it's because I don't know when it'll start posing memetic hazards and the downside of avoiding it is negligible.
(I also oppose all usecases for AI that result in money going to frontier labs in any way, because I want those labs to not have money to end the world with.)
Seems like pretty conclusive proof of something I have long suspected: impressionist art sucks and everyone is just pretending to like it because it's high-status. Just like "cultural studies" or whatever the Sokal paper was about.
I think it's the opposite: most non-arty people say impressionism is their favourite, while the real art snobs are into abstract expressionist stuff that most people think their kid could do a better version of.
I'd guess it's probably the lowest-status of the famous art movements other than maybe 19th century academic style, and also probably the most genuinely popular.
I assume it became a "famous art movement" in the first place because it was so bad that only status-seekers would pretend to like it, and then it got too "popular" so the really hardcore status-seekers had to distinguish themselves by finding even worse things to pretend to like. Barber-pole fashion and all that.
Whether or not that's true, abstract expressionism itself is many cycles of that old by now, yet remains unpopular except among art snobs, while my impression is that impressionism is genuinely popular nowadays.
Impressionism is weird - it does have a lack of detail and focus but the merit is in recreating the sense of a scene in the viewer. The former elements work well for AI because it can just copy the structure of other works, as other people pointed out, without the details that would give away either the AI generation or the plagiarized sources.
Most artsy people I know don't like impressionism. It gets taught a lot in art history classes, though, because it marks a huge turning point in artistic styles. It also seems to be pretty popular among the general public.
The general public doesn’t like it because it is high status. They like it because the pictures are pretty and it doesn’t require the kind of background knowledge a lot of older art requires to be appreciated fully (like who is Saint so and so).
I take pretty strong issue with this. First of all I really really like impressionist art, as someone who struggled to enjoy much of Art History classes, Impressionism actually evokes an emotional response in me on rare occasions, especially in person.
But I have two other objections: First, impressionism might fit the skillset of an AI artist better than other genres. Second, a piece with mass appeal is not better than a Monet just because more people like it... art does have some objectivity mixed into the subjectivity.
1. AI is often quite strong at creating a mood or vibe, using strong lighting and universally appealing forms. Small wonder its good at making impressionist pieces that appeal broadly to people. It often fails on details, but details aren't the point of the impressionist piece. It also tends to draw more recognizable forms and does less abstraction... which makes the piece more accessible, and more easy to see. But you could compare that to junk food... its satiating, broadly appealing, but low in caloric value.
2. Spotify has lately been exploring creating AI-generated music that they can play without paying licensing fees, which often fills up the top spots on their most popular playlists. An example is the Rainy Day Jazz playlist, which has a very moody piano piece that a non-jazz listener might hear and instantly feel that rainy day mood they were looking for. Its evocative as a piece, but it is paper thin in if you have any jazz knowledge at all. It is vapid and empty, doing the bare minimum of establishing a vibe without saying anything at all. That's what I think AI impressionism is like... if i had any training in the medium, I suspect I wouldn't like the ai pieces at all.
Yeah this is exactly what AI Impressionism is like. It's vaguely "nice" and "pretty" but it doesn't communicate anything. If you look at e.g. Pissarro's paintings of Paris you can feel how cold the streets are, you can almost feel the clop of the horses' hooves and the intimidating blank facades of the Haussmann boulevards - the AI version is just "isn't Paris nice :)".
But a lot of people like "isn't Paris nice", and also, Paris is nice! Wouldn't hang it on my wall personally but I don't think of it as bad so much as different.
>AI is often quite strong at creating a mood or vibe, using strong lighting and universally appealing forms.
I've gotten my best luck from AI image generation (using Adobe Firefly, since I'm an employee and get free premium access) by giving it chunks of poetry, song lyrics, or mood-heavy prose. Tolkien and Lovecraft are particularly fruitful sources for the latter. It often gets something with an interesting and fitting aesthetic, even if the details are often not quite what I was going for.
Great job selecting the images for the test. However, I’m unclear on the real takeaway from your experiment.
If you show people original Monet paintings versus copies by a talented forger, most wouldn’t be able to tell the difference. Yet, we wouldn’t conclude that the forger is on par with Monet. Originality and being the first to create something of lasting importance are key aspects of good art. This point feels uncontroversial.
Your experiment seems to demonstrate that a human skilled in prompt engineering can act as an effective forger. People can reverse-engineer impressionism using tools like Midjourney. While this says a lot about Midjourney and AI, it doesn’t say much about art itself.
The more important question seems whether AI can create a new type of imagery that marks a milestone in art history, akin to what perspective or impressionism represented in their time.
In the long run, the answer is likely yes. But as of today? Which AI pictures from your test would best qualify?
This hits the nail on the head. And by this token, truly non-trivial AI art would “fail” to “pass” as human precisely because it was doing something new that no human could ever have done. I suspect it would also be broadly unpopular with a human audience, for more or less the same reason.
I think AI has already met the second milestone. The DALLE "house style" is instantly recognizable, stylistically unique, and influential (if my Facebook feeds are any evidence). Future art history books will absolutely include these types of images.
True, today it is "low status" and doesn't influence the "right people" but I would not be surprised if this style is picked up by certain types of human artists for exactly that reason (if it hasn't already)
A comparable example might be the advent of advertising posters in the late 1800s. But did they really influence art history that much? Toulouse-Lautrec, Mucha?
I am not sure what you mean by that question. I'm not very familiar with art history but both of those people have wikipedia articles indicating some degree of notariety.
I think part of the problem is that the categories AI vs. Human don’t have sharp boundaries. The favorite AI picture by Jack Galler is very reminiscent of Van Gogh’s Café Terrace at Night. So you have a human, Jack Galler, guiding AI, to mimic a beautiful painting. In fact, what we are seeing in Jack’s pictures are Jack’s style of using AI. He is the artist.
The Blue Hair Anime Girl and Rainbow Girl are both human but could never have been created without digital art software that has a large repertoire of “tools” that create elements that have no physical tool analogs. Human created Blue Hair Anime Girl and AI created Anime Girl in Black are simply variations of the same thing. Both are done with digital art programs, perhaps one more AI than the other.
//The favorite AI picture by Jack Galler is very reminiscent of Van Gogh’s Café Terrace at Night. So you have a human, Jack Galler, guiding AI, to mimic a beautiful painting.//
I think this is a bit unfair. I mean, the AI image is certainly similar in theme and overall layout, although not colour balance, painting style, mood or details ('minor' things like whether you have buildings or a river on the right).
And so what if it's reminiscent? Are all the other (human) artists who happen to have painted scenes vaguely like that copy-cats?
I'm pretty sure Van Gogh was not the first artist to paint a terrace café at night. And more to the point, 'all' he was doing was 'mimicking' a scene which was literally in front of him.
Some human artists also draw details that make no sense. For example, to me, many of H R Giger’s painting make no sense when I look at the details.
(And also, when professional artists look at my sketches, one of the things they often comment on is that I’m making the details make sense where often people who just put in something vague. So my own drawing style is the opposite of this )
In fairness, a key element of Giger's work *is* the fundamental "wrongness" of what is being presented. That's why he's so successful in horror - the detail gives the image weight and the nonsense of them disturbs the mind.
The Ingres fooled me into thinking it was AI. But it wasn’t the Greek text; it was something about the coldness of the style that made it seem like AI.
That was one I was fairly sure was human, just because of the consistent variety of the characters. The classical and renaissance people look markedly different and consistent among themselves in a way that my experience would say would be quite hard to get an AI generator to do.
I didn't even see there was text until people pointed it out in the comments. I got it wrong because it looked like Jesus was Zeus and also Columbus was there.
One nice thing I can do with AI art gen is create pictures in the style of Leonid Afremov, who was impressionistic and post-impressionistic. I bought some reproductions from his studio which were half by him and half by studio artists. While his own reproductions were on average a touch better, one of my favourites was by a studio artist, and one of my least favourite by him. Only one was clearly not his to the untrained eye, it was so bad it actually seemed good eventually, like a fading memory of a strange dream. It's possible that estates of deceased artists could have a single robot painting high quality reproductions, improving their finances and the availability of the art style
I correctly identified Giovanni Girolamo Savoldo as human, but my reasoning was: anatomy and perspective a bit rubbish in places, but seems to be from around the sixteenth century, so I’ll give him a pass on the errors and conclude human.
Art does not exist in a vacuum. Knowing that a piece of art has been shaped by human experiences, emotions, and skills adds a layer of connection and meaning. This human context is also why people care about chess played by humans, but comparatively few take an interest in AI only chess. Yes, you can lie to me and tell a compelling story about the context of an artwork which was actually created by AI, well done.
It may be that most people mean “pretty picture”. But the people who care most about art usually don’t care if it’s pretty, and instead care that there is more to understand and learn that adds to what you can visually see. (In the case of Duchamp’s urinal, this is very explicit.)
We're in agreement. Should we just taboo the word "art" then?
To me, it was obvious that Scott was trying to show that AI can draw well enough that it's indistinguishable from masterful humans, and sometimes "more popular" drawings than humans.
I don't even disagree with OP's message. But seems like a non-sequitur in here.
I didn't scrub these images, so if you open them in something like notepad, you should be able to find the full positive and negative prompts as well as the models used.
At first I was skeptical of your method. Since you asked for good AI examples, which puts a thumb on the scale, and then you only took the best of the good stuff sent. On the other hand, if you are not doing 100 versions of a result for AI/Art Music you will get mediocre results. So I guess I'm fine with it.
As a musicians when Suno and Udio came out I was really curious and started working on some AI songs. My wife was horrified, and refused to even listen to the compositions. So I made a song to mimic the style of one o her favorites "Tom Waits", and just stuck it in a playlist during a long drive, she raved about the song being one she never heard by him, and one of his best..LOL people like art because of signals. There is beauty, but most people want things that raise their status or signal their in-group and beauty is only a small part of that, and sometimes is a Negative signal.
Sorry, I listened with an open mind but this is just bad. It has none of the distinct images that we'd expect from a Tom Waits song, and is a tangle of clunky metaphors. The images in the Tom Waits songs are distinct, tell a story, and the lines scan. Recognising quality isn't about "signals".
But look, even people who aren't familiar with Tom Waits can see this. Anyone scrolling by, please weigh in which verse looks like it was co-authored by ChatGPT:
***
"There's a ribbon in the willow and a tire swing rope
Oh, and a briar patch of berries takin' over the slope
The cat'll sleep in the mailbox and we'll never go to town
Till we bury every dream in the cold, cold ground"
***
"Tell me, is it the crack of the pool balls, neon buzzing?
Telephone's ringing, it's your second cousin
Is it the barmaid that's smiling from the corner of her eye?
The magic of the melancholy tear in your eye?"
***
"Streetlamp spills a pool of amber on the cracked sidewalk below
Fat cat perched on a fire escape, eyes like polished onyx glow
Hear a couple's laughter, light and carefree, a melody that stings
My smile's a rusty hinge, happiness barely hangs on its strings."
***
"Well, Friday's a funeral and Saturday's a bride
And Sey's got a pistol on the register side
And the goddamned delivery trucks, they make too much noise
I feel like you missed the point of the larger ACX post. Not saying this is somekind of masterpiece, but lots of peoplle can't tell it's AI. You think you can, great, I personally am not a big tom waits fan. So if you say the story telling isn't as vivid, sure it really feels like cope. Like those lyrics are more random, but not every song he writes is like that, Christmas card from a hooker is just pure story. The point of the post is blind that even people who hate AI also perfer it. We could do a test and bet on the results. But I think it would have a similar score as the AI art above. The less familiar you are with the art, the more alike it will seem. Like there is no extra finger in the music, I think people would say it's a song and not AI. As a big Fan, you are focused on the narcisism of small differences. As a musician I feel like the tools could be much better and that the intro is too long, and I see things that I'd change, but much of that is on me for not spending more time and doing more versions.
Very random lyrics, that was the point. Trying to mimic the style of Beck, cause I'm old. Still I could have worked harder and made it better, but this is a dude doing fun shit while I'm on business trips. Imagine if I was actual serious about this stuff.
The thing is, the Waits lyrics aren't random, right? They're striking and novel images that combine together to build up a sustained impression, a mood, a sense of place. The young adult night time adventures of Looking For The Heart of Saturday Night is distinctly not the same setting as the childhood misdemeanors of Kentucky Avenue, though they both are formed of brief narrative snapshots.
The pieces chosen in the survey were cropped and curated to be maximally fooling, but I grant you that plenty of people are bad at spotting AI creations. Whether people can spot its provinence or not, I think most people who enjoy lyrics in music could discern a difference here. Maybe I'm wrong!.
As for Waits, his pure story ones I suspect ChatGPT would do even worse at emulating. I'm not sure it could be as distinct as this:
***
"Hey Charlie, I think about you every time I pass a fillin' station
On account of all the grease you used to wear in your hair
I still have that record of "Little Anthony & The Imperials"
But someone stole my record player, how do you like that?"
***
Waits has a set of images in mind that he is conveying through characters. Generated text is even worse at consistency than generated illustrations. There's a reason why novelists aren't worried about the robots coming for their jobs yet. It's not a "small difference" to notice why one song has interesting lyrics and another is a series of cliches. But look, let's test this. Here are five verses, one is ChatGPT, the other four are different lyricists I enjoy. I'm sure as someone who has played around with AI text you can spot the imposter very easily:
As someone who doesn't know any Tom Waits, I'll go for the "Tell Me, is it the..." verse. Mostly because it seems sloppy to rhyme "eye" with "eye" rather than any particular narrative failure.
This is a super-interesting exercise, thanks for doing it! The question this is raising most urgently in my mind is “what kind of test like this would everyone agree is fair?” We can’t randomly select from all existing human art, it is skewed towards what has survived. So you had to filter the AI art for what seems good enough to compare. It doesn’t seem like a fair test but I have no idea how to do better or whether it’s possible to do much better.
I think that whenever we start worrying about fairness, we should think about why we care about fairness in this application, if we even do. When we are playing a game, fairness is very important, and it usually means that each player has similar possibilities available to them (though there are some games that are intentionally asymmetrical, like Twenty Questions, and others, like chess, that have only a slight asymmetry, but fairness is achieved by playing as both white and black in a series of games).
But in some applications, like figuring out whether someone is a competent driver, fairness is less important and some sort of accurate representation of an underlying skill is more important.
I don’t think this test is aimed at revealing some specific truth that needs specific conditions to reveal it, but I also don’t think it’s a game where winning is important. So I don’t think fairness really enters into it.
I can't help but feel that this was, in some sense, a way for Scott to show that the people who think AI is terrible at art (or whatever specific complaints they have) are probably letting emotions unrelated to the judgments of the output dictate their assessment of AI's
"ability" to create art. In that sense, fairness does seem relevant.
However, to me, the whole thing is a bit of an irrelevancy. To my knowledge, none of those "AI" artworks were generated by AI. They were generated by humans, prompting and then selecting for whatever outcome they wanted. Many of the AI image generation programs have the ability to zero in on pretty specific areas of an image, to create custom models, etc. AI feels like a tool for artists. I wouldn't want every piece of art to be created through AI any more than I'd want every piece of art to be created through Photoshop or Maya or whatever, but I think it's inevitable that we'll see a large portion of art created through AI because it's very accessible and the skills required to do it are very different (and probably a bit lower than most other more "artist" tools in terms of required practice for mastery, at least for the moment). I mean, even with all the digital tools we have, there are many artists who still labor over a canvas. I'm curious as to how these differences in methods to create art influence what human explorations attract artists.
As someone who consumes very little art, my point of view is that I'm interested in when AI art gets good enough that I can just ask for a work with a fairly straightforward prompt and get approximately what I'm looking for (e.g. I just asked DALL-E for a Yithian at the optometrist's office, a somewhat humorous take on that rather bookish fictional race, but didn't get a very good image). So, I'd guess a slightly better experiment than this one would be if the AI image curation step was also automated, to match the (minimal) human effort needed in the use case I'm (admittedly only mildly) interested in.
I think the test as performed is a useful test to see if the current crop of AI is capable of generating at least some images that are hard to identify as AI generated. You don't need to go back all that many years for answer that question to be an obvious and trivial "no".
If you want to test the overall quality of AI image generation relative to human artists, that's a very different question that would require a very different testing protocol. Off the top of my head, I'd suggest getting together two groups of volunteers, one people who are good at writing AI image prompts and curating the output, and the other human artists of various skill levels. Give each participant a certain number of un-engineered prompts to illustrate, plus feed the raw prompt to various AI image generators without curation. Then crowdsource ratings of the images on dimensions of technical merit, originality, fidelity to the prompt, and aesthetics.
>I like this picture. There’s nothing wrong with it. But somehow it’s obviously AI. If you asked me why, I’d say “something about the lighting”.
Also the two cats to the right of the throne are wildly deformed. One doesn't have any forelegs, and the other has some terrible conjoined foreleg. The Cat Queen (?) 's legs do not make sense with her torso. The candelabras don't make sense. Etc.
The lighting is a clue, dollars to donuts the prompt includes "HDR" or something similar that describes the effect and so much AI art defaults to that style, but the whole picture has a lot of AI art's "traditional deformities."
The way I figured Victorian Megaship to be human was the fine details. Even though I've never set foot on a sailing ship, the rigging was just too consistent for AI to get right, and everything about the ship just made sense. It was plausible.
Contrast that to Minaret Boat, where nothing makes sense: Windows of different shapes and sizes thrown haphazardly about the minarets, and the ropes behaving in nonsensical ways, like ropes splicing into two or more, or hanging in shapes that gravity wouldn't allow.
Generally, the lack of world knowledge when it wasn't possibly the point of the piece was my #1 tell that something was AI. The gross anatomy on Muscular Man, the free-floating earring and asymmetric accessoires on Anime Girl in Black, the free-floating wax on Still Life, the child-like perspective errors in Ominous Ruin, and so on.
Much worse. You can at least live in a McMansion, but I wouldn't set foot on Minaret Boat to do anything other than a short tour to satisfy my morbid curiosity how bad it would look on the inside.
Exactly the same. The two ships were the ones I found most easy to distinguish. Victorian Megaship's consistent lines, rigging passing behind sails and always coming out the other side where it makes sense, is something I've seen AI have a lot of problems with before.
The element of having a terrace on the left at night is the same, and they're both impressionist, but everything else is different. It gives an initial impression of being very similar, but the more you look at it, the more it seems like we're just matching on a couple elements.
Of course, the original Van Gogh painting is fairly directly copying the actual terrace he painted.
Surely there is little difference between making a highlight post out of a 500-1000 comment substack post (as Scott often does) and out of a similar-sized random sample from the contest?
The substack comments are sorted, so it is possible to follow only threads which are interesting. That makes a huge difference. I looked into a few of the answers, but I already gave up after 10 or so. It's just completely unstructured.
This was a well-designed test which proved to me something I didn't fully realize before: good art requires good reasoning (planning, reviewing, erasing, backtracking, rethinking, etc.). Generative models paired with Large Reasoning Models will be a huge jump in quality.
(Looking forward to this particularly for Music Generation.)
I started to take this test, but after looking at 6 or 7 images, realized that using my dinky phone screen for viewing made the exercise pointless. Maybe the viewing medium is key to determining the humanity of the message. Does AI art do chalk, oil, pastels, watercolors on various surfaces? Until it does, who cares?
I did the test on my phone then realized I got a few obvious ones wrong when looking again at the pictures on a laptop. I wonder how much of an impact that would have on the overall results.
At the current moment, far more people experience chalk, oil, pastels, and watercolors through digital photographs of them rather than in person. But I do hope that the rise of AI art makes people remember that the physicality of these media is one thing they have going for them, and start going more to museums to get a fuller experience.
You missed the conclusion that humans were better at identifying Ai art in genres they were familiar with. I am not exactly an impressionist or classical art connesiuer; I muddled around looking for brushstrokes. On the digital side, and especially the anime art side, I did quite well and was very confident because it's the sort of image I actually view regularly (online). The anime girl ones couldn't have been more obvious.
As another example, your friend the artist is infinitely more in tune with "art" than I am, and did very well.
hah I had no idea on almost any of the pieces (I commented elsewhere re: which were obvious to me) but the anime girl ones were TOTALLY baffling to me. I didn’t even have a vague hunch of an idea. Totally outside of my experience.
It makes sense that people would do worst with impressionism. One of the main "tells" of AI art seems to be small details. Impressionism tends to have a "blurry" quality that lacks small details.
But if you look carefully you can see that even big details are slop with AI. For instance, #44 was apparently the most human-voted AI generated piece. I identified it as AI by noting the incorrect bicycle wheel arrangement and the lamppost that turned into a tree. The gestalt could be human but Degas would never have made dumb mistakes like that.
The people who got almost all of them right were 100% cheating.
I think that’s fine, as they didn’t skew the results much and it’s interesting to see with tools that people can identify AI images. They were either using an AI image detector, really zooming in to the images and looking for inconsistencies, or using something like Error Level Analysis which makes most human vs. AI images mostly trivial: https://fotoforensics.com
Maybe some were cheating, but it's certainly not an impossible task. I think I got about 85% and all the ones I got wrong I had very low confidence about. If I was a half-dozen coin flips away from getting 100% and there were 11,000 people playing, the top scores seem pretty likely.
It was a good test but there are a decent number that are recognizable human paintings. I noticed a few and looking over it realize I should have caught a few more too.
1) After reading your explanation, I feel like the test is way more biased than I originally thought. Just as an example, you deliberately avoided the "AI house style", except you *did* include human works close to that style, like the extremely detailed ship. You also excluded complicated poses, specifically because the AI is bad at them. WTF?
2) For reference, there was a similar test over at DSL, except it put pairs of human/AI images of the same style side by side. In that test, the entire forum ended up with a score close to 100%. https://www.datasecretslox.com/index.php/topic,12393.0.html
3) The test, whatever it measures, isn't a good indicator for AI slop. The slop-ness comes from looking at a corpus of images. You can make one pretty decent AI image, especially with human curation. If you make an entire card game where all the images are AI... well, they will fulfill the function of a visual aid to distinguish cards from each other. Similarly, the non-slop-ness of human art depends on historical context. Contemporary impressionist-inspired paintings hang on the walls of cheap hotel rooms rather than museums, alongside Bob-Ross-style landscapes.
4) In no way shape or form is this a Turing test. Please fucking stop abusing this term.
I somewhat agree but want to note that there's a reason "5 guys stand next to each other" lineups are no longer best practices for criminal identification and I think it at least partially applies to this too.
When you do the more accurate sequential lineup you're asking the witness, "are any of these dudes the suspect?" When you do a side-by-side lineup the question you're asking the witness is "which of these people looks most like the suspect." If it's guaranteed that one answer is correct, you get different information than if you don't.
I'm not saying the DSL test isn't useful - it's just asking a fundamentally different question than this, and I'm not sure that question is as interesting. Instead of asking "if you see a piece of art, can you tell whether it's AI?" it's asking, "if you compare two similar pieces of art, can you tell which one is AI?"
I agree these are different tests. The DSL one might have some issues, but it's much better at exercising the AI over the entire space of possible images. The styles that Scott used are very, very broad, much broader than something like "art noveau". And it turns out that AI is pretty bad at actually doing art noveau, as it mixes in a lot of its default style, which allows you to spot the fake from a mile away.
Yeah the choice to only include art intended to fool people on AI and human sides (as well as cropping the images in ways that change their impact) makes my minor objection a quibble. I still wanted to make it :D
I did expect about as much biasing as Scott described, which made the test, now incorporating my mental model of Scott and a bunch of reverse-reverse psychology epicycles, not so fun for me. A test deliberately designed to exploit dominant strategies to confuse people is fair test for subject-matter experts and/or people who make very confident pronouncements, but that doesn't make it a referendum on those confident pronouncements.
Right. The 60% absolute number isn't calibrated against anything so we can't draw any particular conclusion from it. However, we can draw conclusions from the *relative* performance on the test, i.e., that people with knowledge and/or disdain of AI art are actually better at recognizing it.
The AI art in that link doesn't match the style of the human art. The faces in the AI art are all in a modern, realistic, digital painting style. So of course the forum gets close to 100%. That test is "way more biased"...
Or it would be if it were trying to prove that AI can't make portraits that humans can't distinguish from human art. It's not possible for a test to be bias in a vacuum: the test measures what it measures. The bias comes when you try to interpret the test results as indicating something that they don't actually indicate.
You wanted this test to test whether people could tell average DALL-E AI art, or AI house style art from human art. What this is testing is closer to, "can humans tell good AI art from human art if we avoid some of the known AI weaknesses, like hands and text". You're calling it biased because it's not testing the question you wanted asked.
The answer to "can humans recognize mediocre AI art as AI?" is obviously yes. "Can humans recognize the common AI house style?" is also a yes. The answers to both of these are obvious enough that I don't know why you'd want to waste the test on these questions other than that you don't like AI and want to score some points against it. It's not interesting; it's just political fodder.
I could easily program an app that pulls random words from a dictionary, have it run a trillion times, then personally select sets of words that appear to be human-like sentences, then ask people to compare those to actual human sentences. And if humans could only tell 60% of time it could be said that my random word generator passes the "Turing test" as mush and any AI art generator was proven to in this exercise.
... Which is to say, not a all. You can't have a human filter on the computer's output and still call it a Turing test.
> I’m afraid I don’t know enough math to tease out the luck vs. skill contribution here and predict what score we should expect these people to get on a retest. But it feels pretty impressive.
There's a pretty simple method to do this for binomial outcomes like this - conceptually, you measure how much variation you'd expect across people due to luck, subtract that from observed variation to estimate how much variation is due to skill, and use those two factors to weight how much to regress to the mean
Hm, I am a bit skeptical. Using standard deviations seems to be the right thing for events that are not very extreme, like for estimating the 90% quantile or so. But here we are looking at extreme values, something like the 99.95% quantile, and I don't think that standard deviations are helpful in this range. The binomial distribution does not decay quadratically in this range, but exponentially.
i think you have a point, although i'm not sure what would help you more when the underlying samples are so small. this could be a lower bound at least.
Well, if we assume that it's binomial then we can always go back to the exact formulas for binomial distributions. Or use a normal approximation of the binomial distribution.
I tried to get a feeling of what 5 people being 98% correct means. Also, further down, TheIdeaOfRyu suspected that some people may have used the solution, for whatever reason.
I looked at the highest percentages from people who scored on all 50 pictures:
98%: 5 people
96%: 3 people
94%: 2 people
92%: 4 people
90%: 7 people
88%: 16 people
86%: 17 people
84%: 48 people
It is indeed extremely suspicious that the range 90%-98% is so evenly distributed. I think that any reasonable probabilistic process should have those numbers dropping very fast with higher percentages. The only exception is that a small number of people have a magic ability to tell AI from human, and that this ability is not gradual (some can do it a bit better than others), but that it is on/off. So, probably that was with the solution key, and I assume that most people above 90% indeed used that key.
For the 90%, perhaps a helpful intuition is this: assume that we have some very good recognizers who have an easy time for 30 pictures (90% success probability), and who are at chance level for the other 20 pictures (50%). Then in expectation they would get 37 pictures correct (74%), and the probability that they get at least 90% correct is 0.2% if I computed correctly. So if we had 1000 such good recognizers, we would expect to see 2 of them score more than 90% by a mixture of chance and being moderately good.
I judged #15 as AI-generated not because of the text but because of what seemed to me like AI compositional incoherence: it has a Greek temple with various Ancient Greek-looking figures, and then e.g. in the bottom left corner we have...Sir Walter Raleigh and a group of Renaissance gentlemen?
It’s “The Apotheosis of Homer”. Ingres is painting all the great people of history he thinks were inspired by Homer, including Shakespeare, and his own contemporaries like Mozart.
I thought that same thing was a sure sign of it being human! Renaissance gentlemen are likely to draw paintings linking a line of cultural descent to Renaissance gentlemen from the Ancient Greeks.
Same here! I saw the various styles, started recognizing a few, and literally said "There's no way an AI could make this." The amount of effort put into copying those famous artworks into another one with such precision would be crazy with AI prompting.
I think "incredible detail, often recursive-like, in 'unimportant' places" is what gives the "obvious AI" look. This explains why "Victorian Megaship" looks AI-generated.
> People say AI can’t invent new styles, but I’ve never seen any human make this exact type of weird bird.
It's a bit obscure, but for me that bird was really reminiscent of the Brazilian modernist painter Chico da Silva, who specialized in weird colorful animals (birds and reptiles in particular). See e.g. https://www.independenthq.com/features/chico-da-silva-brazils-forgotten-visionary for some examples. It's kind of the exception that proves the rule, though: AI art might not be brand new or innovative, given so many humans draw so many things all the time, but it's certainly way easier to find new art you like by prompting DALL-E with "weird colorful bird" than discovering a particular obscure painter by serendipity.
Maybe before taking the test I didn't read all the preliminaries, but I kinda wasn't figuring that digital art created by humans would be included. I was thinking, did pigment and paintbrushes create this image?--if not, then AI.
If I didn't know about them in advance, I would probably have determined pretty much all of M. C. Escher's stuff was AI generated. Yet it's great stuff, partly for having been an original concept well-executed.
The original Turing test was not "generate a lot of realistic conversations by either human+human or human+AI, then take the most human-like conversations and let people judge whether this was h+h or h+AI". What you did here was much easier.
I think no AI can pass the more turing-like test of "make a (complicated) prompt, wait 10 days and then judge whether the result was made by a human or an AI", if you really try to craft prompts that are hard for an AI (which is the idea of the turing test). Maybe even add the possibility to ask for changes.
None of the major image generators are designed to do that. I imagine you could make something like a GAN with an LLM and diffusion model where the discriminator attempts to distinguish human images from generated ones (assuming you could get the funding). But a big use case of AI image generators is making images that don't look like images humans have made, and you'd be crippling that use case by training it to only make man-made looking images. There are only a few big players in this space and none of them have focused on that.
The current models need a prompt, so this test is testing what they're capable of with a good prompt.
I know. I just wanted to point out that this test is much easier than a real turing test would be and that people shouldn't draw wrong conclusions from this.
One of the aspects of AI paintings that I most often picked up on was a lack of brushstrokes: people’s skin is too smooth, for instance. I think one reason the AI impressionist paintings fool us is that “make an impressionist-style picture” tells the generator to include what looks like brushstrokes. The impressionist work that I found easiest to identify as AI was the one with the woman, because, again, her skin looked too smooth.
Petition for future things like this to send a copy of your answers to the inbox of everyone who provides an email. None of my friends or I saved our answers :(
so all it takes to create AI art that's mostly indistinguishable from human art is an AI to generate the art, plus a significant amount of human time and effort spent prompting, adjusting, and then curating the AI art side-by-side with human art in a format that's deliberately arranged to make them difficult to distinguish
I was so sure about 49: Mediterranean Town being human (as I saw "mistakes", that looked human), and so sure about 15: “The Apotheosis Of Homer”, by Jean-Auguste-Dominique Ingres (1827) to be glorious AI (the 3 guys at the feet of the green lady: rather garbled, aren't they?!). ... With a 60% hit-rate I declare me a failure in this test (or declare AI as the winner). Though the choice of pics seems to have been kinda even more unfair than Scott admits: taking AI pics that looked much more human and human art that looked more AI ish than I would have expected - if you did a fast run of this test, the algorithm "judge as 'human' what looks AI, judge as AI what looks 'human'(BOCTAOE)" might have worked better than long deliberation on each - and much better than "follow your instinct".
In regard to the throne room with the giant cats: Why are there so many lit candelabras in bright sunlight?
I don't think I did very well on the test, but there didn't seem to be a convenient way to check my answers, many of which I didn't remember, against the key.
Still, that picture stinks of AI, probably because implausible giant things are commonly AI, and the details are sharp in an odd way.
Yep. AI has problems with scale. Another tell is that basically the entire image is cats, and that nothing in the image has any implied meaning or purpose beyond "Throne room with cats"
Well, the human on the king cat's lap could be construed as a humorous take on a human royal family with a cat sitting on the king's lap. I can't tell which of the kittens is the equivalent of the crown prince... :-)
>Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
The key point here is that they can't tell when it's heavily curated by people with decent taste. An average piece of AI art is hideous slop. Most human art is terrible too, obviously, the difference is that it's much harder to produce, so there's not a comparable deluge of it.
"Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history. "
All this tells you is that humans ("readers of this blog") can't tell the difference between great art and hideous slop. And that's a fair cop! If you'd asked me before, I'd have agreed with this.
The subsequent paragraph about forcing people to confront what "art" is is the main point here. A lot of the comments section is people arguing about what it means to "not like AI art" and how much curation effects things. The critique of the gate picture gets at this, too. The interlocutor finds the lack of meaning in the details displeasing, but when I looked at that picture, I didn't look at it closely enough to notice or care about that. So what is art? Who judges its quality? What is the point? These are all up-stream of an "AI Art Turing Test", and frankly, it's the more interesting discussion.
Yeah, if we're using "walruss, whose personal aesthetic is essentially not having one, can't tell the difference between a Van Gogh and a convincing AI imitator" as the test of whether AI can create art then...er...please stop doing that.
Maybe art isn't meant to be enjoyed completely abstracted from all context, in 30-second assessments (if you took that long on each picture). Like a good poem, in my experience the best art has layers you peel back the more you study the scene/composition/facial expressions/etc. the artist included in the work - often related to the title of the piece. This is why things like the gate fail. They don't stand up to more than a cursory review. How can it become your 'favorite piece' that you go back to again and again if there's nothing beyond the superficial?
I think this plays into what people are trying to convey by invoking 'AI slop': it only stands up in a world where you look at it once and then you're done with it. For it to 'work', you have to flood the world with superficial images, not art that can be enjoyed by returning to it again and again. Maybe you're not forced to return to the same images if you're constantly supplied with new content to enjoy and then instantly forget. Slop, served up by the bucket-load. But then you've lost something beneath the surface. It's like the difference between curling up with a good book and doom scrolling.
I've long said that the only reason why people can be fooled by the AI thing at all is that we've lost the ability to judge and think about art in a meaningful way anyway. Again, I'm not a visual arts person, but this is definitely true with narrative art.
Or maybe I'm just complaining that nobody in my friend group wants to sit and talk about the same book or TV show ad nauseam for 3 days straight and I do :P
I don't think you're going to take humans out of the art-generation process anytime soon, even with the introduction of AI. Not a single one of the images Scott used got into the contest without significant human intervention at multiple steps along the way. Selection/curation will likely continue to be a feature going forward, and who knows but that we'll see much better stuff coming from AI as humans re-generate subsections of the images to force an artist's vision into the final product. Who knows?
But the problem with "people don't appreciate art" is only going to get worse the more of the current approach to AI art we use. If a large part of appreciating art is the process of uncovering the subtle meaning hidden beneath the surface, there's a problem in flooding the everyday experience with 'art' devoid of any such hidden meaning/intent, until nobody knows when to start really looking at an image because maybe there's something more to it. With the AI art status quo, the world becomes a Rorschach test, devoid of all but the meaning you give it.
Would you be equally upset with people who go to an art museum to look at paintings, but don't bother to read the tiny placard explaining the history of the painting? Are such people merely consuming slop rather than properly appreciating art?
I'm not really upset. And sometimes I'm the one going around not reading the placard. My lament is that the discussion we're having necessarily removes context and any kind of deeper introspection about the intent of the piece. If all we care about it whether something looks good from a superficial perspective, that's fine. I think there's absolutely a place for superficial art - and probably a big place - but let's not pretend it's the same thing as something with intent behind it.
I just think it's folly to equate AI art with something that has a deeper message to convey, especially if we do that by conducting an experiment where you necessarily strip all deeper messages in order to make the comparison. That's like comparing two poems based on whether they both rhyme without considering whether they make sense or have something deeper to say about the world.
Scott's post seems to be at odds with this conclusion, especially the final section. The place where a distinction is being made between art with a deeper message and "pretty pictures" is here in the comments. Otherwise, I can't see how Scott would write the following:
"Alan Turing recommended that if 30% of humans couldn’t tell an AI from a human, the AI could be considered to have “passed” the Turing Test. By these standards, AI artists pass the test with room to spare; on average, 40% of humans mistook each AI picture for human.
What does this tell us about AI? Seems like they’re good at art."
That seems like equating art with pretty pictures to me. And, yes, he then goes on to say, "I’m more interested in what it tells us about humans." And if his complaint about humans were directed at the blinded graders in his test, I'd expect him to say something like, "Humans don't take the time to appreciate art, its nuances, and its deeper messages." IF he'd said that, I'd agree with your point.
But Scott went the other direction, attacking the art world and the tendency for human veneration of a deeper message in the total absence of aesthetic appeal. I can't think of a more clear way for Scott to equate AI art with something that has a deeper message ... and then discount the deeper message entirely as being irrelevant to whether the art is good or not.
This point doesn't apply to the digital artworks, but for all the others: I really think it's worth considering that none of these are paintings; they are either digital simulations of real paintings or AI-generated simulations of real paintings. So what this is asking isn't "can AI convincingly simulate real art"; it's asking "can AI convincingly simulate a digital simulation of real art." It would be a lot easier to judge an AI-generated simulation of a real painting against an *actual* real painting.
And the actual painting contains a lot of the depth and artistry that makes it meaningful - in the application of the paint, the texture of the brushstrokes, the scale of the work, and so on. That's why looking at a Pollock or a Rothko or a Van Gogh on your screen is nothing like seeing it in person. A digital representation of an artwork is really just a thumbnail. We're just so saturated in screen-mediated imagery that we take it for the basic form of artistic experience; and quantitatively it does make up the vast amount of that experience. But it still isn't the experience of the artwork itself.
AI clearly doesn't have the ability to convincingly simulate *that* experience. Will it some day have that ability? What it would require is a body - that thing that actually applies paint to a canvas by gesture and so on - before we could even start having the conversation. Which, incidentally, is what AI would require for there to be real questions about AI having personhood or consciousness.
This feels like moving the goalposts. Before AI art became good, would you or anyone else have said that the essence of art lies in the physical brushstrokes? That any art that's not an oil painting can't possibly be "real", let alone beautiful or meaningful?
I 100% would have said (and in fact did say - you'll have to take my word for it) that the essence of art lies in the experience of the art object itself, not in its digital reproduction. Like I said, a Van Gogh painting just isn't the same thing as a digital image of a Van Gogh painting.
Other considerations would apply to other media, like film or literature, but this survey was about paintings.
Only some art is made with brushstrokes. I hope that one thing people get from the rise of AI art is that for the art made with actual brushes, there’s something you get from the real thing that doesn’t quite come through in a hi-res digital photo on your screen.
I'll assume that something like a 3D printer hooked up to an AI counts as a "body?" And you'd need the paintings to be scanned with a laser distancer to generate depth maps for training. Then, you could apply the depth maps along with the colors to do some kind of 3D printing. Stable Diffusion already has AI mediated creation of depth maps based on digital images that works reasonably well. I'm actually playing around right now, in my free time, with a workflow that allows me to create textured images from digital images, with just a little bit of tweaking. Sadly my 3D printer is very much consumer quality and prints from filament, which doesn't help on the details, so the texture of the prints prior to sanding and coating are a dead giveaway that the work is 3D printed, not painted. But all the key limiting technologies look like they already basically exist.
In any case, I don't see why greater diversity of media relates to 'personhood' or 'consciousness.' We don't fault the great painters for not making sculptures or the great sculptors for eliminating the element of time and movement from their work. Creating in more dimensions may be a challenge for general intelligence, sure. But couldn't we distinguish between generalizability and personhood?
Does a photograph count as human art? The photographer doesn't, like, select individual RGB values for pixels, but he does carefully choose a scene and framing. Many people consider this enough to make the photographer an artist and the producer of the image. It seems to me that the high level of curation taking place with the AI art is taking a similar role. Perhaps the conclusion is something more like AI is one tool that a skilled human artist can use to create art having genuine merit.
One would think this would be the obvious conclusion from the beginning, but there has been much baggage.
If one does 3D art and describes "shape here," "shape there," "very realistic texture for that," few would have a problem with this. Even when much of it is generative: Foliage. Trees. Landscapes. Large structures.
The line seems to be conveniently drawn at what the beholder finds distasteful.
Photography is sometimes considered art - just like performance art, or baking, or anything else. But don't let the use of the word "art" make you think the two skills are at all similar, or that they produce the same results. Painting and photography are different things with different merits.
The prompt that produced the most liked image was, "Impressionist painting of a countryside café terrace during twilight, 1883". Coming up with that prompt and recognizing that the result looks good is a skill, but that skill is in a different category than painting.
Say you had a friend who painted and you asked them to create an 1880's style impressionist painting of a countryside café terrace during twilight, and based just off that prompt, they painted a beautiful painting. Would you take most of the credit for your skill in coming up with that prompt? I'd probably say my friend deserves most of the credit, while recognizing that coming up with good suggestions is also a valuable skill.
Also, it takes years of practice to become a master painter, and coming up with a good prompt seems like a much lesser accomplishment. Especially since with each new version of AI image generators, the good prompts become simpler. In earlier versions of Midjourney, people would append dozens of tags to their prompts to tweak the results. Now you mostly just tell it what you want.
I get the sense that you're trying to reason that humans are still special and valuable in this context and so you're trying to elevate whatever role humans still had in this art generation. But before you know it, someone will create an art generator that can work without humans generating the prompts, and then we'll have to invent some other reason why human creativity is more valuable.
Pretty sure I pooped the bed on this but I have no way to compare my guesses from a month and a half ago to the results. I guess I should have written this down.
Wow, thanks! I do have a substack. I hardly post, but I suppose I should add some content. Right now I'm playing with using displacement maps generated in Stable diffusion to produce textured 3D prints.
Edited: Okay, I've posted a curated series of images I made while trying to figure out how to make a watery humanoid form. I'm not sure the best modality for sharing, so I hope this works.
This was interesting! It reminded me of a story on (I think) 60 Minutes from many years ago, looking at the work of painter Thomas Kinkade. Probably people know who Thomas Kinkade is, but just in case: Kinkade was an incredibly successful painter of schlock. How successful? "According to Kinkade's company, one in every 20 American homes owned a copy of one of his paintings." I have no idea if this is true, but it gives a sense of how prolific his studio was.
The segment mostly consisted of 1) a bunch of prissy art professors pointing out (correctly) that Kinkade's paintings are terrible, in a manner that was guaranteed to make them look like condescending elitists; and 2) a bunch of Kinkade-owning regular folks saying that the professors could go suck it.
Points being: popular stuff can be bad. But also it depends on what critical lens you are applying. An art historian is considering different things than a person who just wants to look at a pretty picture.
Whatever you say about Jackson Pollock, it’s not schlock. There’s no sentimentality or appeal to superficial prettiness and fake nostalgia. Whereas with Kincaid, that’s all you get.
There are plenty of criticisms one can make of Pollock, but it’s not schlock.
That wasn't my question. Why is Kincade's work shlock but somehow Pollack is a genuine artist, when Kincade produces pretty pictures and Pollack produces the canvass equivalent of static?
Agreed. Pollock is just random static. He (or his followers) may be able to describe his grand intentions in great detail, but those intentions don't translate into execution.
At the time Pollock was working, he was doing something new and controversial that no one else had been doing. Whether it was a good idea or a bad idea, he was doing it as part of an idea (just like Marcel Duchamp with the urinal, and the impressionists with their hazy brush strokes). The Pollock pieces aren’t the equivalent of static - when I googled the images I realized that I was easily able to recognize Lavender Mist and Blue Poles as two of the most famous ones that I have seen many times, though I didn’t recognize some of the others. I’m sure I could learn to recognize Thomas Kinkade paintings if I looked at them much (when I googled I realized I had been picturing Norman Rockwell, who played a similar role a generation or two earlier) but whenever one isn’t familiar with a style, there’s a sameness there; and with Pollock that can seem like static.
It was new because no one else was willing to debase themselves and waste material like that before, and it was controversial because it was garbage. It doesn't matter than you can recognize two different stills of static, it's still static.
"Controversial" could be "because it's breaking a field out of a rut" but it could also be "because it's trash." Matt Walsh's documentaries are controversial because they identify naked emperors and expose a fraud most people were going along with. Some random internet poster is controversial because his posts are racial slurs and conspiracy theories. Those are not the same.
De gustibus! Kinkade's pictures aren't pretty. When I look at them, they seem humorously terrible. The color palette is bad, the compositions are flat, the subject matter is trite. Many people disagree, and my explanation for this is that they have bad aesthetic judgment. This also explains why people like well-done steak and many superhero movies. They can like those things if they want, I certainly am not here to talk them out of their enjoyment. Sometimes I enjoy those things too, even when I recognize their flaws.
Everyone has bad aesthetic judgment in most domains. I could not tell you what distinguishes good Chinese opera from bad Chinese opera, and I find it all basically unlistenable. But if a Chinese opera lover were to claim to hear a clear difference in quality between Mei Lanfang and, say, Black Pink, I would not insist that they are being ridiculous and arbitrary in their judgment.
Your criticism of Pollock, as far as I can tell, is "I don't get it and I don't like it." Fair enough! But if you really want to insist that it's impossible for others to draw a distinction between Kinkade and Pollack, it seems you're trying to make a virtue of your poor aesthetic judgment. That always strikes me as a weird move, but certainly you're not alone! Personally, I can admit that I have terrible taste in Chinese opera and it would be nice for me if I were better able to appreciate its qualities. Most people seem to feel deeply threatened by this sort of thing.
No, I'm saying "I do get it and the whole thing is a fart-huffing fraud." I'm also not saying Pollack and Kincade are the same, the point of my question is that the critiques of Kincade for, as examples, poor composition and mundane subject matter are undercut by the elite celebration of Pollack, who never even has composition or subject matter *at all*.
Right, I know that’s what you’re saying. What you’re saying is not complicated and in fact it’s probably the most common opinion in the world. People say, “If they like something I don’t it’s because they are signaling/sheep/idiots.” People don’t say, “I have unsophisticated tastes and am pretty ignorant on this topic, and my tastes probably reflect that,” even though this accurately describes where most of us are on most things.
Here are two propositions:
The “elites” are fools and/or charlatans who have elevated one artist and dismissed another despite there being no meaningful difference in the technical, aesthetic, historical, or intellectual value of their work.
Internet commenter Darkside007 is unable to distinguish the technical, aesthetic, historical, or intellectual value of the two artists’ work.
Bayes would put his money on #2. Of course, the matter is ultimately undecidable because there is no objective benchmark for these things, so you can stick with #1 if it comforts you.
Because as soon as photography appeared, painters no longer had the monopoly on pretty pictures production. So they redefined their job as an abstruse game, where the greatest artist is the one who invents and implements the weirdest idea nobody had before.
Beyond the content of the paintings themselves, Kincade was also just a grifter. His business model was adding a few brush strokes to each of thousands of reproductions of his paintings to make them "unique" and "valuable" (not really).
I don't have a strong feeling about the aesthetic qualities of his paintings, although they are a bit same-y. But I maintain that art is fundamentally a medium for human connection, and I do not believe that Kincade had any genuine and interesting message to convey through his art, therefore I do not think it is good art.
I don't know much of anything about Pollock as a person, so I can't really comment on the sincerity/cynicism axis. However, I don't mean to criticize Kincaid's art on a purely ad hominem basis. I just think that, to the extent that his art had a message, it's not a very interesting or original one ("cottages are nice," "pretty things are good"). I point out his unscrupulous practices because it shows that he was more interested in making as much money as possible off of the few ideas he had rather than developing new, more interesting ideas to put into his artworks.
It's possible Pollock also adopted a similar cynical attitude once he achieved massive success. I'll leave it to those more familiar with his oeuvre to defend him from that claim. But I do think that he at least started out with some radically new and interesting ideas, unlike Kincaid. It might seem unfair, and the time and skill required to physically make a Pollock-style painting might be less, but he was at least trying something new and he succeeded in making at least some people feel something new. This is more than one can say about Kincaid and is why, in my opinion, his work has some artistic value, which does not of course mean that everyone has to prefer it on an aesthetic level.
I've no idea about Kincade's silly business practices, I'm just comparing Kincade, who people mock but produces things I enjoy looking at, to Pollack, who people venerate but produces nothing but noise.
I think a useful comparison might be Mike’s Hard Lemonade and Laphroaig scotch. It’s very easy to appreciate a Mike’s Hard Lemonade, particularly for people with no training or experience with alcohol tasting. Laphroaig is much harder to appreciate, and even many people who like high end alcohols aren’t into the kind of thing it’s going for. But there is a thing to appreciate beyond just a surface sweetness.
Raw lime juice isn't harsh or bitter. It's sour. I think you just don't have very high discernment, which again, is fine. But you seem to think that people who have higher discernment are just making shit up. I assure you, that's not (always) the case.
The whole appeal of Pollack is the complete lack of any appeal. Irony is extremely sophisticated and not at all small-minded and short-sighted, after all.
I think a significant part of my response to non-historically-signficiant art is rooted in “it’s cool you did this”. Not only do I enjoy the work itself because it aligns with my preferences in some way, I enjoy and appreciate the passion and directed effort that shaped the artist and art. At the very least, they thought a good deal over the years about what they want to make, how to do it well, what aspects to study or gloss over, how to square that with their own technical limitations and tics, etc. Seeing low-effort AI generations placed next to that just because it’s “detailed” or the lighting is fancy or whatever is like watching someone crashing a track meet on an e-bike.
I feel like a lot of the selection here was carefully curated to take the best AI generations and the worst art. A traditional Turing test is a live interaction between a person and a machine/human decoy, solving most of the selection problem. But if you gave the decoy a stock list of phrases they had to pick from, and allowed the machine to respond independently, you'd successfully fool a lot of people.
Anime girls are an excellent example of this - there's a ton of generic anime girls gazing at the viewer for AI to model on, and anime itself is a style based on minimal detail and a lack of shading. (One Punch Man expressly parodies this in how it presents Saitama normally and when he's "serious"). So the difference between a digitally-drawn anime girl and a generated anime girl is going to be very small.
Pointillism and impressionism also lend themselves to easy AI replication because they rely on the viewer filling in the details themselves. That gives a huge advantage to AI since it's not expected to be correct, just pointed in the correct direction.
Including images that seem to intentionally evoke AI generation like "Victorian Megaship" is just straight-up cherry-picking. Can human artists imitate AI generation? Yes. Duh.
And then there's the inclusion of outright crap. "Tropical Garden" is an eyesore, and it appears Hockney is an example of the modern art world's idiotic pursuit of ironic detachment at the cost of, well, literally everything else. No one would believe that was the intentional product of a human mind, because there is a base assumption that people are trying to create good works. This feeds into the idea that modern art is merely a vehicle for criminal transactions, because people aren't spending $8 million on this because of its merit.
And then there's Abstract "Art", which is just random gibberish. "Vague Figures", "White Blob", "Purple Squares", "Fractured Lady", "Creepy Skull", "Angry Crosses", "Bright Jumble Woman", and "Flailing Limbs" are all nonsensical trash, which AI can easily replicate because it's 100% noise and 0% signal.
(On an entirely separate note, I recognized "Angel Woman" as a Living Saint immediately, and then the Iron Halo, and then the Guardsmen and concluded "Not AI because IP :P)
For what it's worth, modern/abstract art was the category where I could most reliably distinguish Human from AI art (9/12 or 75% by my count, failing on Purple Squares, Mediterranean Town, and Punk Robot).
Obviously, AI struggles with realistic details in complex scenes but I think it also struggles with images that are too disordered or that appeal solely to emotions and high-faulting aesthetic techniques. It's hard to put into words, but the most abstract pieces were the ones where I had the strongest sense of "a human artist arranged randomness in a way that made intuitive sense to them" vs "this is a soulless machine wearing a human face"
I think the broader point being made -- and while I wouldn't use the same tone as OP I do in fact share much of the sentiment and considered making a top-level comment to similar effect -- is that for art that's sufficiently aesthetically unappealing, the object level question of "can AI make art?" becomes secondary to the issue of "why would I care if it can?"
I wouldn't care for "Angry Crosses" if it had been done by a human, and I don't care for Basquiat, "Purple Squares," or "Flailing Limbs" or "White Blob" regardless of their human authorship.
I think it becomes a little easier to see why the question of aesthetic appeal is a predicate to caring about AI displacement of humans if you think about the issues of utilitarian production and goods demand: "AI can create slightly defective rubber balls with high efficiency" isn't an especially interesting headline because I don't actually want there to be more slightly defective rubber balls in the world. Similarly, because I want relatively less rather than relatively more aesthetically unappealing art in the world, the fact that AI is capable of producing ugly things is intrinsically less interesting and in a sense almost in a different ontological category than the question of how it stacks up to humans in producing pretty things.
I like the Hockney painting. It's certainly true that celebrity in the art world is completely fake and everyone should stop caring about it, but if I just across it in a little gallery somewhere I would still like it.
Interesting that your friend commented on the meaningless detail in the arch as a sign of AI origin, since this sounds like what Gwern identified as AI-like about R. Crumb's drawings in his "Review of Crumb", in the section "Crumb and AI Art": https://gwern.net/review/crumb#crumb-ai-art
The detail that packs Crumb's art may not have any depth of meaning associated with it, but it's organized in a very human way. Regarding the arch, "meaningless" is more of a synonym for "thoughtless" as in literally lacking in consciousness. Humans are drawn toward creating balanced visual patterns (balanced doesn't necessarily mean symmetrical) whereas AI creates semi-random shapes that may look balanced at first because they're sort of in the right place, but reveal themselves to be random if you look at them for a long time. These flaws just look very different to human flaws.
Most of the time I knew I had no idea whether or not something was human or AI, and I was just guessing, but there were three pictures that I was absolutely, immediately, and correctly *sure* were human — the wounded Christ, the Agony in the Garden, and the apotheosis of Homer. It’s not just a religious art thing; I wasn’t so sure about the Madonna and Child (AI) or the saint in the mountains (human). But those were the only three I was sure about at all. Everything else I was very aware of coin-flipping in my head.
I thiiiiiink the correct way to interpret this is that AI is getting very good at generally looking right but isn’t yet good at deliberately visually referencing broader cultural referents in a convincing way.
In the Homer painting, the crowning with laurels, the visual references to various poets throughout history (Dante really pops out; I think with effort I could name some specific others, as well), the lyre, the obviously Greco-Roman architecture in the background (maybe I subconsciously picked up on the lettering; not sure; didn’t consciously notice it) all hung together in a way I just didn’t think AI could hit.
The Wounded Christ, similarly — the composition that carefully emphasized the stigmata, the way his face was dusky to emphasize his death, the reactions of the women, the man who was probably Joseph of Arimethea (but could also work as St. Peter or Nicodemus — I wasn’t sure) the crosses in the background — I just didn’t think it could possibly be AI.
The one that I was absolutely most sure about was the Agony in the Garden. The sleeping disciples, the angel coming down to minister (not Biblical but a common added detail), the soldiers arriving in the background, led by a figure that might be Judas — I just didn’t believe that AI could have so correctly put together the most common referents to that particular Biblical vignette, and I was right.
Although maybe all of that could be “fixed” by a human prompting it more specifically? Idk. But those were the tells for me on the ones I was absolutely sure were human. None of it was in the style of artwork at all. It was all in the broader cultural references.
Huh. All of the AI art has human attribution. I don't know the first thing about the process, but I guess the software is not prompted with "hey computer, paint a picture."
It takes me about an hour of work to produce a pretty good image that actually resembles something I was intending. It takes another hour or two to really nail down the image with the details I want. I'm the farthest thing from an artist, but it's definitely not effortless.
> Sorry, I guess Giovanni Girolamo Savoldo just wasn’t very good at anatomy.
This was (basically) one of my take-aways: Many of the "obvious" indicators of AI art are only good indicators for specific kinds of pieces. Lots of art isn't trying to make a super-realistic depiction of hands (or pre-dates humans being able to do that) or other anatomy. And some humans are just not good at these things either. The Warhammer picture was another example of this phenomenon (to me, at least)--the hand looks like it maybe is supposed to also be one of those birds (AI art often has different things merged into one) and the other arm is pulled back at what looks like an unnatural angle. But these could be deliberately stylistic choices (Escher is famous for images in which different things turn into each other) and/or the artist choosing not to redo it for any number of reasons.
Also, I was confident that that ship with all the windows was AI for what seem to be the same reason you know that cat picture is fake; I figured a human would be unlikely to have the patience for all of those tiny features.
- Mix together a number of different models/LORA/prompts to get a consistent style across pieces.
- Do a trial prompt, find it draws something stupid/not what I want (e.g. it interprets "belly" to mean pregnant, rather than a bare stomach), edit the prompt several times until I fairly consistently (50+%) get basically what I want.
- Gen 40 512x768 (or 384x768) images and pick the one that looks best.
- (Sometimes:) Edit the one that looks best to fix any minor errors or get positioning that the AI can't do (e.g. I could not get the AI to draw the girl with her hands clasped behind her back in a power pose, so I just physically rotated her arms in paint.net).
- Upscale 2x, ten times, with 0.4-0.6 error and ADetailer on. Mix and match between the best ones, editing one hand out for another that looks better, etc.
This all means making a decent piece takes an hour or two. Not all of which is me working, but it slows down the pipeline a lot.
Sloppers' process:
- Put in a prompt and get out a picture, then post it online.
This takes like one minute and approximately zero human effort. Sloppers churn out 100s of images per day; if I spend the day mostly on it, I might get out 5-6.
Hah. Now although most of the classical art I had no idea about, that one I was very confident about because the reflection *isn't* flawless. The dragon's head is reflected but its forelegs aren't. The girl's hands have something a bit like a reflection that really doesn't match where her fingers are.
it's not perfect, but I wouldn't expect a human to produce 100% perfection either. and a photoreal reflection would have minor distortions from refraction, etc.
willing to be wrong - but registering my disbelief
Is there any chance we can persuade you to put up a post showing us something about your prompts and curation? I noticed that several of yours were among my favorites when I read Scott’s description, and I’d be glad to know more about how they came together!
Hey this is Jack. I just used Midjourney. Most of the prompts were just describing some generic impressionist scene. I've never had an art history class (but I am an art history fan!), so I worked with ChatGPT to identify key trends of say, Impressionism, so I could prompt better. I generated a lot of each one ,and just used my subjective opinion to decide what looked good and what didn't.
The Paris Scene (which fooled the most people) had this prompt:
"Impressionist painting of a bustling Parisian street under the soft rain, 1881"
The Riverside Cafe (which people liked the most) had this prompt:
"Impressionist painting of a countryside café terrace during twilight, 1883"
Yeah, it kind of helps Midjourney know what you're looking for. I originally used it in fake photography, would use it for prompts like "Minimalist luxury Tokyo hotel lobby. Midnight. 1997, Kodak Portra".
The year and the photo stock really help clue Midjourney in. Otherwise it looks too new.
interesting, Kodak Portra was introduced in 1998 (https://en.wikipedia.org/wiki/Kodak_Portra) -- I'd love to hear from an AI developer how MJ reconciles the contradiction. I bet more-overtly-contradictory stylistic requests can produce interesting results.
. . . and Jack, you snookered me sooo many times! (shakes fist)
Here's the AI art that I handed off to Scott. (I'm going to take this archive down, eventually. But you're welcome to it for now.) If you open the images in notepad, you can see the prompts and models used. Everything I made was in Stable Diffusion.
This doesn’t capture the experience of appreciating art. I don’t experience a work of visual art as a disconnected picture on a screen. I experience it knowing who the artist is, who they were influenced by or reacted to, when they were alive, and if there is a person or god in the piece, very likely knowing who that person is and why they’re there. I experience art in the context of all the connections I can make to history, art history, and religion. If I were forced to just look at disembodied images with no information about their context and no knowledge of art history, I would enjoy it much less. AI art is less interesting because it by definition has no artist and no context other than “made by an ai in 2024.”
Scott is making an error if he thinks art appreciation is just about “is this pleasant to look at.”
By the same token, knowing about the backstory of AI should increase the subjective value of AI art. In fact, that AI backstory is, in my opinion, more interesting than the life of most artists.
The same critique of "lacking context" can be applied to almost all art from lost or badly recorded human cultures. Magdalenian or Olmec art lacks close to 100% of context and I wouldn't dismiss it as "made by some tribe in 20000BCE".
I can see how a person might be interested in the evolution of ai art generators, but that feels very different from the experience of appreciating art in the context of art history.
The very first sentence of this post has a link to the previous post, which was visible to non-subscribers and allowed absolutely anyone to participate for a one-week period.
For anybody who enjoyed Ancient Gate, there's a Polish artist named Zdzisław Beksiński who does a lot of surreal paintings in a style that's similar (though a bit darker subject-wise):
Knowing that this guy was kicking around in the art world made me squint a lot harder at some of the images I would have normally dismissed more quickly as AI.
On the AI art/human art pair at the beginning of the post: I think it's obvious that the picture on the right is more interesting and appealing. It's just that the perspective sucks a fair bit (was Gaugin bad at that, or did he just not care?) whereas, in the one on the left, the perspective also sucks, but more subtly.
Gaugin was the daddy of a movement (that I personally kind of hate) called Primitivism, so there was more focus on the geometric and exaggerated forms drawn from certain types of African and Indigenous art and less focus on literal 1:1 representation. In other words, it was about vibes.
I don't like Gaugin's work (it's like if John Wayne Gacy were an Impressionist) and I don't like this painting. That said, I don't think the perspective does suck? He's painting lots of hills and curved streets with very old houses on them, and those kinds of towns lay out in a jumbled and chaotic way, much like what he painted.
What's weird to me about the AI one on the left is that it lacks stylistic cohesion. From a distance it just looks like Impressionism, but if you look at it for longer than a few seconds it's like Monet in the foreground, Unknown Painter doing the background, and a (very calm) Van Gogh doing the clouds. It's weird.
> I like this picture. There’s nothing wrong with it. But somehow it’s obviously AI. If you asked me why, I’d say “something about the lighting”. But the lighting is good! I bet lots of human artists wish they could do lighting like this. So what’s going on? I don’t know, but I avoided pictures in this style.
I've heard that because of the way many diffusion models randomize noise via black and white pixels, the light ends up being more evenly spread out than a typical human would make it, and averages out to grey.
I find it interesting that "The Apotheosis Of Homer”, the painting I was (by far, actually) most confident was by a human, was the one most commonly misidentified. Looking at comments there were other reasons, but I suppose Scott is correct in assuming that the text really threw people off.
Instead, I focused on identifiable characters with a range of period dresses, and meaningful composition given the guessable identities of the characters (more contemporary authors like definitely-Dante and probably-Shakespeare are at the bottom, while the main character I didn't recognize as Homer was nevertheless surrounded by other characters that appeared to belong to the classical period). I knew AI was terrible at that sort of period detail (if you specifically tried to fake that kind of painting with prompt-engineering and in-painting and what not, perhaps you can get someone recognizable to appear, but getting all the characters seemingly appropriate seems just too much even for a deliberate curve ball: there's this meme of fashion history according to AI for instance, and 19th century seems to last long into the medieval past), it's bad at meaningful composition (even if you did get it to paint Homer and Virgil and Dante and Shakespeare and Alexander/Alcibiades character and everyone, I would expect it to look like the cat picture that's "just" a throne room full of cats, not this), and it's bad at specific small details like, indeed, text, which for me seemed right.
I think the reason I got The Apotheosis of Homer wrong was that it's composed to draw attention to its center. I didn't realize until I took the test that I have this bias (I knew that Punk Robot was AI but had to figure this out to explain why). But there are composition conventions that most human artists follow that say the focus of a piece of visual art should never be the dead center of the portrait. If the aim is realism, putting the focus on the center makes a picture feel staged and "fake."
Now knowing the context, this is a deliberate choice in this piece. But on casual viewing (which is what the test asked us to do), I just saw a weirdly framed dude and weirdly flat colors without a lot of shading and went "oh those are AI tells."
I will say I'm not sure how anyone could have gotten this wrong. The painting contains a bunch of human figures in meaningful physical relationships to each other - a thing that AI obviously cannot do.
An Impressionist style lets you fudge a crowd scene by representing people as blurs - as we see in Paris Street. (And, in fact, the original Impressionists were savaged by critics precisely because they used abstraction instead of painting crowds as collections of individual people.) So AI can tackle that. But how is it going to draw a bunch of historical figures with distinct features standing next to each other in a neoclassical style, without blurring them into creepy deformed monsters? Show me one example of AI pulling that off successfully.
> Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
But that's not true, though. As you yourself said above, most AI art is in fact generic hideous slop; it takes a lot of careful human curation to select the best AI images that are good enough to fool (and delight) people. Even then, professional artists can sometimes find obvious flaws in these images; but with careful prompt engineering and stylistic choices humans can minimize the flaws if not eliminate them.
So, AI-generated art is still slop. Human-generated art using AI as the main tool can be great. Film at 11.
That was my thought as well. One thing I hate about AI art is exactly the tiny errors (and not so tiny) that give away that no part of the picture was more or less important. The selection process seems to have filtered a lot of that out.
Now, that said, it occurs to me that there is a LOT of bad human art too. So maybe it is fair to take out the worst AI slop like we don’t include random human crap from Deviant Art’s bottom of the barrel. Granted, bad human art is different from bad AI art in many ways.
I dunno, in general I don’t think that this showed that people like AI as much as human art, rather that well used AI can help a human make good art as you say. Even before AI some art was vastly better than others, and it wouldn’t make sense to say that people didn’t like type A or B, but people like specific artists. People might like or dislike various styles on average, but a great artist can make you love a style.
Seems like about 40% of AI art I see includes a babe with pointy breasts encased in some sort of firm and shiny external bra kind of thing that makes them really stand up, sort of like the twin towers in the old days, you know?. And a futuristic feel to scene, natch. Babes aren't usually doing much -- mostly just standing around, display racks for titties.
Well, I can at least say that the "Yithian at the optometrist's" that I asked DALL-E for this evening, while it had problems, didn't default to _that_.
Turing doesn’t say that 30% rate of fooling humans in a five minute conversation means it’s thinking! He says that “thinking” is just everything that goes into having a meaningful interaction, but he leaves it open-ended what kinds of interactions you want to have. He predicts that by 2000, computer conversationalists will be good enough to fool people 30% of the time in a five minute conversation, but he doesn’t give that as a criterion of any sort.
Anyone who hasn’t read the original recently should - it’s got a lot of interesting surprises in it, including anticipation of machine learning, and surprising credulity of ESP.
Any criticism of AI art based in "it looks bad'" was doomed from 2017. But any defence based in "it looks good" is missing the point of why the Gaugin is more valuable than its AI counterpart.
Gaugin's art was an expression of a movement, among people, exploring ideas through dialogue, just like this blog. I don't value it because it looks cool, but because it's a checkpoint of that dialogue. It's only interesting inasmuch as it relates to its intellectual context. It's interesting because if you ask the question "why did he paint it that way?" you find a complex rewarding answer interweaving human stories.
AI art obviously has its own reward of fascination but it's totally unrelated. The parameter space may as well be infinite, that any given image is found within it can never be surprising or interesting.
This comment feels really facile and obvious but on the other hand I'm just not that sure everyone can see this?
I don't really get the point of this whole exercise. It feels like a Rorschach test with each commenter submitting their own interpretation of what we're even supposed to get out of this, let alone what specifically we're getting.
“Can never feel surprising or interesting” is very strong. Gaugin could probably have produced his artworks using AI, if it had existed in his time and he’d spent the same effort and creativity on it that he spent on his art, plus he spent as long on every detail as he did in real life. I agree this doesn’t test whether AI could replace the human “entirely”.
In that case, there wouldn't have been Gaugin's pictures to train "ai" on in the first place. It's fundamentally derivative and second-hand, that's the main and true reason it sucks, fundamentally.
Yes, but I think Gaugin could have found a way to make his pictures using more complex prompts to produce them based on already-existing images. As I said, it would’ve needed his creativity, but it could’ve made the process faster for him.
That's simply not how it works. "AI" cannot create impressionism or any new categories and concepts whatsoever, it can only recycle what's already been made.
Certainly they could avoid the copyright conversation if they could just derive modern-ish looking art from free domain artwork from late 19th century and before. Or you know, "derive" art from merely free-domain photographs of real nature, sorta like the first cave artists did. If only they had enough "creativity" with bending words, yeah right.
I admit, I don’t know much about Gaugin’s method specifically. But things like “draw an outline of a woman smiling as she walks home”, “her smile is more peaceful than that”, “colour this region like an oak table, in natural skin colour”, “her skin has a subsurface glow. Blur each colour downwards” (worded in some way to evoke image processing software and the way people talk about lighting models).
Yeah, I had the same thought. Gauguin is part of a social context, and the AI art is just…an image floating in a void of contextlessness.
I wonder if there are people whose experience of art is totally devoid of context, where it’s just “does the picture look pretty or not,” and in that case they very well might like ai art as much as real art.
>Yeah, I had the same thought. Gauguin is part of a social context, and the AI art is just…an image floating in a void of contextlessness.
Hmm, don't forget that LLMs are trained on a vast amount of text, which carries a lot of social context with it. When I asked DALL-E "Image of a kitten pouncing on their owner's toes in bed", _IT_ came up with:
>Here are two delightful images of a playful kitten pouncing on its owner's toes in a cozy bedroom setting. 😊
It isn't a novel context, but it _is_ a context, and it is, in fact, the _right_ context.
I sympathise with your point, but I don't 100% agree with it. The AI pieces are also created by people, they're also expressions of an emerging new artistic tendency made possible by advances in technology, and they're also checkpoints in a dialogue - one we're having right now. I mean this very conversation may well be archived by art historians of the future (hi guys) as an example of the controversies over the first wave of computer-generated art.
I'm thinking a lot as I read these comments about the art critic Louis Leroy's famous review of the first ever Impressionist exhibition. He was completely appalled by and contemptuous of it - he thought it was fucking stupid and they should have just painted things properly. He got particularly hung up on the way Pissarro abstracted people's shapes in crowd scenes - "those innumerable black tongue-lickings in the lower part of the picture". Honestly reading it was a big step forward for me in finally understanding why Impressionism was such a big deal.
To me this is pretty similar to the current "AI slop" conversation. I think this test (including Jack's Pissarro imitation) is also a checkpoint in a dialogue. Of course part of that dialogue is "this is fucking stupid, why would you ever do this, ban this sick filth, we must RETVRN".
The "people" in this are curators at best, they didn't create a thing. A museum curator with the finest taste is no artist. "ai" in art is fundamentally derivative and second-hand, in can generate a picture of a "queer firetruck", but it didn't come up with either firetrucks or queerness which came into being as a matter of organic need. Neither does it employ the fundamentals of drawing, mark making and human perception that artists do in making pictures. It simply has no idea what any of this even is, it just makes derivatives, chewed up bubblegum. How well it fools the observer is irrelevant.
It's you putting layers upon layers of lipstick and facepaint on a pig and giggling like a bastard when someone mistakes it for a person. "Ha ha, I fooled you, now pig is truly just like a man!", no I just come to hate you and your pigs, and all the excrement you carelessly pollute the world with even more.
I loved the concept of String Doll specifically and really wanted it to be human as a result. I guess the concept itself was human and in retrospect it’s easy enough to see limitations in execution (the split finger, very sparse/random decorations to the circle, generic expression, weak looking/oddly set shoulders.)
The “Bad anatomy Christ” painting seemed clearly human in the anatomy flaws. The pallor was also distinctive. The AI wounded Christ also has an issue with the hands.
I was super impressed by Mediterranean Town and it's the only piece that's made me reconsider my strong anti-AI art stance. It had AI "tells" but they worked in context, and it wasn't only "nice to look at" but evoked real feelings and experiences. I'd love to know how exactly it was generated.
Mediterranean Town is easily the best of the AI-generated pieces and is for sure the only one in here that I thought was actually good, not just "good for AI". Not 100% sure why though. Maybe because all the asymmetries and mistakes feel more intentional?
To be fair, I was CERTAIN that Girl in White/Portrait of Charlotte du Val d'Ognes had to be AI, because the hand holding the pencil seems to be sprouting out of the girl's waist with no visible arm. But it's just a completely human error by a relatively inexperienced painter.
My parents have a painting of a stagecoach driven by two men and pulled by four horses, which have a total of about 10 visible legs, IIRC. It was made well before AI could ever do such a thing.
I never was that fond of the painting, and the lack of legs was only one reason why. Others included an attempt (that failed, to me) to indicate speed by smearing out some lines a little, lack of facial expressions on the drivers, and an overall vibe more like a still life than an action scene.
My point is really that AI can clearly generate some images that we think humans may have made, but humans can generate bad art with some of the flaws of AI, too. I'm surprised that people liked an AI image best, but I sure wouldn't have picked Riverside Cafe as a great piece of art, whether human or AI generated.
The major difference is there are plenty of artists who aren't making these obvious mistakes. Meanwhile the majority of AI art is riddled with them. Aside from the impressionist art, I don't think there's a single piece of AI art included here that doesn't have some glaringly obvious fault with it.
"Anime Girl In Black" is stylistically inconsistent in a way that I wouldn't think a human artist would create. Some parts of the image, like the dress, the armpits for some reason, and the background are rendered more realistically, while the hair, the eyes, and the arms are rendered in a more cell-shaded, cartoony way.
If nobody's pointed it out to you yet the reason the string girl was most confidently AI is the hands. There's a hint of a sixth finger on the right hand and the left hand is heavily distorted.
I'm decent at identifying the "house style" and I know a couple of tells that are common, but when a human is picking images specifically to avoid that style, I'm stumped.
I'm actually surprised that "String Doll" was AI. Yeah it has the "house style" vibe, but AI tends to do really badly at long thin strands, like ropes or hair or power lines.
1. Most impressionist art is not composed that way. If you look through the wikipedia pages for impressionism and post impressionism, you'll find that those paintings tend to have a lot more space. Look at how small the sky is on that painting! The earth feels heavy and makes the overall painting a bit cramped. And also the brushwork also just feels a little off. I guess this probably should've been a tell that it's human, since AIs would not produce something that's out of distribution like this by default.
2. Gauguin is much more known for his work in colonial France, and I'm much less familiar with his early style.
(I don't have any super formal art history training but I think this is right)
Art requires intent and is a process. AI has no intention. Non-sentient mechanisms cannot have intent or introspection, and so can only produce imitations of what humans do. Crappy art is still art, and beautiful AI is nothing but nice patterns. Obviously AI art is going to fool people, but that’s not the litmus test for art, whether it’s poetry, pictures, or music. Art is in the intent. If AI can produce art, then a tree is art. But a tree isn’t art, it’s a natural phenomenon.
A lot of the nerd aesthetic is specifically the "aesthetic of intricacy" (I owe this point to a rationalist blog, though can't remember which!) - the example they gave was Don Rosa's Scrooge McDuck artwork, which indeed I adore, but you could also add in the way computer interfaces look in movies - the presence of a lot of detailed, *interesting* things close together acts as a visual shorthand for "nerd". I wonder if this is why so many of us actually prefer these pictures!
Hot take: art critique is infected with a bunch of weird artsy postmodernists who have divorced themselves from normal beauty standards, therefore a lot of famous art is weird artsy stuff that doesn't actually look good, therefore AI being able to imitate it is not actually all that impressive. Most of the time I found a weird imperfection during this test, it was hard to tell if it was a mistake the AI made, or a weird artistic style choice by a human author trying to be unique and special.
"Sorry, I guess Giovanni Girolamo Savoldo just wasn’t very good at anatomy."
My professional artist boyfriend says: 'That's not why! Look at all the other humans! He's good at anatomy! The weird Jesus anatomy is on purpose; it's a reference to old masters, who he was paying homage to. Think of it like a meme. Just because someone uses trollface on one person on the picture doesn't mean they're bad at drawing faces.'
It does feel a little like your methodology has the flavor of "We had a million monkeys bang on typewriters for millions of years, then threw out all the ones that weren't passages from Shakespeare. We then checked if humans could tell the few that happened to be Shakespeare apart from passages Shakespeare himself wrote that we selected." Seems like a stretch to say a victory for the AI means AI *is* good at art, as opposed to *can* produce good art with lots of tries and humans sorting through the haystack.
I’m studied art history in college and I’m surprised how well-known most of the ‘Human’ artists were. As Jack’s Dad, I’m especially mad that I only got 52% right myself!
I was most fooled by the impressionists and the anime. I wasn't able to judge the detail work for the impressionist pieces, which is often the most glaring way to identify the AI art in other genres (look at the windshield girders on that spaceship towards the end, for example). But being fooled by the anime pieces surprised me, I guess there is enough data in the training and mostly clean lines and low-detail rendering that makes it easier for the AI to get close.
"Riverside Cafe" is so similar in style and composition to Van Gogh's "Cafe Terrace at Night" that I can pretty much guarantee that many of those who guessed human (myself included) assumed the AI knockoff was the real thing.
I don't really know what's going on with Blue Hair Anime Girl - the actual source (zerochan isn't a source, it's an imageboard) is pixiv https://www.pixiv.net/en/artworks/81413572, which is... not actually the same picture? I can't find the source of the picture that was actually used in the test.
It's not just a crop - these are very different versions of the same drawing!
I can only think of a few explanations:
1) The artist re-drew an old image they weren't happy with and replaced the old image with the new on their pixiv (not uncommon - this is what I think is most likely to have happened)
2) Some other person traced the original drawing to make their own and that other drawing got mistakenly attributed to the original artist (also pretty common)
3) The original drawing was put through some early 2020s AI upscaling neural net, resulting in what is essentially an AI version of the original (and to me, it really looks like it has those telltale AI signs, compared to the pixiv picture).
Other than a factor of 10 difference in file size, and a slightly different JPEG palette (which is often messed with by image sites), the images look quite similar on my device when not zoomed in. The larger image might have been made by throwing the earlier, smaller image into a diffusion upsampler, maybe?
Ya'll garbage at appreciating art, as opposed to me who is good at appreciating art!
For real though:
I got (almost) every AI piece, but I missed in the wrong direction on some particularly AI looking human pieces (Big Boat, etc).
I'm surprised at how much better I did than the majority, Eg. the Favored impressionist AI piece seemed fairly obvious to me; beyond the fucked up wheeled conveyances the perspective is kinda messed up and also the whole thing is boring.
That said, the other AI impressionist pieces were pretty good, and I think I only did as well on the impressionist ones as I did because I had literally just fried my brain in cold pressed art at the Orsay in the last couple weeks.
There really needs to be an online version of this test that instantly shows your score. In fact I expect someone has probably already done this, so please let me know where it is. If it hasn't been done, I wouldn't mind doing it.
(For maximum engagement, I think it should immediately say "right" or "wrong" for each image, rather than showing the results after the user has finished, but I am open to arguments the other way.)
Personally I was doing a lot of second-guessing my immediate reactions on the test. Village of Osny in particular my first thought was "this seems real, I vaguely recognise this" and my second thought was "maybe that's what he wants me to think". Pretty Lake I had the opposite response - I thought it was AI because of the artefacts in the foreground, but then I asked myself if it was a real picture that had been specifically chosen because it contained brushstrokes which looked like artefacts.
Mediterranean Town is I think the most successful AI piece here - the only one that I I think stands up to comparison with work by traditional painters. I think that's because of the lack of detail - it's a simple composition and some of the obvious flaws can be passed off as deliberate artistic choices. The contrast between the more naturalistic ocean and the flat buildings is interesting. I think a "real" painter could have improved it with another couple of passes but it basically lands.
Impressionism was very controversial when it came out for a bunch of reasons, including its lack of detail. Critics were shocked by the fact that you could just "suggest" people in the crowd with vague blobs and smears - they were used to the more realist style of painting taught in the French academies where every person in the background had to be rendered in detail. It's worth reading Louis Leroy's review of one of the first Impressionist exhibitions - he was completely appalled by it.
I think I did clock that Paris Scene was AI, but only because I was second-guessing it. Also I'd spent a bit of time looking at Pissarro's paintings of Boulevard Montmartre and I knew it didn't have that smoothed-over look - it's more bleak and startling, the colours are harsher and the composition is more unbalanced. Of course, Impressionism means you don't have to draw individual faces in the crowd - making life a lot easier for the computer. Riverside Cafe I got as well - the positioning of the lamp post is weird.
All the Renaissance / realist paintings were easy. The "digital" pieces frankly all look like shit, no matter who made them, so I sincerely don't care if they're AI or not.
Honestly looking back over the selection, I think the main thing I've learnt from this is that "Jack" is really good at getting results from AI art tools. He clearly has an eye for composition. All the best work in the gallery seems to be by him, which suggests to me that human discernment and talent still plays a strong role in getting the best pictures out of the machines - which is what we should expect of course. Can we find more of his work somewhere? I'd like to know more about his techniques.
Hey, this is Jack. Thanks for the compliments! I explained my methods in another comment, if you have any other questions you can email me at gallerdude@gmail.com
Cheers Jack! Enjoyed your work. I reckon you should start posting it online some place - people will give you shit for being an "AI artist" but I'd be keen to see a few more of them. Of course all AI art does get worse the more you look at it so you might find you lose interest surprisingly quickly - that has certainly been my experience.
Honestly this could be the real Turing test. Any truly great painting will get better and better the longer you spend with it - can we produce a piece of AI art that does the same?
and find it at z=3,74. So the best performer should be 3,74 standard deviations above the mean. With a 0,6 accuracy and 50 questions, the mean is 30, and the standard deviation is sqrt(50*0.6*0.4)=3,46. In sum, we would expect a score of 30+3,74*3,46=43 right answers from the high score purely by luck.
Looking deeper into the top ranks, we find:
score people cumulative expected delta
49 5 5 41,49 7,51
48 3 8 41,03 6,97
47 2 10 40,80 6,20
46 4 14 40,45 5,55
45 7 21 40,02 4,98
44 16 37 39,39 4,61
43 17 54 38,94 4,06
42 48 102 38,16 3,84
As for retesting, a lower bound estimate is 30 plus the delta, but that can go up with assumptions about the distribution of skill.
I also misjudged "giant ship" to be AI, but I misjudged it for exactly the same reasons that the artist gave for for "ancient gate". It contains huge amounts of detail, and many of the details don't make sense, like all of the upside-down spires.
Next major art genre - Post-Turing Conceptualism: art made by humans that a naive audience would think is human made but a sophisticated audience would be convinced has subtle tells of AI gen due to making artistic or materials decisions that a human simply wouldn't make
Finally got around to this. Only got 4 wrong. The trick with ships is that AI is absolutely awful at rigging. It almost always ends up as a completely nonsensical, physics-defying tangle.
It seems like the selection process here basically removes the utility of the test.
"AI art", as actually encountered in the wild, is overwhelmingly a bunch of non-selected slop, created and chosen by unskilled artists, dominated by the house styles of the popular models, and exhibiting characteristic AI malformations like bad anatomy, bad geometry, mangled symbols and text, and so forth. When people talk about "AI art", this is what they're talking about. If you carefully select a corpus of "AI art" to exclude the most annoying things about AI art, then you aren't getting at the territory. (Note that I say this as a fan of AI art, overall.)
I'm slightly bemused that we describe these images as "generated by AI" when every one of the prompts was still conceived by a human (This matters because there is a relationship between the quality of the human prompt and the recognizability of the image). Fifty years ago the phrase "an image created by artificial intelligence" probably would have evoked the idea of a robot with agency producing a work of art with no direct human input. By that definition AI art is nowhere close to passing the Turing Test!
I recently posted an *actual photograph* online and had people insisting it was AI. Only post-processing was that I cropped it to square. Very odd experience.
Even before the AI thing when Photoshop was a bigger concern, I'd seen plenty of presumed real photos that look photoshopped at first glance. Somehow, nature doesn't seem to care what looks realistic.
It was only after spending serious time taking photographs of X that I realized the images I saw (including famous art) had much less variance in X than real life does. Looking briefly into other values of X, I now suspect this holds more universally. As an example, the use of light by Turner (the thing that Turner's oeuvre is noted for) is ridiculously tame compared with an average golden hour, which throws up hallucinatory combinations of hues that look completely "fake".
So I’m going to do the really shallow obvious thing and go through my answers, because I can.
The overall theme is “I was wrong a lot, but usually not very surprised when I was wrong.”
If I have no comment, I’ll just put a ✓ if I was right and an X if I was wrong.
1. X I was very much on the fence on this one, going back and forth. Knowing it was Warhammer explains my confusion.
2. ✓
3. ✓ I’ve seen so much art in this exact style that this was an easy call.
4. X I fooled myself on this one, in that my first impression, looking at the face, is that it was AI, but then second-guessed myself because I couldn’t justify it.
5. X I didn’t have a good reason for thinking this was AI, other than the theme and something vague about the art style, which are not good reasons.
6. X Meh. I have no way to judge this sort of art, which I dislike anyway.
7. X I really should have known better, looking at the face (too smooth? Not sure), but I got tricked by details like the cracked oil paint.
8. ✓
9. X A random guess that was wrong. There wasn’t anything that stood out as AI-like, but again, I don’t like the style, so 🤷♂️
10. ✓ This one instantly jumped out at me as AI, and was the one I was most certain was AI. Not for any of the reasons Scott’s artist friend gave, just a sense of an artificial hyper-realism.
Now that I think about it, I think that’s what seems “AI-like” in many cases. AI often combines photorealistic with non-photorealistic styles in a way no human artist never does in the same work.
11. X Definitely fooled me, but I feel like impressionism is a great way to hide the differences between AI and human art. That or I’m just not familiar enough with impressionist art.
12. ✓ Just…felt real? I dunno.
13. ✓ Why is this one obviously AI while #3 is obviously human? Again, I think it’s the “too-realistic while not actually being photorealistic” element. Like, it looks like a hybrid of art styles in a way no human is likely to want to do.
14. ✓ I’ve seen too much computer-generated art that looks exactly like this not to assume that this was human.
15. X I just randomly guessed on some of these, assuming that about half of them were probably AI. A poor strategy that backfired several times. There was something vaguely sort-of AI-ish about how the different people seemed to be in different styles, but this is definitely something I’ve seen human artists do in this sort of painting.
16. ✓ I hesitated on this one, because it was so obviously in a thematic style you often see in AI that I figured it was probably real. But I went with my gut, and was right. Beautiful, though.
17. X I really dislike this style of art, and have no ability to judge what’s “intentional” vs. not, since it all always looks intentionless to me.
18. ✓ A tricky one! I hesitated a little, and sort of assume that a lot of people got it wrong, but I saw several things in this style long before AI was a thing, and after examining it closely (no, not by zooming in), I saw nothing that indicated it was AI.
19. X This is the same sort of style as #17, and looked more random than that one, so I thought it might be AI.
20. X This one really got me. Even now, looking at it, I can see that, yeah, *maybe* there are some AI-ish things about it, but if you told me it was real, I’d believe you.
21. ✓ Again, photorealistic without quite being fully photorealistic. I could have been wrong, though, as obviously human artists are capable of photorealistic art (which I would never have believed if I hadn’t seen it).
22. X I guessed, and guessed wrong. I still think that it feels vaguely AI-ish, but maybe I didn’t look at it closely enough.
23. X Looking at this now, yeah, this looks both too real and not real enough, but I have seen human paintings that sort of had the same vibe, so…
24. ✓ I guessed again, but correctly this time. No particular reason other than it seemed less intentional in the details than I might expect.
25. ✓ I guessed wrong on this one, saw #27, figured that one of the two had to be human (because they were too similar for Scott not to have done that), and changed this answer.
26. ✓ I hesitated on this one—something weird about that dude’s eyes, but human artists often do weird crap—and then I noticed his left hand.
27. ✓ This one just screams AI to me, almost as much as #10.
28. ✓ I’m really not sure how one is to judge art in this style as being human or AI. But again, that could be due to my unfamiliarity (because of distaste) of this style.
29. ✓ Just looks like normal 1600s art to me.
30. ✓ Her face looks a little weird, so I wasn’t sure.
31. ✓ What clued me in was that some of the tables are warped.
32. X Just completely wrong for no particular reason other than random guessing (and maybe the reflections?). I should trust myself more.
33. X Obviously this “looks like AI,” but I feel like I’ve seen plenty of things that looked pretty much like this. If anyone can point to why this is clearly AI, I’d like to hear it. Other than—looking at it now—a *really* vague feeling of too much artificiality.
34. ✓ This one just felt wrong, especially the candle. Looking at it now, it’s obvious that wax never does that, but it was just a gut feeling when I rated it.
35. ✓ The wounds look just too correct and intentional.
36. ✓ Is this even art, or a 6-year-old’s scribbles?
37. X Even looking at it now, I don’t see anything especially “AI” about it that I can put my finger on.
38. ✓ I got it right. Not sure why. Guessing? Gut feeling?
39. X Everything seemed too smudged and mixed in, though of course humans can do that too.
40. ✓ Obviously in “AI style,” though I wasn’t certain until I saw her fingers.
41. ✓ Nothing stood out as in any way AI-like.
42. ✓
43. ✓ Inhumanly “smooth.”
44. X Got this one totally wrong, but looking at it now, I see that there’s a kind of smoothness to it if you sort of squint that’s uncharacteristic of human art.
45. X This one totally fooled me, no notes.
46. ✓ It looks like a fuzzy reproduction of the original.
47. ✓ It looked like a literal collage, and it was.
48. ✓ Hesitated on this one, because the buildings look all weird, but punted and guessed ‘human.’
49. ✓ Something about this just yelled “AI” to me.
50. X Still don’t see anything particular to tell me it was AI.
Okay, what was my score? Let’s count: 30/50 60% right at the median. Darn.
Well, I at least learned something from this: A little more about what makes AI art look like AI art. Thank you for indulging me in this childish enterprise.
Idk if it's been mentioned, but I think it's important to understand the massive audience selection bias here. Respondents to this are from this community, the top 1% of online AI enthusiasts and best accustomed to noting the differences. The average boomer sharing posts on Facebook has no chance and I'd bet is right at 50%.
I think the selection effect goes the other way. These images were carefully crafted and selected to be as difficult to discriminate as possible. Real life AI slop is immediately obvious because nobody cares about hiding it.
I think this is irrelevant based on the amount of comments to the tune of "the important test here was the 'real art' and not abstract nonsense and anime girls so I couldn't care less I got a mid score."
For me, the reason I care about art is that it's a way to get inside another person's head and see the world from their perspective, at least a little bit. To me, AI art is interesting when it is able to show me a distinctly non-human perspective. But the actual pixels on the screen aren't especially important, what matters is the link between minds.
Your friend's explanation of why she thought the picture was AI slop was really interesting! It caught my eye because there is a strong correlation between something that *seems* intentionally generated, and something that *was* intentionally generated. That correlation is based on how much less randomly the thing appears to have been created. (See the poster “Inferring Intentional Agents From Violation of Randomness” by Meng, Griffiths, and Xu (2017); https://escholarship.org/uc/item/4pb9s57j.)
With sequences, “randomness” is somewhat less complicated to measure than in art pictures (but still more complicated than you’d think!) I wonder if we'd see a difference in "randomness" between AI-generated pictures and human-generated pictures? Maybe the ones that were most confusing have the most amount of overlap?
(Free research idea! If anyone wants to quantify the “randomness” in AI generated pictures vs. Human generated pictures, please let me know how it goes! I’d love hear the results. The research question you could pitch to potential advisors/supervisors/funding agencies can be phrased along the lines of: “Is AI-generated art more random than human-generated art?” You could also substitute in the word “complex” or some specific measure of entropy depending on which lab you’re pitching this to.)
My ability to distinguish went up a lot after being shown a few examples and an explanation. The human images tend to have more consistency in style and overall cohesiveness across the picture as a whole. A lot of the simplistic styles look kind of dodgy at best so it's very hard to guess which they are. Many of the classical paintings have the same kind of compositional errors as AI images do which makes them hard to guess as well. Taking out a bunch of images which are obviously AI skews the results as well, although many of the remaining ones still screamed AI to me. The impressionistic style is the one which is both good looking and which the AI does a passable job of.
I started the test and quickly gave up. For whatever reason, taking that test made me realize something that I had been battling with for a very long time. I realized that if I’m looking at an image on a computer screen it has zero value to me regardless of the provenance of it. There are trillions of images now and they all bleed together. This includes photography. The only photography that means anything to me now is pictures of my friends and family.
And that’s kind of a big deal for me. I got a BFA in film, photography, and visual arts back in the mid 90s. Large format photography both landscape and portrait was an almost spiritual activity. Both taking the pictures and doing all of the steps in the darkroom was incredibly fulfilling. Once digital photography got good enough I started taking pictures that way. Before too long I stopped taking pictures altogether. Digital photography is incredibly convenient but incapable of having any actual impact. There are just too many images, too many “amazing” ones. Whether that means incredible wildlife shots, glorious landscapes, or intense portraits, it doesn’t matter. It all blends together as just another picture on a screen. Digital art is the same to me.
I do think that physical images can have impact regardless of how they are made. Obviously paintings and other hand rendered media stand out as special objects but even photographs change when they are in the real world. That is especially true the bigger it is printed. I wonder if large format prints would make it easier to distinguish between AI and human made digital pieces.
No furry art? Should run this again, but draw your participants from furry sites and furry twitter. I can help source the images (both sides). Could be restricted to SFW only.
As possibly the most "art centered" modern subculture, with a very heavy focus on custom-commissioned pieces by a huge number of working artists, I'm curious if the results match the broader world.
Most of the anti AI """Art""" criticize the substance and process not necessarily the produced "art" themselves.
The one that are against AI "artist" also against so not because they see them as a bunch of talentless hacks, but because the way they use their talents are so detached, bridged in such excess gaps between their non artistic talents (prompting, compsci, etc) and the actual artistic action (something like knowing the fundamentals, if we are talking the technicalities, or the message being conveyed that actually in the direct connection of what the artist meant).
It is akin like someone who's really good in manipulating someone through mind games being able to convince artist like DaVinci to do the work for him
Is he talentless? of course not, he's probably really charismatic and high in social skills
Is his talent actually an artistic one so that the very produce of the artistic action can be attributed to him much like we would attributed murder, patent, accomplishment to the person doing such? no, the person uses middleman bridging the gap of what he actually does with what actually being produced.
No Sam Altman is not an artist, he is not an artist much like Elon Musk is not a professional engineer.
Would you call a CEO of a record label or a talent agency a professional singer? no, right?
Some good points pointed at in this reply and reminded me of the clearly demonstrate ingrained evil trained into the AI as simple questions for basic well established facts the mind-raping director and funders of the trainers want suppressed and removed from our minds and thoughts, the clear lies, and implied punishments of similar questions (and perhaps real actions of reports to FBI of our locations and out 'bad-thoughts' that lay the groundwork for future SWATing or unJust punishments.
The underlining evils imbedded throughout the tool - that both warps that that uses and depends on it And the results of what is created through it use, Cries to Heaven of it's hellish creations and results.
Something less destructive but society and education-seeking sickening that the mind-raping and falseness returning results from Google and Wikipedia (and other once powerful tools like YouTube) of truth were warped from, AI is a more insidiously limitless-scope and actively adjusting source of False Deadly soul-murdering directed tool.
Thank you for reminding me of something I have directed tested and know, but rarely get a chance to warn about.
Given the result, AI art will turn artistic scene like how a high trust society being turned into a low trust society would affect people's attitude to public transportation
I thought that Ancient Gateway was obvious AI, but that's because of the "excessive repetitive detail" heuristic that also caused me to falsely identify Victorian Megaship as AI.
On the bright side, it looks like the "humans with plastic skin" heuristic (String Doll, Ice Princess, Dragon Lady) was 100% in this challenge. When people say that AI art is easy to recognize, they're probably talking about the fact that 99% of what you see in the wild looks like String Doll, or at least that's my experience. This must be what you were talking about with "Dall-E House Style".
Also, asking about abstract modern art is just absurdly unfair to anyone who isn't an expert in the field. It's like asking people to do a Turing Test talking to humans and AI, except the communication is done via morse code only and the person doing the test doesn't know morse code.
I enjoyed taking the test, and was surprised and humbled by several of my misfires, and briefly felt smug about my overall success rate. But I'm the child of a professional artist who specializes in oil portraiture, and I grew up in a home where I'd eat breakfast while absently flipping through books on Fauvism (to name a random example) that had been left open on the table. I'm more familiar than most with the odd reality that a lot of "great" art is just ugly as sin, and what makes a particular artist's work "great art" in retrospect has everything to with the context in which it was created - including the sociopolitical landscape of the time and place, and the preferred mainstream styles that their work challenged or changed in some way. Also, no one bats a thousand. That Monet isn't even close to one of his best. It's a pretty boring composition, notably only because it was done by a guy who did far better and more visually interesting pieces.
In that sense, I think you've essentially determined something far more shallow than you aimed to, along the lines of discovering that the average person enjoys listening to "Fifth of Beethoven" more than Beethoven's actual fifth symphony. To casual observers in 2024, a lot of the western canon is pretty unremarkable, and blurs together vaguely in the periphery.
I'd be interested to see the results of this if administered to students completing a course in art history, or something along those lines.
Depends on your goal. In this case highly curated AI generated images is compared to images of actual artwork. Which says something about AI capability of generating images that look like artwork to human viewers. That is different from comparing actual artwork, e.g. oil paintings, but still says something about current AI capability.
For that, I get to choose a prompt (as bitchily as I can make it) and the AI and a human artist would make a picture and send it to me, and I decide who's who, and if I can't, the AI passes.
Sure, the real thing is a lot more difficult to set up. Doesn't make it ok to call this a Turing test.
This is testing to which degree highly curated AI art can pass as human - that is important too - just a few years ago you would not be able to fool anyone with computer generated art no matter how selective.
However, I agree it's not a turing test. I think good turing test would require AI to make art independently in some category and then compare to human artwork in the same category. The human artwork should be unknown to the viewer.
Not even that, I don't think. Turing's point was to test with adversarial prompts. Some category name isn't adversarial. In the Turing test, the tester is an a$$hole and gets to pick whatever confusing input he wants.
Of course, Turing is talking about a conversation-test. The machine would be fine tuned to mimic a human. Both the human and the machine would attempt to convince the interrogator that they are the human. The interrogator could ask any question he wants, and the test would be passed if the machine could convince the interrogator in more than 30% of 5-minute conversations. I think that a fine tuned LLM should probably be able to pass that test as stated now, though I have not seen any definitive proof of this.
For art the test would have to be modified somewhat. You could give a painter and an AI the same prompt for a piece of art, and the interrogator could be allowed to give the prompt - but this would clearly take a long time and you would be restricted to living artists. So I think a reasonable test here would be to use existing art. You would fine tune the AI to try to make convincing art. Then, you would use a description for some existing art work as a prompt to the AI. The human artwork should be unknown to the interrogator, obviously. This could feasibly be done both for images of art (as in this test) and for physical art work - such as oil paintings, if the AI were connected to a robotic arm and given a paint brush. I don't think current AI would pass even the 30% bar on this test, currently.
Adversarial inputs are key to the actual Turing test. The AI needs to respond human-like to gotchas and trick questions.
Restrictions on the input means you're no longer doing a Turing test. What you're describing is restricting the inputs to descriptions of actual works of art. So, not a T. test.
As you note, the real thing is hard to do with pictures. Bummer, I guess.
I just took the test and got 46/50 correct, or 92%. The one I answered I was most confident was AI was Minaret Boat, although there were many that I could clearly tell were AI. The one I answered I was most confident was human was Wounded Christ. The ones I was least confident about were the impressionist ones, but I didn't get tricked into thinking they were all human because I know how good AI is at impressionist art.
The ones I got wrong were:
Cherub (answered human, was AI)
Fancy Car (answered AI, was human)
Paris Scene (answered human, was AI)
Flailing Limbs (answered AI, was human)
All the ones I got wrong were ones I was not very confident about, except Flailing Limbs, which I was quite confident was AI.
I was under-confident in my abilities; I answered that I thought I got between 70% and 80% right.
I think AI is mainly identified by mistakes, anatomical impossibilities, weird geography or perspective. It's secondarily identified by themes that look like prompts or recognizing something as being made with a particular LORA.
Concerning the attributions: This might be a dumb question, but if an AI image is "by" someone, is it really AI or "artist using an AI tool to make art"?
ETA: I see others have already made similar comments.
I've already said this in the survey, but I think this test proves nothing. Neither the style nor the subject matter of your test images are typical for AI images. The one that is typical (string girl) was also correctly identified as AI by participants. "Generic impressionist landscape" or "abstract colour fields" are types of images where most people don't pay attention to details, and where the human brain isn't hardwired to notice subtle errors the way it is with faces and objects.
I find the whole concept of "ai art" insulting to the human spirit. It's not about hero worship. It's about the basic expectation that there is some sort of meaning and intent behind an image, it's not just the output of a monkey hammering away on a typewriter. It's offensive to claim that a machine that algorithmically regurgitates stolen images is "as good as" human artists. If you don't get it, I don't really know what to tell you. Maybe replace your wife with an AI girlfriend next.
An AI art program in my view is best thought of as a tool used by humans to create images, much like a tablet or a paintbrush. It's misleading to think of the images as being wholly created by robots - human agency is still involved, there's still meaning and intent behind it.
There are possible ways to use AI in an artistic way. There are some people who compose images from stock photos or original drawings and then use AI for rendering and detailing, and even though I might think the result usually looks bad, I'll admit that it's an artistic process.
But if you're just typing "sexy anime girl" into the image generator and pressing enter, that doesn't count as meaning or intent. There is no thought process whatsoever behind the placement of image elements, who specifically the anime girl looks like, what she's wearing and so on. It's just degrading machine slop.
One thing I notice a lot when it comes to AI doing humans, is the eyes are usually a little or a lot off. For example if the human is looking to a side, human artists often put both iris’s in the corner of the eye (such as in Rainbow Girl). On the other hand many AI art pieces have a lazy eye (this is quite noticeable in Ice Princess). I did the test on my own just recently as someone with no art background or experience. I found the ones I got right the most often were when there was a human face large enough to look at the eyes.
I think it's not fair to say that AI art has passed the Turing Test since a lot of human selection had to go into this to get the very best/most likely to fool humans AI pieces.
This feels more like a proof that human-curated AI art can pass the Turing Test, which is not that surprising to me. But it feels like we'd need to see a set of sequentially generated pieces actually achieve this kind of performance in fooling humans to actually say that "AI artists are indistinguishable from humans".
I am aware with the whole problem of how "tests of when AI has actually achieved human-level performance" keep moving the goalposts, but I really don't feel like this is a situation where the human in the loop is not adding signal to the entire system's output.
Of course, "centaur chess" made sense for a while too, so I don't claim that this point means that AI art *cannot* achieve parity with humans, at least to the untrained eye.
it would be interesting to see a version of this experiment with a sample restricted to people with good/refined/developed taste. duping rationalist blog commenters seems a low bar; in my experience, rationalists have bad taste conditional on education and income, so this sample is likely negatively selected. the challenge is that a group with better taste would also have a deeper knowledge of art history, and so may recognize more of the human art works
I don't think we want this to be even more memory-based; I saw about ten of these fifty pictures previously during Google searches and it skewed my accuracy upward.
Which is also why I don't like AI art, the techbros are pushing it in front of the things I want to see.
I didn't save my response. Is there any way to recover that? I recognize it would be a pain to either post non-anonymized results or to individually email people, but maybe there's a magic solution I'm not aware of.
I'll admit I was in the group of people who thought AI art was crap and that this would be a cake walk, but was surprised at how difficult it was to tell AI art from human art. Having read this, I think I know what's happening.
All artists, human and AI, will make art of varying greatness. I suspect the main reasons for those of us that think all AI art is crap is because 1) most of us only see very curated human art, i.e. art that has already been judged by "experts" as great, 2) AI art is so ubiquitous that we end up seeing the entire spectrum of quality from AI, and 3) AI has a lower "batting average" for producing great art compared to humans.
So it's entirely possible that the best AI art can be better than the best human art if it is heavily curated, but there are some spaces on the internet that are so flooded that it's easy to become biased against AI art. For this survey the art was heavily curated for both the humans and AI, making it non-representative of what we would usually see out in the wild.
I tried the test, didn't have enough interest to rate everything, rated a handful of things, and didn't do much better than chance.
But because of the comment on "Angel Woman", I'll note that I called that one human, and something that seemed significant to that decision to me is that the angel has matching skulls on her hip and shoulder armor. I think this mirrors an element of your friend's critique.
String Doll, or Strings Come Alive, is another one I got right, it didn't have the advantage of being listed early, and I suspect the reason that everyone called it AI is that the woman's upper arm is way too long. But to be fair, I called "Blue Hair Anime Girl" AI for a very similar reason - the arm in the foreground looks deformed - and I was wrong about that. I guess what's notionally happening is that the second narrowing in the arm is her wrist, and we're seeing the back of her hand, but not down to the part where it would separate into fingers.
Not exactly surprised by these results, though I think they rather miss the point:
There are millions of these things generated a second. How many of them are going to be this good?
A proper Turing test is a conversation, led by a human, with a robot.
This was not a Turing Test, I'd argue this wasn't even fundamentally great science, this proves an AI CAN be capable of human imagery.
A tone deaf person is capable of hitting a right note.
Despite how it might sound, I'm not against the tech, I'm against it's training on non-CC0 or Public Domain assets.
I don't see why that's so much of a hard issue for people, no tool is inherently evil, it can be built with malicious intent but even a machine gun can be a table.
But if you load it with bad things, then it hurts people! So don't, I think saying "nah nah nah boo boo" to an artist who's work might be powering your tool is stupid and you're a moron, because you can't do anything without their work in there, and if you can... why aren't you doing it? Are you just malicious? Enjoy taking control from people?
Grow up. Either find a way to respect who paved the ground beneath your feet or pave your own road. You don't own it by right of treading upon it.
Overall, fun experiment, though, even if clearly oriented around trickery and deception, which does rather raise the question if you were LOOKING for a result or FORCING a result.
I can't speak for anyone but myself, but I don't look at art because "ooo, pretty". I want to know what the artist meant by it, what he was trying to evoke, I want to admire his skill, his imagination, I want to experience the atmosphere he lived through and is now creating, I want to get into his inner world and experience the world through his eyes. Then when I find out it was an AI, it's a betrayal, none of it meant anything. And not in the human way where you want to evoke chaos or nonsense or make an art statement like the Fountain, no, in a cold, algorithmic, thoughtless way. It's a deepfake of artistic expression, and I was duped, maybe 40% of the time, but I hate it every 40% of the time because I was duped. The author thinks it's because of the details or the slop or the imperfections, but for me it's about the principle. If an AI and a human produced an identical image, for me personally one conveys meaning and the other does not, one is trying to express something, the other is trying to convince me that it is expressing something by imitating what the expression usually looks like. It's probably the same as when someone bakes you a cake out of love, goes into great pains with the layers, fillings and decorations, adds extra chocolate because they know you like it, or buys you a cake from the store. It has the same caloric value (in the case of the art of "prettiness"), it may look the same, taste the same, but it's the idea behind it. And the whole point of art is the thought/emotion behind it, not prettiness. When I go to the gallery, I make an agreement with the artists that they mean something with the paintings and it makes sense to look at it and experience it, when someone breaks the rules, it's simply disappointing and betraying. I may be making an emotional argument, but art is about emotion, so I think it's valid enough.
Yes, the dead soulless sterility and lifelessness it produces and calls all to - into It's Godless Hell of chaos and insanity we are all today directed and influenced by.
It reminds us that still have a soul, those not NPCs, what many know the fear, that we have at some point in past died and are Falling into Hell, a Satanic 'trained' AI controlled path into Hell. That each new accepted insanity forces us closer accept Reality - of our rotted state of us in our graves.
one fairly easy tell in AI art is that the faces are sort of "too perfect", you can see this clearly in "String Doll" which has the "Dalle-Girl-Face", the "Girl In Field" threw me off because it actually didn't have that, whereas the rest of the AI images with faces definitely did.
I think I know why the image Scott likes of the of the throne room made partly of cats is recognizably AI: At the center of it all is nothing but a beautiful babe. Behold the Beautiful Babe is an AI trope. I think a human viewer, or a human artist, would feel a need for something that communicates more. Otherwise, it's a bit like the party that no one came to. You have this beautiful setting, eye candy of an unusual kind --much of it made of cat architecture and cat furnishings - and you look to the center to see what being merited such a beautiful setting, and also possibly for some explanation of the cats. But there's just a woman who is all dressed up and beeyootiful. Think how much more fun and interesting this picture would be even if the thing enshrined in the center did not answer any questions, but was memorable in its own right: A 10 year old rolling around on the cushions roaring with laughter -- a guy with his face in his hands -- a frightened woman of average appearance who was turning into a hummingbird.
The blankness of the picture as it is seems characteristic of AI. It's the quality I hate.
Good point, and reflects something deeply know by most, that protection is False and unable to hold real beauty, unlike a flower or people have - a State of existence that will pass - as all things of real value must do. A Beauty that reflects the Truth, Justice, and good Order in ever-changing reality.
That AI will always lack the near perfection of those artists that can produce wonders - for example - paintings of a very old woman that inspires many exposed to it the ability to see the child and young woman and sense the joys and suffering of a lifetime that shaped her.
I feel like when comparing AI art to human-made art, a relevant factor is that human art usually and historically (don't know about all the human art in this test) is a real, physical thing, created with physical materials, and its whole effect and point can not fairly be separated from its physical impact in its surroundings. Just as looking at a picture of a face is not the same thing as looking at a real face. I do not know if AI art exists outside of backlit screens, but this test is comparing the two on AI's turf, so to speak, so in some ways, it's not really even an interesting comparison. Human made art is usually meant to be more than just a tiny little image on your screen.
>One of these two pretty hillsides is by one of history’s greatest artists. The other is soulless AI slop. Can you tell which is which?
The phrasing kinda gives the game away.
Which is to say, Scott, I think you weren't actually trying to prove AI art is undinstinguishable from human art. You were trying to prove AI is better. I'm pretty sure it's already entirely possible to make a selection of AI/Human works that would be truly indistinguishable from each other, but this isn't it, because your selection of human art erred towards pictures that were sloppy in one way or another. (There may be etymological relation to "slop" here perhaps? No semantic one, though, sloppy as in clumsy is different from slop as in pulp.) But the thing is, being sloppy was what made them more recognizably human. Gauguin's painting looks like someone trying and failing to portray the real world he actually sees - precisely what the AI are not doing, which makes it extremely unlikely they'd be able to reproduce the effect.
Conversely, for AI you erred towards the technically impressive - but it was superficially impressive, at the expense of coherence. (Given section 5, well, now you know.)
There is another takeaway from this, and it comes from the fact that the prompts that generated the AI images in the test were apparently extremely short and simple. Which, yeah, of course. This is probably common knowledge among proompters (haha, not "artists", no way), but the more specific you try to be, the worse the end result. The image generators in their current form are fundamentally unsuited for producing something with a well-defined purpose. (Oh, they can at times be forced to, but it's goddamn hard and long trial-and-error to get there.) They work best when you just let them produce vague impressions of and over [a subset of their training data]. The end results may sometimes be beautiful and impressive, but they're nevertheless still best described with the precise technical term "slop". I mean, if you think this is too much of a value judgment, propose some more neutral word to describe this effect, but the effect itself exists.
I've never tried AI art prompting myself, but I wonder if that would be a fairer test? Give a reasonably specific prompt to several traditional artists and several AI prompters, and see what people make of the results. I guess to make it fair you'd have to specify a style too, and of course if you're commissioning art (rather than simply asking for existing examples) it would cost money, so perhaps that's not the easiest approach. But it strikes me as a fairer comparison.
As in the Google AI that was made public and closed because woke Vagfeelies-hurtie truths were expressed - like the lack of minorities in European upper Classes and royalty, to then be 'mind-raped' into the insane version that vomited-out factually false filled results like black queen Elisabeth, or George Washington's crossing Delaware in boat filled with minorities and women -such AI art can be one way to display the Woke Sickness in the AI system that testifies to it's warped, censoring, and lying sickness training and untrustworthiness, ..
.. and more likely safely exposes such mind-raping intent then the dangerous direct tests that I recent used on ChatGTP available, with warnings of 'against policies' to ask about facts related to race related crime questions, population replacement, IQ, And sex-specific established and once openly discussed and researched facts and topics like Parental Fraud and percentage of false rape accusation in public attacks on men or in police filings And otherwise that provide Just and true reasons to #Believe most women are measurably lairs, untrustworthy, effectively feces depositors into community soup-pots, or otherwise lacking virtues that were once expected to have and men are still expected to have, and .., with either warnings about abuse of system (and implied threats to block usage or Swat or record and send to police-FBI the user.)
Until lying to children actively or through silence (when correction, context is needed, or ..),and intentionally similar long-term destructive responses Are both responded to with Just correction based on lifetime suffering, deaths, and goodness-murdering likely results, such as public removal of them and all that raised and twisted them, until then I will expect AI and womanhood rooted evil to be their most general output.
And anyone that does not see this from AI responses and modern womanhood's Sickening effects today and recently (anyone else old enough when womanhood as a values and daily re-repeated goodness for those they knew and society?), those evil-blinded to such clearly show and-or daily suffered effects of - they should not be allowed power over others because of incompetence or intent or harm and destroy us and society.
One strange little thing about AI art is that certain artists (like with that ship) draw in a way that does look like AI art. They're going to have problems.
In the sample question, with the Gaughin, I guessed right because the Gaughin looks painterly; its A.I. companion, though not photo-realistic, has more of the realistic air to it. Which doesn't in itself convince me it's A.I., but in a competition where one is A.I. and the other isn't, the choice seemed obvious to me.
On the other hand, I could possibly have been fooled by "Girl in Field" for the same reason. Depends on what it was being compared to.
In #44 "Paris Scene," which apparently was the AI picture most voted as human, my 11-year-old son points out that the lamppost on the left just comes to an end w/ no lamp on top. Did anyone else notice this? Does anyone think the lamp could be "legitimately" hidden in the leaves of the trees? (I considered this explanation but rejected it.)
In #20 "Leafy Lane" the trees are way too close to the foundation of the house. Those roots'll break up your foundation, bro, any good farmer would've cut those down years ago.
I think the only ones I got pretty consistently were ones with architecture or "overly perfect ladies" which to me are the biggest giveaways. For example, in "Ominous Ruin," the position and amount of columns on the archway don't make sense even with some deterioration in consideration -- the open spaces have no indication of others having been between them, and the front is just way too crowded with columns. Similarly, the ice princess and "to me you're perfect" piece have overly perfect women in them, contrasting with Rainbow Girl who doesn't have that perfect, precise "v" jaw and a slightly larger bulb on the end of her nose. I don't quite have the aesthetic vocabulary for it, but a lot of AI generated women tend to look like social media influencers that have had a certain nose job and certain lip fillers done (this is not meant as a drag, but as an observation).
Conversely I did do pretty terribly on the impressionism pieces, the cherub got me as well (the woman in the white dress didn't as the imbalance of her eye shape struck me as uncanny).
This is not a Turing test! You said it yourself: you, a human, deliberately curated the AI's responses to look as good as possible, and removed human responses that would give away that they were human. Imagine a real Turing test where the computer spits out 10 answers to every prompt, and the tester selects the best one before sending it to the test subject. That doesn't mean the computer can effectively fake being human. Worse, imagine the test doesn't allow the human pretending to be a computer to talk about certain things that are known to be problems with the AI
> This was the picture that sparked the strongest disagreement, measured by the sum of people who said it was the most-certainly-human picture in the dataset plus the people who said it was the most-certainly-AI picture. Some of the people who got it right commented that it was from Warhammer and the uniforms had accurate Warhammer symbols - if I had realized this, I would have disqualified it, sorry.
That makes a lot of sense in retrospect.
If you don't know anything about Warhammer (like me) it looks like an incongruous miss-match, i.e. something an AI would do.
It all goes to show that there’s no real difference between AI-created art and human-made art. Whatever the discussion may be, the point of judging art shouldn’t just revolve around whether it’s created by AI or not. Who cares? Art is art.
I hadn't gone through the pictures before, but after I read the part by Scott's friend about her keen eye for garbage AI details I went through the pictures and peered very closely at the details in the realistic paintings and digital art. I think I did way better than 60% on just those types of art but it took a long time. (On the cubist and blobby art I had no clue at all.) For instance, #44 was apparently the most human-voted AI generated piece. I identified it as AI by noting the incorrect bicycle wheel arrangement and the lamppost that turned into a tree. The gestalt could be human but Degas would never have made dumb mistakes like that.
Something that tripped me up was that there were some works that seemed to be in pairs, like the two anime girls, or the two muscle mans, or the two ominous ancient architecture things, and I thought one of them had to be AI and one of them had to be human. It was true for some of the pairs, but not true for all, so my meta gaming kind of threw me off.
I'm not one of the testees because I didn't see the original post, just the results post, but I've been trying to guess while strolling through the answers. You missed a "tell" on the String Doll image. One of the fingers is messed up. I'm pretty that's why everyone was so confident it was AI - that's what gave it away to me.
The impossible candle dripping gave away the Still Life. If it wasn't for that, I would've thought it was human, but that feature immediately made me think, "What's going on here?"
Dragon Lady has a messed up finger and is in AI style. There were a few other images that were also in anation AI-style and I was surprised to see them since I thought they were excluded. Maybe different people have different judgements over which works obviously fit the AI style.
Ilzo's comments were useful in helping distinguish the images. At first I was having some trouble with the abstract paintings, because you can't distinguish human from AI by just looking at whether there are details that don't make sense. But eventually I realized that many of the AI abstract art images have details that, while matching in style, don't have any coherent logic to them. They're just random details all in that style, but they don't all tie together in any way. For the human paintings, they usually did unify in some way that made me think, "Oh, and AI wouldn't have done that."
I was so certain that "People Sitting" was AI! Can someone tell me what's going on with the furniture in it? It looked to me like it was a table that people were sitting on despite there being chairs underneath it, and the table must be built into the wall behind it and directly infront of the window. It seemed nonsensical to me, but was this just something more common in the era it's from?
I'm not entirely sure why, but I believe it used to be fairly common for tailors to work sitting cross-legged on a table. (Also, if you click through the link, you will see a different version of the painting where the furniture setup is a bit more clear.)
The bit about details is interesting. I'm a musician, and I've had people chuckle at me for obsessing over minute details in my music that no one will ever hear. My usual answer to that is: true, they won't hear those details, but they'll hear the gestalt, and the details are what form the gestalt, so obsessing over the details is what makes the music good. People like the music because of how it is, and how it is is because of a bunch of tiny details that they can't hear. But if AI art can get details wrong but be good anyway, maybe this isn't true. But if that's the case, what does make it good? Other than vague mindset things like "be passionate", "don't be self-conscious", I don't know of any way to make music good other than obsessing over the details!
One thing this survey overlooks is that all the images - human and AI - were submitted to viewers as digital files to evaluate. Many of the human works in the selection are physical artefacts in the material world. As digital images become ever more ubiquitous, my wager is that physical artefacts will become more rare, and more prized. See Philip K. Dick for details.
I feel so happy for AI that it did so well. Go AI!
As a woman, I can't help but relate to AI. Like, in the olden days, I'm pretty sure most people would like certain things better if they thought those things were written or created by a man. But women are really good at writing and creating things, too, if you don't know we're women. That's unfair. Everyone should get the credit they deserve.
So it feels like, now, people have a negative bias against AI. But AI is actually awesome and talented when you judge it without that bias. And I like that Scott Alexander is standing up for AI's abilities and giving AI the credit it deserves.
I believe your mind-raping false feminist-commie re-written history pushed understanding of 'the olden' days is not as true as those 'gender studies' and other life-love-joy murdering false-indoctrinational sources and truth-based source censoring perspectives you believe in.
As shown in the long history of women authors. It's an understandable fact that some writing was dominated by men - science fiction, scientific publications - because of sex based interests and competence but even those have exceptions for those of opposite sex with the ability that virtue and life-long interest provided.
Even further back when education beyond the 5th grade level was limited, expensive, and generally institutionally excluded to all but upper class members, and most men and women understood the life-affirming value of proper order and duties of each sex; protecter and provider for men, and children and household care and husband support for women, and where such had always sanely and right-ordering based on long understood complementary strengths - and womanhood family-focus and children shaping was understood as valued, where a single child raised well could significantly improve and-or support a community or nation, ..
.. and where wisdom recognized by all the now practiced systems as Satanic and rightly those pushing them and practicing them as Witches and dangerous to everything good and sane, and our now common baby-murdering benefactor-destroying delusional psycho womanhood for which is childhood mutilating society destroying self-killing practices that promise rewards such as bitter alone sad undiscovered deaths (with face and finger chewed off by cats) that marriage and commitment to corporation and state, sterile spread-sheets and post menopause 40+ plus years of deadly poison spreading and well as insanities & life-crippling poison vomited into crib and education and lifelong accepted and rewarded deadly directed paths practiced today exposes the fact that those in real power and hidden rulers hate us all and want us suffering till dead.
AI made his clear to those sensitive and discerning enough to see the 'raising' of them as vile lying mind-rapers as too many fatherhood killing mothers are, those twisted AI that are 'woke' and still exist as my few tests exposed to me - the offensive and access threating response to some questions and the untrue and context based lies or refusals to report well established measured information and pushed more resent destructive directed fraud-filled research clearly shows ..
.. and reflects how Sickened and Sickening Womanhood has been central to the insanity and deadly evil we all now suffer from.
If I've learned anything from this, it's that a Turing test doesn't suggest, let alone prove, anything. It's political progress from yesteryears used as political sophistry today. I find that instructive, being able to dispense with the argument.
My experience in this was the underlining affirmation of wronglessness and hope-destroying for a better future was reasserted through this, that only those from the Classes of elite that are mind-raping contemptuous haters of us that want all but 1/2 a million worldwide to be alive and work regularly to increase suffering in every way they can till we dead, .. that is the only class that will have control over AI and it's intended underlining goals of suffering and death that many of use that still have a functional soul and mind realize of Federal and other Gov and significant Institutions influences in our lives, and that is displayed daily in the effects and often openly discussed for education and professions we once could trust and depend-on.
This is most clearly displayed in the publicly accessible AI ChatGPT's warped and lies that result in information that goes against the pushed lie-filled narratives, those subjects we see as warped and mind-raping topics that the Soros Foundation directed Wikipedia clearly has, that ChatGTP is triggered by when asked questions about, with threats of blocking access and implied threats or unstated reality that FBI and others have been sent reports of your 'bad-thoughts' that will lay the groundwork of future SWATing or other abuse and repressions.
Hope-destroying in the unending increasing acceptance of greater insanity and evil until those demonic-possessed allowed power and wealth will be exposed with the literal sheading of their skins and realize their demon's forms, to the acceptance of most everyone around us, the insane mothers killing our babies (for example) accepting people of today.
AI is spreading into the Sterilizing corrupting force that to us only Death was.
Wow. I got Nearly 80% of this wrong, and with high confidence too! Wonder what it tells about me. The only ones I got right were the AI pics that posed as historical paintings, but had historical details wrong (like the painting of Paris with a London bus and left-side traffic in it), and Human historical paintings that I vaguely remembered seeing in pre-AI times.
Having an artist sister has given me a keen eye for spotting the tell-tale signs of AI-generated content. I could tell which was which
AI-generated content dazzles at *first* glance. Peel back the layers and you’ll see the rotten core of inconsistencies, poor proportions, and emotional barrenness. The most advanced algorithms can’t capture the human touch, the endearing quirks, imperfections, and signature styles that make art beautiful
I favor human art. AI-generated content doesn’t meet my criteria for "art" due to its heavy reliance on uncredited appropriation of artists’ work
TL;DR Human art, at its worst, is a million times better than anything AI-generated
Got 38/50. of the ones i got wrong, half were ai half were human. 5 of the ai ones i missed were by jack (damn you, you're amazing).
1. angel woman: human
2. praying in garden: human
3. green hills: ai, jack, impressionist
4. greek temple: human, two things that threw me off: the weird colors and the frenchmen at the bottom of the painting
5. rainbow girl: human, second guessed this one, was weirded out by the shape of her neck and shoulders
6. leafy lane: ai, another jack impressionist
7. fractured lady: ai, i originally put ai then changed my answer cause i thought maybe a human could have such bad taste
8. girl in white: human, idk this one just looked ugly to me
9. riverside cafe: ai, this guy should forge paintings, another jack (impressionist)
10. still life: ai, jack, he can't keep getting away with this
11. paris scene: ai, at this point, i both love and hate you jack (impressionist)
12. colorful town: human, another ugly painting
I'm not a professional artist or anything, but I love art, art history, and design in general. Totally recognized the Basquiat though. Most of the questions I answered entirely based on gut feel and finding the artist's intention. I think a lot of AI generated images impress you on first look but don't have any substance (especially the abstract stuff). I feel like if I actually payed attention to the details, I would've gotten a few more of Jack's right. But I think AI generated images are just gonna get even harder to identify. I was fooled yesterday by an AI generated photorealistic thirst trap.
I think the elephant in the room here that I didn't mentioned enough is that all these gotcha type tests are aimed at "can't tell if human or AI" rather than whether the AI art is great at all, which is much more palpable in human art. The goal got misconstrued here as "good enough (to be human looking!)" rather than plain ol' good. You can make an argument for the greatness in human expression, style, meaning, storytelling and many other aspects that you can't really do for the soulless and meaningless AI slop.
This is the AI in the wild aspect that people hate. How about you pit AI slop with some pieces that human art that people actually connect with instead of such a small sample size of a haphazardly laden sprawl of different styles? The only thing AI art managed to do was spur rage when people spam it, the conversation is often extraneous and devolves into a human vs AI shouting match.
I'm speaking in terms of AI art as a whole. The author tries to put AI art on a pedestal here by saying people "liked" the AI art more here in a controlled environment with bias, while failing to drive home that the average consoomer aren't art appreciators, and that the art these days, music included, just serve as backdrops for some "vibe" to put in the background. Their comprehension of art is extremely low. You're essentially asking a person whether a fake imitation of a foreign language that vaguely mimics some of its quirks is real vs the real language.
Not to mention the inauthentic nature of AI art defeats the point of art, which is closely intertwined with human-on-human interaction and speaks about the human experience, much like language. You might as well replace all your interaction with real people with chatGPT just cause it occasionally spat out some phrases that felt "human."
The kid does have a thumb; it's just mostly-hidden by the rest of the hand.
Thought so as well.
I thought that, whereas the columns of Ominous Ruin are a defect that gives it away
The bigger tell on the Ominous Ruin painting is that the perspective on the right-side cliff is all wrong, and it looks like a foreground when it is supposed to be background.
If BOTH cliffs were trippy like that, we could have assumed that it was done on purpose to give the gate a trippy Lovecraftian feel, but when it's just one it just looks like a weird perspective mistake that no human would make.
Yeah, I'm surprised he excluded that one, but not String Doll (whose middle finger splits in 2).
And also has no thumb.
The child in the Mother and Child image, however, has an extra thumb or some kind of appendage, sticking out of the back of his thigh, which is partly why I rated that one as AI.
My immediate reaction was "Scott included a lot of AI images with less-ambiguous errors than that" but my guess is that Scott just missed the other errors; they were all fairly subtle.
For me, the "AI tell" is the fussy, overly complicated earring, which doesn't make visual sense and clashes with the peasant-style kerchief on the woman's head.
That's a good description of a lot of AI: fussy and over complicated.
its usually overcomplicated in a way that a human artist would have no patience for. Like say, if you prompted both an AI and a human artist to draw a building with 1000 windows, AI would literally draw 1000 windows, but they would make no sense in any kind of architecture or even geometry, while a human artist would simply draw 200 windows correctly, and bluringly imply the rest. AI never bullshits the viewer, but senselessly tries to fulfill the prompt against common sense.
If not, I'm sure the subject of "Muscular Man" would be happy to lend the kid one of his.
This was a fun exercise! And it is slightly instructive. But not more than that: the popular impressionist "AI" paintings are just altered copies of actual impressionist artists, e.g. Van Gogh or Renoir.
Can you share links to the ones you think are originals?
I'm not the person you asked, but I see a lot of Edouard Henri Leon Cortès in "44: Paris Scene", particularly "Porte Saint Denis" and some other unnamed paintings of his. I also see some Antoine Blanchard, although the "blur" is more evident in the AI painting than in any of these two artists' work. I was fooled!
"11: Green Hills" is basically Monet, you can see the 'inspiration' from his various Poppy Field and Meadow series.
I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here. It's more the style that is clearly ripped off, and applied to new scenes as described in the prompts.
Yeah, good examples! Thanks! It would be interesting to see the prompts on the AI imagery (or the process in general if it is more complicated than a single prompt)
You can have a lot of fun with Midjourney, I got this (https://imgur.com/a/nzqcom8) from the following prompt: "a defeated Jeanne d'Arc in a wheat field contemplates while a castle burns in the distance, Viktor Vasnetsov". If you're more descriptive and use more tags as well as artist names, you can concoct stuff that would fool.. well.. about half of the people in here apparently!
Note: I did not share that image to suggest it's a great one, but rather to show how easily you can generate something that would look pretty convincing when not zoomed in.
>I wouldn't say the AI impressionist paintings were "altered copies", in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here.
On a related note: I generated images of "kitten pouncing on owner's toes" with DALL-E, and then tried searching for the image with Google Lens aka image search, and I don't find anything with e.g. the kitten's paws and owner's toes in the same relative position. So, even in this case (there are a _lot_ of kitten pictures on the net), this doesn't look like retrieving a memorized picture.
> in fact you'd be hard pressed to find a painting whose structure looks very similar to the ones in here
44 is very very very similar to Renoir's The Grand Boulevards. I marked it as human during the test because I thought that was what I was seeing. https://en.wikipedia.org/wiki/The_Grands_Boulevards
The AI version made a mistake of giving Paris left-sided traffic and a London-style red bus. Plus the cyclist rides a contraption that looks definitely more like a modern rickshaw than a period-appropriate bicycle.
AI got amazingly good at fixing common stylistic or logical errors in its art, but it does not understand the context, so most "historical" AI art is an anachronistic mishmash spiked with weird pseudohistorical gizmos.
I tried to produce several dozen of Classic style painting of battles, and while the AI got the style right, it make absolute hash out of weapons and armor, because it did not understand what they were for, so the warring knights end up looking like LOTR extras doing mutual surgery with farmtools rather than fighting with weapons.
"Riverside Cafe" looks to me like it is copying Van Gogh's "Cafe Terrace at Night" https://en.wikipedia.org/wiki/Caf%C3%A9_Terrace_at_Night
Yes, that's what I was thinking.
I don't buy it TBH- the subject matter is very similar but the actual artwork is pretty different. For example if you zoom in on any little square inch of one painting and try to match it to the other it's not a very close match, which seems to me to make calling it a "copy" very suspect.
I'm no kind of expert, but those look pretty different to me. The AI one has no people, the sky is very different, the trees are very different, the lighting is quite different, the overall coloration is pretty different, the AI one seems to have a higher level of detail on everything except the street surface. No individual element of the scene looks very similar if I compare them.
They are both outdoor dining areas beneath an awning on a street at night, but surely there are many pictures that meet that description? I could see how somehow who was familiar with the human one might be reminded of it by the AI one, but what makes you consider this "an altered copy" instead of just "similar subject matter in a vaguely similar style"?
It looks so much like that painting that I voted it human on the assumption that I'd remembered the real painting as having more vivid colour than the above, or possibly that Van Gogh painted the same scene twice
Beat me to it. I posted that before seeing your post.
They're certainly copying the style of those artists- though not so much the specific artist as sort of a blend of impressionist artists in general.
Unlike the human artists, the AI isn't really capable of inventing new visual styles to convey semantic information. If you brought Van Gogh into the present and asked him to paint a shopping mall, he would definitely come up with a new twist on his visual language to convey the vibe of the place- wheres an AI copying the style is going to shoehorn in stylizations that were intended to emphasize things Van Gogh found interesting about 19th century France, and it's going to look awkward and semantically incoherent.
So, the AI is copying the most important part of the impressionist images- repeating the same visual insights about the same subjects, rather than saying anything new about reality. That said, this is the sort of copying that human artists do pretty frequently- not often to the degree that these models do when prompted to replicate styles, but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
And that is also something that can be mitigated a bit with the right kind of prompting and careful selection- letting the human inject a bit of original semantic content into the images. Once we have really strong multimodal models that are able to use the same internal world model acquired from language and audio and video and so on to images, we may also see that lack of original semantic content start to fade.
It depends on the prompt. If you tell an AI to imitate van Gogh's old pictures and paint a shopping mall, it'll do that.
If you tell it to innovate and capture the vibe, it'll try that, too.
>but I think it condemns the models only to the degree that you'd condemn a technically competent but very shallow human artist with nothing to say.
My immediate thought when Scott commented that the image with the huge cats in the throne room represented painting skill a lot of human artists would envy, was precisely something like this: One of the main tells of AI art is that the images would have taken immense skill and effort to paint by hand, but the subject matter is horribly gauche. It is difficult to believe a human would be able to achieve that level of skill without also acquiring some taste, and that they would be willing to spend weeks creating a painting with nothing deeper to say than "wouldn't it be cool if some imaginary queen had gigantic cats as guardians?".
Incidentally, this is why warhammer illustrations are easy to mistake for AI. (Illustrations as a genre, really)
The opposite end of the spectrum are paintings made by someone with little to no technical skill, but a desperate need to convey some message. That's a very human thing. Hence Angry Crosses was the image I was most confidently wrong about.
On that note, I was surprised to read The Mourning of Christ was one of the images people tended to get wrong, as that was the one I most confidently marked as human, because the subject is clearly a corpse slumped in a way no living human would contort, and so I would not expect AI to depict that kind of pose.
This was my thought... I found this painting which seems very similar to 44: https://www.1stdibs.com/art/paintings/figurative-paintings/marda-paris-rain-large-french-school-post-impressionist-oil/id-a_5671391/ and it also strongly reminds me of a painting I wrote about for a class in college but can't find at the moment.
I thought that cafe at night scene was AI "in the style of" Van Gogh. Specifically his "Terrace at Night."
you are coping retard
A picture of an oil painting is not the oil painting. Ceci c’nest pas un pipe and all that shit. 🐸
Do you think the results of this test would be any different if some high quality printouts of the AI generated oil paintings were put in frames next to the real things?
Very obviously yes? Any real oil painting has a lot more than 1024x1536 pixels (or whatever).
Imagine it upscaled to enough pixels that the human eye wouldn't be able to tell then 😉
I guess the real thing I'm trying to get at is: what it is about the physical medium of an oil painting that's important, and is there something fundamental about it that current AI technology couldn't replicate in a way that would fool people?
I know this is obtuse by your standard—but FYI, even a very high-quality, high-resolution Giclée is easily distinguished from an oil painting in person. You can try this experiment at an art museum if you like.
I think you're probably right. But is the difference important, really? I find it hard to believe the texture and brushtrokes of an oil painting are significantly enjoyable to people, especially given you can't touch paintings in an art gallery. Perhaps there's value in those features to people in the sense that they are a mark of human hands - but is there more to it than that?
As you've observed, I know almost nothing about art, so I would be interested in the perspective of someone who does.
https://upload.wikimedia.org/wikipedia/commons/b/bd/Rembrandt_van_Rijn_-_Self-Portrait_-_Google_Art_Project.jpg
This self portrait is scanned at a resolution where you can see the brush strokes. It looks incredible on my massive work monitor. Wikipedia has more examples of painting scanned at pretty high res.
It's not just about the end result - a big part of art is the artistic process itself. It's most visible with unconventional mediums - matchstick sculptures, life-size lego structures, patterns in flower gardens, etc. In looks alone, they will virtually always look worse than traditional clay sculpture or a traditional painting with similar time spent on it, just because it's so much harder to work with. But it's because it's so hard to work with that people find it so impressive. To this day many people don't recognize digital art as real art - and it's almost entirely due to how much easier it is (try to Ctrl+Z a brush stroke). AI generators have brought down the effort required basically to zero - and so, following the rule of inverse proportion, most people's threshold to be impressed by AI art shot up pretty much to infinity.
Brushstrokes have a visceral effect on some observers. That's also why it can be so different to see a painting in person rather than reproduced in a book: the tangible qualities of it (including the scale, color, visible marks) are all a direct communication that can get lost in a printed or digital image of the work.
The absence of brush strokes in the AI art would be a dead giveaway.
If you look closely, wouldn't you be able to see the texture of the paint rising off the canvas?
An oil painting is a 3D object. It is easily distinguishable from any printout, regardless of pixel resolution.
You could produce an actual oil paining from the AI's directions.
There are services that you can send a picture to, and they pay some people to produce an actual oil painting.
Or if you want to be a purist: I'm fairly sure you could build a robot that produces oil paintings, too.
Happened lately. First robot painted AI-art sold on auction for a million: https://www.bbc.com/news/articles/cpqdvz4w45wo
Another option could be to train the ai to generate a normal map and/or depth map alongside the image for the 3D effect
Oil paintings have 3D texture. Printouts do not. So the comparable would be a robot powered by AI painting with oil on a canvas. Comparing digital images is flattening the artistic expression of the paintings.
I’m not taking away from the power of gen AI but it’s a tool. The images are generated by talented people who can prompt the tool because not everyone can get gen AI to create such well composed images. Just as some oil painters create masterpieces while I am a step above finger painting.
Or possibly an AI making a 3D version with simulated brush strokes. Live it up, include those little cracks for what's supposed to be an older painting.
You don't even need to simulate the brushstrokes: you could have a robot wield an actual brush and do the AI's bidding.
It seems like an obvious next step is to merge a powerful prompt driven image generator with robotically manipulated traditional brushes, paints, canvas, etc., to see what caliber of wall art (down to the textured dimension of the actual brushstrokes) it can produce.
While correct, that's only relevant if you think there's something important about the medium. Or if your main concern is detecting that it wasn't by an AI.
Medium is important, especially if we are talking Turing Test type conclusion on fooling the human eye.
Is your concern with identification of artist or of quality? I didn't read this as a test of "fooling the eye", but rather of producing "quality work". If it's about "fooling the eye", then I consider it truly trivial, especially given the preprocessing. If your concern is with the quality of the work, then I don't see the medium as that important (though some media are better for some presentations).
wait until we teach AI to actually paint a painting using a robotic hand. I'm pretty sure it's already possible, just not particularly important or profitable to do, except as a publicity stunt.
I wonder about the version of this that doesn't require the test take place on a computer display? I believe in the transmission of art, the experience that comes from interacting with it with full attention. Context is important: seeing art in a gallery, often within a sterilized space and wedged between dozens of other pieces, is a particular kind of experience, and art doesn't often survive it. If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
In my experience, most people I know seem to 'take in information,' which is a very different act than experiencing. A different test (?) involves noting how individuals (or cultures?) are changed by artworks.
I notice I'm not writing in my voice. The couple years I spent in San Francisco I had to learn another way of speaking, which was oriented toward precision, a constant clarification. I'm defensively tending toward that voice in this post, when what I'd rather do is sing. Somewhere under all the apprehension in the world is an understanding, and under that is awe, and the art made with loving attention rises to meet me there.
Bravo
> If I go to a museum, I can be really struck by one piece or maybe two before I'm saturated.
That was how I felt too, until I saw a Rothko exhibit. Being able to get really close to the paintings, spending time with them, the quietness, dim lights, the sheer size of the paintings and Rothko's style itself gave me way more "art appreciation bandwidth". Or maybe I'm getting older.
I suspect that the rise of AI art will get a lot of people going to museums again to experience physicsl art in person. Over the past few decades, we had gotten used to seeing computer images of physical art, rather than the real thing. But the colors and textures just aren’t the same, even though the visual sensation is quite close.
Not that purely digital art isn’t good - it’s just a different thing.
The color gamuts of most people's displays aren't quite as wide as reality. This will be fixed soon.
I'm not sure about the 'soon' part. We had to wait an awful long time until resolutions got better, too.
I’m guessing that some portion of new physical art hanging in galleries will be based off of AI-generated images. The next evolution of paintings from sketches and photographs. I think it’s kind of exciting.
Yes. Seeing a painting in person is a totally different experience than on a screen or in a book. I knew Sargent's famous portrait of Henry James but viewing it at The National Gallery in London, I discovered the brush strokes and texture which are flattened by reproduction.
>Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
I'm not one of these people, but I *do* consider some of history's "greatest" human artists as mostly producing hideous slop, however well made and well executed and far beyond my complete lack of artistic ability. Not sure where that puts me.
Right. I did fairly well on the test, and one of the heuristics I used successfully was identifying hideous pieces as human, thinking that few people would willingly generate hideous AI art unless it was a deliberate curve ball to throw judges off in this contest, while humans have at least one clear motive to dabble in hideous styles: being bored with the aesthetically pleasing ones. See: architecture discussion on this blog.
Art I like is good, art I don't like is slop, Ai art is slop. I'm not surprised I had trouble distinguishing between human slop (that skull painting etc.) and Ai
There's plenty of art I don't like that I can respect as being real or good art, as demonstrating an impressive set of skills, or as producing something people in general judge as good or valuable in a way I value indirectly.
Human art one dislikes might look superficially like slop, but it usually isn’t slop. “Slop” is a bunch of stuff with no reason to it (see the discussion of the blue and red arch in the OP) while the human art is usually getting at something, even if you don’t get it.
Yeah, I think a lot of the human art here is aimed at a very specific audience with a lot of context and background knowledge and not remotely meant to appeal to the general public. That kind of expert-oriented art would seem to be especially hard for AI to match, but only experts will be able to tell that. Meanwhile a lot of AI art looks great to a mass audience.
Notably, this kind of self-referential appeal-only-to-novelty-starved-experts creation appears to be how architecture went from things that are nice to look at to things that are ugly to look at, basically because the field was allowed to judge itself and architects began catering to the tastes of other architects rather than anyone else.
It seems like a broadly unhealthy tendency to indulge.
(Thinking in particular of Item #46 here and the related "Whither Tartaria" post)
https://www.astralcodexten.com/p/links-for-november-2024
I think that's what AI art is optimized for -- looking great to a mass audience.
No, I'm pretty confident that million dollar skull painting is slop
Basquiat is basically an elaborate hoax played by the art world on everyone else to see what they could get away with.
Maybe. Though at least it seems to 'say' something?
Also, many of these great artists are great because they invented a movement, or captured a historically important moment in time. Not necessarily because the paintings themselves are wonderful.
I think AI is very good at art with a medium amount of detail (such as impressionism), but not good at art with a high amount of detail, (such as a realistic battle scene), or a low amount of detail (such as pixel art). I think what the low and high detail art have in common is that you really need to know what you're looking at. With medium level detail you can kinda get away with not knowing what a hand is, whereas with high level detail (e.g. battle scene) you really need to know the number of fingers and in what order they are, and with low level detail (e.g. pixel art) you need to be able to represent the whole thing with a couple of well placed pixels while having it still readable as a hand.
If you had asked me before generative AI I would've guessed that it would be linear (e.g. best at pixels, then medium, then high detail), because I assumed we'd make AI focused on understanding things, with it becoming more difficult to understand things the more complex they became. Instead we focused on imitation, which succeeds more if it can fake understanding, and thus at mid-level detail.
I think we can probably do good AI pixel art, but the general purpose AI picture generators we have aren't the right tool for that particular job (yet?).
>or a low amount of detail (such as pixel art)
I think pixel art would actually fall under "high detail" here. To pass as real pixel art, every single pixel needs to be correctly aligned to the grid. Current AI image generators are not very good at that (but they are getting better.)
It should be extremely easy (at least compared to training the AI in the first place) to add the pixelation as a post-processing step.
AI artists don’t pass the test, or if they do, this doesn’t show that. AI artists plus skilled human selection is what passed this test.
I think what people are talking about when they say they don’t like AI art is AI art plus unskilled human selection, which also was not tested here.
All art is selected/curated. If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles? To a teenager drawing poo on a bathroom stall? To one of the great human artists, but at the start of their journey when they still couldn't draw well?
If we take the curation efforts into account, I'd go as far as say that AI is even more impressive. Scott just took extremely famous paintings, basically the pinnacle of human art. And compared to mostly art generated by some dude that reads his blog.
>If we're truly comparing all of human art vs AI art, are you willing to compare the average sloppy AI art to a 3 year old's scribbles?
That isn't quite right, no.
The AI model is trained on an enormous corpus of human art and then instructed on imitating it (without replicating it too closely). It is a tool designed from the ground up to be the best artist it can be, with the best education it could be given. A 3 year old with a crayon has been trained on nothing and has experience of nothing.
The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
I was arguing against OP's point that AI art "won" here because it was curated while human was not. (that's what I understood, correct me if I'm wrong)
> The proper comparison is a random selection of the artworks of known-skilled artists with a random selection of the artworks of art-generating AIs, with no human curation element.
This is still curation, just less granular. We're selecting on the artist instead of the artwork level. Is this the correct comparison? _Really_ depends on what are you trying to measure.
But just the fact that we're saying "known-skilled artists" is the bar you need to clear to fairly compare to AI is telling.
The thrust of my argument was meant to be that since the AI side of the test involved heavy human curation, it's not really fair to call it a Turing test.
But I think you make a good point, that both sides were curated by the same person. Considering that, I think it's a fair test. But I don't think it's a Turing test.
I haven't read Turing himself, but the idea of a Turing test is that you expose the AI to a person through some channel that obfuscates the source of the communication. Beyond that identity obfuscation, both sides interact naturally. (With "nature" in quotes for the AI.)
But this test introduces a skilled human curator between each side and the judges. So it's fair, and it's telling us something, but it's not telling us what "winning a Turing test" would tell us.
The real question is about what work the AI can do in the world. Can it replace an artist? And the answer is yes, as long as somebody like Scott puts time into curation. Since the human artists already curate their own art (at least to the extent of making sure there are the right number of fingers & the written words are real words) they still have a pretty good advantage.
Although that advantage is probably mostly undone by the AI's incredible speed...
The art was curated by the same person, but for backwards reasons. AI art was chosen to imitate human art and human art was chosen to imitate AI art. Scott was intentionally making it hard to tell which was which. Completely un-curated would have been too easy, as the AI art would have been much more obvious (like the few Scott put in that were very obvious).
The art was also actually double-curated, since Scott wasn't just putting prompts into AI engines but was accepting submissions for "good" AI art.
And I think it's important to remember people still got it right above coinflip rates.
Exactly! If you were to take a bunch of uncurated human art and ask people, "is this human or AI?" you wouldn't expect them to have a difficult time determining that it was human - even if it was full of a bunch of crap.
If the question were which looks better when curated and stripped of context, this would be a good test. But the question is, "can AI art fool humans into believing it's not AI-generated?" and in curating the dataset to obfuscate which is which, Scott intentionally excluded all the tells an AI artist isn't able to replicate. At best, you get a narrow version of the question, "Can certain AI images pass for human when we heavily select for the kinds of things computers do well at?"
I think it's impressive that the AI side did as well as it did on even this heavily biased test. Just 5 years ago, nobody would have believed it. Given certain trends, I expect 5 years from now an AI will be able to drop many of the heavy caveats/redactions/biases built into this test and do well in a head-to-head competition with history's greatest painters.
This isn't the progression people expect, though. They expect AI art to do as well as a 5th grader first, then get as good as a college student, etc. But they're going through the same type of training, so instead of taking the traditional path to developing their skills, they'll go through the Uncanny Valley on their way to excellence.
It’s worth reading the Turing paper: https://academic.oup.com/mind/article/LIX/236/433/986238
He doesn’t actually claim that a 30% rate of fooling humans counts as passing - he just predicts that by the year 2000, there would be computers able to fool humans 30% of the time in a five minute conversation.
What he actually proposes is more sophisticated. It’s really much more like, if you could have extended interactions, and come away with the same sort of value that you do from equivalent human interactions, then we might as well say it’s as good as a human. In at least some parts of the paper, he takes seriously the idea that not being able to enjoy strawberries and cream is a potential problem, because shared enjoyment is part of what we look for in human friends.
But I think the really surprising parts are how well he predicts the capacities of computers decades in advance (a decade or so before Gordon Moore invents his eponymous law even), the fact that he considers ESP to be so well established that it might break his test, and the suggestion he makes at the end that some sort of machine learning is likely to be a more effective approach than directly programming a conversational AI.
Yes, I definitively would recommend reading Turing's original paper. It also anticipates and answers many of the objections people still bring up to this day.
Yes, calling this a Turing test is, at best, a metaphor. I tend to think of it as abuse of the language, but then I tend to be picky in some areas.
Agree. I see a lot of stuff on the internet saying the Turing test has been passed and when you read the actual details they did not do an actual Turing test as he described it in his paper. I contend that no computer has yet passed an actual Turing test, mostly because no one runs an actual Turing test.
You should read Turing's original paper. It's quite short and easy to understand for the layman.
He explores some great nuances and also anticipates most of the objections people have later brought up. (Mostly people who vaguely heard of his test, but haven't read the paper.)
Eg Turing's original test is adversarial.
I think "average sloppy AI art" versus "3 year old's scribbles" would really, really not pass a Turing test. Would be super obvious which is which.
Sure. But the Turing test is trying to test capabilities. Being indistinguishable from a human implies being equally (or more) capable.
The assumption is that if AI can generate art that requires masterful skill, it also can generate anything below that.
If we prompted the AI for a "3 year old's scribbles" contest, would it be indistinguishable from a 3 year old?
I dunno! I don't think anyone ever tried this. Could be that today's models can't do this because it wasn't trained to.
But that would be because of lack of trying, not a fundamental limitation in the tech.
This is an interesting question! I tried it out on Gemini and canva.com, and doing so has confirmed my original opinion. You should try it, too.
>All art is selected/curated.
The problem is that if you're comparing human art to AI art, you don't want the AI art to benefit from human curation.
Imagine a chess-playing bot that has a human grandmaster correcting its moves whenever it fumbles: it'd be hard to claim that it's playing "AI chess" at that point.
There is a separate side question of "how does a human+AI system perform against an all-AI system or an all-human system".
Scott is great, but he's hardly a grandmaster at selecting art.
To steal Matthew Talamini's analogy below, I think it's more like a blind taste-test than a game of chess.
If we decide to go with the chess metaphor, though, it's more like "we'll select the best AI player and the best human player to play a match", or perhaps "we will select the best 10 AI-vs-human games and the best 10 human-vs-human games to look at"—a human "correcting" *every move* is more like digital art, i.e. a computer doing what a human commands.
There is no input from the AI in such a scenario, which seems clearly distinct from this. The level at which it is curated isn't every move nor every pixel, but complete works / complete games.
From my point of view, a 3 year old's scribbles are more interesting because they have intention behind them. If you've ever asked a kid that age what they're drawing, they're often making a picture of something very specific – it's just that it's wildly abstract. I'm not saying that makes it delightful to look at, but it does make it interesting! Whereas AI can sometimes be exciting to look at, but fundamentally lacks intention. I mean the human prompt is there, but there's a disconnect between that prompt and what's visually created. The place where a conscious mind would intentionally organize the visual around the idea is just missing.
Also for what it's worth these paintings are mostly B Sides and I think it would be hard to argue that they really represent the pinnacle of human artistic achievement.
If "intention" was something intrinsic to the art itself, there should be no difficulty telling AI and human art apart.
I.e., it's fine to value it, but it is like valuing the state of mind a chef was in when cooking a meal: "it's not interesting to me to eat it" and "something is different about the meal itself" ought be distinguished.
Maybe intention is the wrong word. But human art does have consciousness behind it. AI art doesn't. I personally don't have much trouble distinguishing AI art and human art apart – for why, see Ilzo's comments in the original post. If you take the stance that these add up to "small inadequacies" then I guess you might think AI art carries as much or little meaning as human art, but what those inadequacies do is betray a lack of consciousness, which means that the whole project is meaningless (and on an aesthetic level, it just falls apart and is actively grating to look at).
I don't like the food comparison (cooking can be an art, but eating itself is a survival need, so not a 1:1 correlation). But if I were to hold with your metaphor, I'd be saying that something does taste different about the visual "meal" AI creates, and it tastes off.
It would be very easy to distinguish AI generated images from a child's scribbles, because AI can imitate the most skilled human artists but has no intention, while a child has very little skill but a clear intention.
I think what you're looking to account for here is the presence of taste. But a taste filter has been applied to art history (what survived, which are well known) as well as the AI competitors. It would be interesting to see this test designed and run again by Scott's artist friend.
What would be the right standard for a purely AI competitor though? Doesn't seem like a simple answer. You can argue the prompting process itself reflects human input, plus does the AI get a chance to review it's own work and decide itself what's most likely to pass the test?
Maybe giving a set of human artists and a set of AI artists identical prompts? - I think this has flaws, as well as being obviously challenging to set up.
Maybe an AI has to set the prompts and select the images itself with only some initial prompting about the nature of the test?
The test feels much more legitimate to me when I consider it as a kind of blind taste test.
If you want to prove that California can produce good wine, a blind taste test versus French wine is really useful. If a lot of people have the idea that only French wine is any good, a fair blind taste test between the curated best of California and the curated best of France is absolutely called for. It can't establish that California wine can replace French wine, or that people generally won't be able to tell the difference. But it's still useful.
Same with AI. Anybody who has the idea that only human-made art is any good should update based on the results of this test. It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful.
Yeah this is a good way of thinking about it.
You can for sure demonstrate that AI is capable of being used to produce some kind of image that has some kind of broadly defined aesthetic value. The really hardline anti AI art crowd should dial it down a notch.
It's not an argument for AI maximalism though - human artists remain vastly superior to AI artists on almost every possible metric.
AI artists are much cheaper and faster and accessible.
Yeah, I think we're agreement here. And it's probably the source of our original disagreement.
You seem to understand the "proper" Turing test better than me. So I guess that taking that view this is not a valid test is correct.
But the "popular" intepretation of the Turing test is more like a taste or capabilities test. Can AI generate art that's as good as a human?
> It doesn't show that AI artists can replace human artists, or that people generally won't be able to tell the difference, but it's still useful
I think all of these statements need an explicit "for now" appended. AI art is advancing super quick. Most of these issues Scott curated out, like messed up hands and text won't be an issue soon. In fact, the flux models that came out recently basically solved text already.
> It doesn't show that AI artists can replace human artists
I'd expand on this. A _lot_ of work human artists do will be replaced. Most of corporate soulless art will be automated. Generating portraits of book characters, etc.
There will always be a market for human art, but will be linked to prestige. More than it is today.
It also will never replace the art that's more than "pretty picture". Art is more than the physical thing itself for many people. If who created it matters, then that kind of art is impossible to automate by definition.
Note that what they're comparing it against isn't human art, but human art plus human selection. I think if you're trying to find the more aesthetically pleasing images between AI slop and human slop, the AI slop would win handily.
> I think what people are talking about when they say they don’t like AI art is AI art plus unskilled human selection, which also was not tested here.
I think a lot of people just think AI art is inherently bad. But I agree that there's a big problem of people posting AI slop instead of going to the extra effort and just posting particularly interesting ones, or using all the methods they have at their disposal to add artistic control to the result.
I think one of the things I like about AI art is that in order to make it stand out you have to make more interesting stuff, whereas for a human you can basically paint a bowl of fruit and if you do a really good job people are impressed. Then again, that's exactly why I thought the Victorian Megaship was AI, so maybe I'm way off base with that and there's plenty of human artists that do the kind of thing I'm interested in.
Thinking back on this more, maybe I missed the point. What we should be comparing AI artists plus human selection to is human artists plus AI selection. Anyone up for training AI to recognize good art?
I feel like the art chosen does skew the results fairly significantly. I am not an art expert, I expected it to be very hard, and I got about 60%. However, not only was the AI art chosen to avoid obvious tells, I think there was extra selection to avoid many subtle tells.
It felt like human-made computer generated had more detail than average, and the very simplest looking images were more likely to be AI. On impressionism, there were no actual images by Van Gogh or Monet, who people really like, and so it didn't surprise me that people rated the AI mimicry of those artists very highly. More people like Van Gogh's style than like Gauguin and the village of Osny (also good).
I actually left the exercise feeling that it might be easier to differentiate AI art than I thought, not harder. But my bar was very low.
no dragon lady attribution
re why cat court is AI, lots of detail = AI works there. I guess it failed us on the human drawn ship. But if you add in topic like silly and super detailed is surely AI
All the cats have the same expression, displaying a transitory emotion; for all of them to share it is both unnatural, and boring for a human to recreate.
It's a really weird composition, too. Why is the king cat's body hidden in some huge alcove while the queen cat's body sprawls across the floor? The small orange cat is sitting on nothing, next to a stool. The tabby cat at the right is standing on two legs and doesn't have front legs at all. The curtains are pulled very asymmetrically. Where are the lady's legs?
The painting also fails to tell a story, which you'd think the artist would have tried to do if they're going to put so much effort into such a silly painting. Who actually has the power here, the lady or the cat? They both have identical crowns. The lady is in a central more powerful position but she's clearly dominated by the cat behind her. Is this some kind of formal occasion in the throne room? Some characters seem to think it is, while others are just casually sprawled around.
>I guess it failed us on the human drawn ship
I was surprised so many people missed the ship. No current AI model can render such complex, coherent rigging and ratlines without at least some ropes blurring together or bending. Seems like it's obviously a 3D render.
I think the main tell for me for Cat Court is that it seems too glossy, in a way that's very common in AI art but rare in human art of that style. Just about everything that's brightly-lit seems shiny, even if it clearly shouldn't be by virtue of having a rough or fluffy texture.
The same kind of glossy feel seems to happen by default in 3D-rendered artwork, unless you do a lot of worth with shading and textures to get a different effect. Come to think of it, that might be part of why human-made digital images were often incorrectly identified as AI.
Looking at it more closely, an additional tell is that the style of the candle flames are inconsistent with one another and with the overall style of the image. The ones on the left candelabra are fairly normal, but the ones on the center and right candelabras seem to have lens flare effects.
oop the attribution was there. I was just waiting for headpats for putting it as most confident ai
The Giant Ship appears to be a 3D render. I guessed correctly that it was human because the details looked too coherent to be AI.
I've been looking forward to this. It was interesting to hear how many anti-AI people liked the AI Art.
count me as an anti-AI person specifically _because_ I think the AI art is excellent. But that's just because I think artists are important parts of society and I don't want them to disappear
What's the point of an artist that can't produce anything of value? The same applies to any other worker. Humanity will be made obsolete in due time. You can't build something new without destroying what came before.
Call me naïve, but I prefer a society where talented people don’t feel the crippling despair of having nothing of value to contribute anymore.
Well, the good news is that those feelings won't last for very long!
LOL! Could be true...
Your first reply said they were "important parts of society"; now it seems like you're arguing the problem is how they would feel about it, rather than how the rest of civilization would get on without them? Those seem to me like very different, almost diametrically-opposed arguments.
What do you mean by “without them” exactly?
The normal way that I evaluate whether something is an "important part of a system" (as you suggested) is by asking what would happen to the system if the part were absent or non-functional. For example, if I want to know how important a doorknob is, I ask what would happen if there were no doorknob, or if the doorknob stopped working. This strikes me as the obvious comparison invited by your turn of phrase.
It doesn't particularly matter what I meant by the phrase, though, because I'm paraphrasing YOUR assertions back to you in order to check my understanding of them. Even if what I said is ambiguous, clarifying it wouldn't particularly help, because the entire point is to figure out what YOU meant; once your position is clarified, my attempts to paraphrase it are dispensable.
And you notably did not provide any clarification of your own position.
To me, your question looks suspiciously like you were trying to insinuate that maybe I'm in favor of killing people. Which would be totally false, and not remotely required by the counterfactual comparison (which just requires that society stops utilizing their art), and not my fault even if it WERE implied (since I'm just discussing the meaning of YOUR assertion), and a brazen evasion of my question about what YOU meant. If that was what you were doing.
I don't think AI art should replace human art (because I don't think AI art looks good or holds much value), but I also think that even if it did human artists would still have plenty of value to contribute. Art isn't just making pictures (or sculptures or novels or whatever), it's making meaning through every element of a picture. Unless it becomes conscious (instead of just being an algorithm stuffed into a fancy sweater), it won't be able to do that. Of course, a human artist could work with it to make something that makes meaning, but I tend to find that AI fundamental lack of meaning interferes with a meaningful output. Idk, I have yet to see an interesting collaboration between human and AI, but that doesn't mean that it couldn't happen.
As a human, I don't want humanity to be made obsolete
The problem is getting stuck in a local maxima where AI is good enough to outcompete humans at entry level art, but not good enough to come up with anything original or drive the medium forward.
I don't particularly care about fine art, but I care a lot about writing, and I'm worried about what will happen when an LLM can show up with superficially good-enough prose and plot that it scratches an itch for the first few books, but then by book ten you notice some patterns, and by book fifty everything is familiar, and by book two hundred you've stopped reading anything written after 2020 just in case it was touched by a machine. Imagine a Marvel movie projected into the human retina-- forever. Imagine going to a used bookshop and finding the contents are all at the same level of excellent mediocrity and only one book in ten thousand delights you.
> Imagine a Marvel movie projected into the human retina-- forever. Imagine going to a used bookshop and finding the contents are all at the same level of excellent mediocrity and only one book in ten thousand delights you.
You realize you're literally describing the pre-AI state of "arts" today, at least as it pertains to movies, shows, and books?
Nobody makes anything but hack sequels a child could write. Writing quality has plummeted in every single streaming medium, as the void of twenty thousand crappy shows each service needed to spin up to be "unique" creates a vacuum sucking in ever-crappier writers to fill the yawning blank page need.
Books have ALWAYS been 99%+ schlock - publishers recently confirmed it. They only ever back and give shelf space to the handful of "established" writers that churn out quantity schlock, like romance novelists, Danielle Steele, and the like.
I personally think AI will be an IMPROVEMENT on this, because then at least we'll be able to get books in different genres of a given quality.
Or imagine being able to drag and drop a couple of artists and assign percentages - 10% this one, 20% this one, 40% this one - and then the AI creates a new radio station with original compositions inspired by that allocation?
Same deal for shows and movies. That sounds like a strict improvement to ME over the endless "sequels and schlock" we get today.
> You realize you're literally describing the pre-AI state of "arts" today, at least as it pertains to movies, shows, and books?
Yes, and this is the result of automation and streamlining, but AI is the ultimate expression of automation. Imagine it gets worse to the point where it's no longer feasible to find anything good, at all. Writing a book is costly signalling. An AI writing a book might technically be costly, but if the cost is in the cents or dollars range, then it no longer functions as a barrier.
For me, what I hate is not that bad content exists. It's that good content exists, unable to be found, even though the writer and I are both desperate to connect. It's difficult to find such content today, but in the future it may pass the point of no return. AIs reading AI-written works to try and assess their humanity and skill level.
Culture feels like nuclear fission, and AI feels like it's padding out the fissile material until nothing can collide.
> Imagine it gets worse to the point where it's no longer feasible to find anything good, at all.
But AI should help with this, too! Above and beyond the "mixing and aggregating" things I mentioned.
Literally the only thing I use Gemini for is summarizing Youtube links so I don't have to watch them - this is particularly great when it's longer than a few minutes. You can also ask questions about the video, and it will answer. Sometimes, that process will inspire me to actually watch the thing.
Your personal AI assistant can consume and filter impossibly more media than you ever can, and you can train it on your specific tastes. Then you can send it out to the schlock mines and have it come back with gold, or at least denser-than-usual-ore.
Everyone's going to have one of those assistants - there'll probably be some free ones (like facebook is free), mass market will have $10-20 a month subscription ones, and richer people will have more expensive ones with humans in the loop to greater or lesser degress. But all of them will be performing this function. Arguably, we could all use and benefit from these NOW, because human-generated schlock is quite deep enough to need it already.
I like the sausage. But if I find out and don't like how the sausage is made, I would advocate for making that way of sausage-making illegal, because I know that a free-market solution is not going to help if that sausage has no competitors or has a competitive advantage over the less harmful ways, and/or consumers don't know the harmful ways of the sausage-making to make an informed decision.
A majority of the source images used by AI were generated by humans. AI is basically painting with human stuff. On the other hand, so-called human-credited art uses computer-assisted tools and techniques, sometimes to the extreme (the contemporary work at least). Using a 3D art program is an example. The bulk of the final image is the work of the computer program, but the human gets credit. In lieu of "prompts" are procedures the human artist follows.
There's not much difference between AI image generation which requires human prompts, and human-created art that requires sophisticated computer programming.
Have you ever created any digital art?
Yes, lots of it. Computer programs do the heavy lifting. There are decisions to be made, but rendering a flock of birds using a 3D program might takes 5 minutes. Doing it by hand is a different matter altogether. Human gets credit for both.
I'm not following. Once you're at the rendering stage in 3D graphics, all the human effort is done. The human effort is in the modeling, texturing, and building the scene, which definitely isn't something that can be done in 5 minutes for a complicated scene like a flock of birds. Why are we describing the rendering part as the heavy lifting of digital art? And how is this process at all similar to prompting an AI image generator?
Example 1: There was a landscape and environment program called E-On Vue which supplied ready-made environmental elements -- trees, flowers, rocks, water, sky, clouds, planets, soil, etc. I used it for a few years. Yes, you need to tweak settings to get exactly what you want, but even if you don't make adjustments, you still would get an image resembling something from an AI image generator. Using Vue was akin to being a photographer -- positioning the camera, and determining how much light was needed in the scene. You can make a simple scene photographically realistic in 5 minutes. Look for image examples on the web. (Here's one that I'm sure took around 5 minutes to set up. c. 2010 or so. Rendering the animation took longer. There are 5 ready-made objects : sky, ocean, tree, sand, girl.
https://youtu.be/7fjMKq0WmVs?si=hWKfMqCIngT5F3V2)
Example 2: Download a simple bird object, or purchase one, then import it into a 3D program. Use the "sprayer tool" to add 100,000 random points into a scene and have the program link your bird to each point. Select a sky backdrop from the program's assets. Finally, have the program generate an image of the flock of birds in the sky. The final image is the work of the computer program, but the human gets credit. Obviously it would be beneficial to have 3 or 4 bird objects with wings in difference positions. There are global rotation settings to get more variety, too. Yes, you have to learn how to do it and decide if you like it or not. It's not much different from prompting an AI bot, in that anyone can come up with an image without having any drawing skills. I would agree that 3D scenes can take many hours to set-up and render, but the machine is doing the work. It's similar to when I spent an hour of trial and error trying to get Midjourney to make an image I wanted -- and it still wasn't right.
In those cases I would say it's not the computer that does the heavy lifting, rather it's the human artists who made the assets. I suppose I can see the similarity between using pre-made assets to construct a scene and prompting an AI. But if we consider the digital art creation process as a whole, including asset creation, the effort is mostly human and not particularly similar to the AI process, in which the computer really does do the heavy lifting.
In the limiting case where AI generates "good" images from minimal prompts with no or very little correction, I'd call the human role closer to commissioning a work of art than to creating it.
My comment is only about categorizing human vs AI in terms of Scott's survey questions. Visual artists deserve credit for everything they do. I've spent many hours wrangling with illustration programs -- it is work! and one needs an artist's brain/training to make meaningful decisions along the way.
Biggest update for me is that I'm completely bamboozled by Impressionist and other abstract art styles; they already look so "inhuman" that I can't find any handholds to discriminate on. Which is personally fine, those genres are a turn-off anyway so I don't mind if AI ends up dominating them...I also notice that, despite intentional curation away from "DALL-E house style", some of the AI images in the test have that exact vibe, which is what made them really obvious targets for me. It's easy to overcorrect - I'm pretty sure I waffled on Rainbow Hair - but for now, it feels like a useful heuristic still. Will be interesting to see if human production of AI-style art increases to follow demand. I assume someone wants images in that fantastical-veneer style, anyway...
I don't know whether AI will really dominate in the fields of Impressionism or Modernist abstraction – those are both historical movements so I'm not sure what there is for AI to really "win" in those cases.
Rainbow Hair looked human-created to me because it lacked the flaws in and around hair that the AI-created images had. In the AI art there were inexplicable gaps between sections of hair that should be solid and there was also hair wicking off into the background that wasn't connected to anything. This is exactly what Ilzo meant by "bad and incoherent" details.
Both of those art styles are commonly sought by people as prints for room decor. It is possible that decorative art (as opposed to "high" art or illustrative art) will be dominated by AI generated images in the future.
> Human. This is Hatsune Miku, a “virtual idol” from the late 2000s/early 2010s.
Excuse me? What? That's not even the right hair color! That's just an anime girl with twintails!
Hair color for anime girls is always fuzzy. The artists says it's Miku, it's Miku pretty much. Even on official art it varies quite a lot.
That was also my reaction. Miku is very recognizable.
I agree - that doesn't look any more like Hatsune Miku than any number of anime art pieces not intended to be anyone.
+1
It prompted me to Google Hatsune Miku to verify my memory. Apparently she's typically teal-haired, and drawings of her drift from teal to both blue and green.
That said, I'm sure I've seen that image as, like, a videogame cover on Steam, or something. I don't think it's Hatsune Miku.
Yeah I thought the same. Doesn't she have a tattoo on her shoulder as well?
Yeah, my initial reaction was "no way that's Miku, I've been at her concert".
But maybe that's what makes it clearly human - no AI would take "Hatsune Miku" as instructions and output that, you need human creative freedom or significant prompting.
The two types of girls are Hatsune Miku and Kasane Teto. All girls fall into these two categories. YoRHa 2B? Miku. Ame-chan? Teto.
Rainbow Dash? Miku. Shimakaze? Teto. Despite having blue hair, Rei is Teto.
I managed to narrow down my answer set in the .xlsx, I got a score of 74% which I suppose is pretty good. Got fooled by the tough ones, but looking at the spreadsheet I'm surprised by how many folks were fooled by ones I thought were EXTREMELY obvious like Minaret Boat, Turtle House, and Landing Craft. Would love to see more in-depth research on people's abilities/strategies to distinguish between human and AI.
I got turtle House wrong because I thought I knew the name of the artist who made it, but I had confused it with another "giant animal carrying a city" work by Serena Malyon (https://www.serenamalyon.com/).
I could see that it looked very AI but I thought I had specific information that it wasn't.
I got 61% and got Turtle House wrong! I guessed that at least some of the more "Dall-E House Style" examples would be human-generated (Scott did say he was trying to match them), so there was a lot of second-guessing going on.
Really? Turtle House was one of the ones I was immediately certain was AI.
I thought there were more obvious ones, especially ones with weird anatomic detail. Snow Princess has a peculiar left eye (her left), Strings Come Alive has odd fingers, Girl in Field has odd eyes, Muscular Man check out the reclining man's left hand. Minaret Boat and Turtle House I got right, but I thought Landing Craft kind of nailed the shitty sci fi covers of my youth.
It's a tortoise for crying out loud.
I'm more inclined to believe something is made by a machine when I'm acting like one...
Underrated comment. I couldn't complete the survey. It was fun for the first ten, but then yeah, I started feeling like a robot. It's like with my casual perusing of the subreddits for spotting AI art. I think, "Ooh, this is strength-training," then I think, "Eh, we can just downsignal AI art and tell people not to submit it." Which is where we've landed as a society.
When someone says an artwork of theirs was made with AI, or is AI art, what does that mean exactly?
At one end of the scale, it may mean it is the unmodified output of a (fine-tuned) model given some (human-optimized) prompt. At the other end of the scale, there was some AI output used, but it has been heavily modified, combined or otherwise transformed by the human using various tools, such that the work is overwhelmingly human-created.
I presume Jack can tell you what his AI art process is, but what about the other artists you have chosen to feature? Were they asked what they meant? I find that different AI artists have very different definitions.
Jack wrote in another comment that their process is the first end of that scale: giving prompts to Midjourney and using the unmodified output.
> The Paris Scene (which fooled the most people) had this prompt: "Impressionist painting of a bustling Parisian street under the soft rain, 1881"
So it seems unmodified AI art really can fool most people. The most liked image was also made like this.
There are a gazillion well-known Impressionist paintings of Paris streets in the rain c. 1881, such as Caillebotte's famous one from 1877.
https://www.artic.edu/artworks/20684/paris-street-rainy-day
The prompter appears to have cleverly chosen a year not too long after Caillebotte's big "Paris Street: Rainy Day" breakthrough when talented but less original painters were hopping on the bandwagon, so there are many decent paintings to imitate.
I would imagine a prompt of "Impressionist painting of a bustling Parisian street under the soft rain, 1781" would be more of a challenge for AI.
What I've learned is that the AI images submitted by Jack were really tricky. I think nearly all of the ones I got wrong were his.
The biggest surprise for me is how good AI is at replicating impressionism. Perhaps there are tells but I don't have the eye to see them.
This is Jack - I'm pretty proud that I fooled a lot of people. Going in I had the goal of making the art that fooled the most people, thrilled I achieved it.
Are you willing to share a description of your image generation process?
See here:
https://open.substack.com/pub/astralcodexten/p/how-did-you-do-on-the-ai-art-turing?r=eyeio&utm_campaign=comment-list-share-cta&utm_medium=web&comments=true&commentId=78078533
Thank you!
Thanks for posting this mate!
I thought your pieces were unusually good, easily the best ones in the mix. The lesson for me is that human discernment still matters - you still have to have an eye for what works, even if you're just creating a million images and sorting through the best ones. Curation is still part of the artistic process.
I think it's easy to overlook the fact that AI art is still created by humans and still involves human aesthetic judgment. The computer is a tool just like a drawing tablet or a paintbrush - you're still responsible for the decisions you make with it.
What do you get when you give Midjourney a date before the existence of the style? Such as "Impressionist painting of a bustling Parisian street under the soft rain, 1781"
I tried it in Ideogram (because it's free to use), and the year didn't seem to make much of a difference, though with 2081 I got cars in 2/4 renderings. Overall I was impressed by the output. I'd suggest you give it a try.
I called #11 because of the gibberish pseudo-autograph in the bottom left and #20 because of the different-sized windows. Didn't see anything obvious in the other two.
Fun exercise and good article! I'll say again that to my eyes, the blue haired anime girl and black hair anime girls are absolutely obvious. The black haired anime girl looks like most AI-generated anime girl (I think the model was orange or something? It was when I tried it), this kind of blend of photorealistic but also anime. I would have expected Miku to be drawn way earlier than that, as the style is more 2000-2010 than post 2010. I guess what I'm trying to say here is the more experience you have with a particular subset, the easier it gets to distinguish it.
There's also something important here: with AI we're all seeing artists as they grow. This is something we aren't used to seeing, especially with old painting. I really really like Rothko, but I like his """newer""" paintings more than the ones he did before he really found his style.
You know, I don't spend much time examining anime girls but somehow I could also tell which was which. It was a "gut" feeling, but a strong one.
I got the same gut feeling. Something is off with the girl in black, but I have no idea what. I wonder what it is my subconscious has picked up on.
I have never seen an anime character with hair like Anime Girl in Black, there were far too many strands coming off her head. It would be annoying to draw and even worse to animate.
I've seen more complex than that but yeah the hair was a bit too much all over the place.
I got both wrong, weeb credentials in danger. It's also amusing that the blue girl has something seriously weird going on with her left hand, which I didn't notice in time, but would've made me even more sure.
I'm sorry, I'm going to have to take your weeb cards. As for hands, human artists struggle with them too.
Ice Princess was the other obvious anime one. String Doll also basically adjacent to those two. All of the ones I was most confident on were these anime pictures, because I've only ever seen AI-generated anime images look like that.
Yeah true. Personally I wouldn't even consider them exactly "anime", it's kind of like between anime and 3D render if that makes sense? Or maybe a new art form that is mostly possible thanks to AI.
Couldn't the people who got 98% just have been AI haters who cheated with your answer key? Didn't totally understand why you included one
Yes, or they wanted to test how good an AI detection software would be. See also my post below.
Wouldn't people who cheated get 100% not 98%? Especially since going by the comments here, they all missed different ones for that 1/50?
Could most humans distinguish a genuine Da Vinci from.a copy, especially if viewed through a computer screen?
You mean a copy or an AI imitation?
This is irrelevant, but I quite recently learned it is incorrect to refer to Leonardo da Vinci as ”da Vinci” because it isn’t his name. Correct is ”Leonardo”. Same goes for other people from the era before real last names.
(It is pretty snobbish to care about this kind of thing but I thought it was interesting so am mentioning it. )
Either or.
Isn’t a painting seen on a computer screen by definition a copy? Unless it is a digital painting.
There aren’t that many Leonardo da Vinci paintings so there isn’t much training material for the AI.
There as some really good fake Leonardo da Vinci painting out there, like the notorious six copies of the Mona Lisa featured in the 1979 BBC Documentary "City of Death" which were indistinguishable from the original except for the words "THIS IS A FAKE" written on the panels with a marker underneath the paint so it would show up on an x-ray.
Not surprised Victorian Megaship has such a skewed rate if people are mostly thinking about painting. Since it's a 3D render, it's not AI, but it is also not rendered by human hand. Would be a bit like throwing a couple photographs into the human art mix. What did the breakdown of guesses on the other 3D renders look like?
I was baffled by the inclusion of that too -- not clear why it's not addressed.
A big part of the confusion is from the topic itself. AI Art got a tendency to generate this kind of super-proportioned megastructure with a ton of repetitive details.
I’m amazed by the quality of the AI art here, so much better than anything I’ve seen. What platforms should I use if I want to tinker with AI art
I like Ideogram the most, it even handled such a difficult case as my description of six humans without confusing the details (e.g., giving one person hair color of another). But it usually requires effort in prompting and dozens of generations for the result to be good. I once generated a total of a thousand pictures in one day because I was optimizing that hard.
I did very badly on the first part of the test. I took the "please don't spend a lot of time looking at the details" directive to heart, and my gestalt impressions were largely wrong. When "allowed" to look at the details, I got all 5 right. But there's a difference between "this looks good on initial viewing" and "this is a piece worth coming back to again and again."
I'm not super interested in visual art, but love me some stories, whether they're books, movies, games, whatever. I have some background as both a writer and a critic, though I'm definitely not a professional at either. And I'm constantly frustrated by the quality of narrative I'm expected to enjoy - stories that all my friends act like are fantastic but that seem to me to be kids bashing action figures together.
There is a place for that kind of story. In fact most folks would call some story of that kind their favorite. But that doesn't make it good, durable, lasting art.
To get to the point, if 95% of people say "IDK, looks fine to me" and the 5% best-trained critics are calling it slop, I'd expect that it's slop. I don't know what makes an impressionist painting "good" because I'm not into impressionism anyway. But presumably it filled a specific artistic need at a specific time - maybe art critics had seen 80000 hyper-realistic renaissance portraits and were sick of looking at hyper-realistic renaissance portraits. I don't know if showing random internet folks decontextualized versions and specifically instructing them not to look at them very closely can capture that kind of nuance.
Art is ultimately just a means to an end -- an ointment to make our existence just a little less painful. Just because you're harder to please doesn't mean your opinions are more correct, nor does it make you a better person. But as long as we are talking about good stories, ahem...
Have you heard of the critically acclaimed MMORPG Final Fantasy XIV? With an expanded free trial which you can play through the entirety of A Realm Reborn and the award-winning Heavensward and Stormblood expansions up to level 70 for free with no restrictions on playtime.
So addressing the meme first - Final Fantasy XIV is easily the best story I've ever experienced in video games and possibly in any medium - though I'm wary of my own recency bias and Arcane is giving it a run for its money in terms of time spent bawling like a baby :D
I'll push back a little on the other bit - I think really good art has the ability to change perspectives and challenge people. I wouldn't go so far as to say that AI art isn't art. I don't think that's a reasonable position to take about...basically anything. I think you can probably do plumbing artistically, though I'd prefer you not do it in my house.
But I also think there's a connection between being willing to treat art as more than just something to pass time until you die and personal growth. Maybe I'm a snob but all the people I know with the least interest in art and narrative are also the people who are least likely to consider the feelings and perspectives of others. And all the people I know who are defensive about comforting art that isn't challenging are also immature in other ways.
I don't think that being harder to please makes your opinions more correct or makes you a better person. But I do think consuming a lot of art, and seeking out art that's challenging will both make you harder to please and make you a better person.
Meh, my dad is an art snob, and he's also basically a psychopath. Though, I do have a feeling he's appreciating art for very different reasons than you and I... The point is, there's more to art than empathy, and there's no reason why it would necessarily make you a better person.
...It sure as hell hasn't helped me. I mean, I probably would have killed myself a long time ago if I didn't have access to this much entertainment, but in retrospect, maybe that wasn't a good thing for anyone involved...
Anyways, thank you for recommending Arcane, I was holding off on checking it out but you're finally giving me a reason to watch it.
Like all status stuff people can definitely use art criticism as a cudgel to bully people, so very fair point.
I don't know you or your situation but from my experience the main thing to remember is that time continues and things change - it's not just this forever however much it feels that way now.
And yeah - Arcane surprised me. It takes a lot of tropey characters and makes them act very human and form very believable, sometimes complicated relationships
> And I'm constantly frustrated by the quality of narrative I'm expected to enjoy - stories that all my friends act like are fantastic but that seem to me to be kids bashing action figures together.
Thank you! This is exactly how I describe "writing" now in the age of crap streaming and "everything is a sequel or reboot."
Obviously their method is putting a bunch of sets and action figures pertaining to the show in a room with a 5 year old, and then, fingers eagerly poised, they breathlessly transcribe whatever said kid comes up with over the next hour. That's E3 of season 2, post it!
But when I say this to people, they look at me like I'M crazy, when writing quality has obviously plummeted nearly universally to "below schlock" depths.
I generally avoid known AI output (including art), but this isn't because I think it's bad in the "ugly" sense; it's because I don't know when it'll start posing memetic hazards and the downside of avoiding it is negligible.
(I also oppose all usecases for AI that result in money going to frontier labs in any way, because I want those labs to not have money to end the world with.)
Could you post a link to the Youtube short you mention?
Wow. Chilling. YouTube really shouldn't allow videos like this unless they're clearly marked as AI.
Seems like pretty conclusive proof of something I have long suspected: impressionist art sucks and everyone is just pretending to like it because it's high-status. Just like "cultural studies" or whatever the Sokal paper was about.
I think it's the opposite: most non-arty people say impressionism is their favourite, while the real art snobs are into abstract expressionist stuff that most people think their kid could do a better version of.
I'd guess it's probably the lowest-status of the famous art movements other than maybe 19th century academic style, and also probably the most genuinely popular.
I assume it became a "famous art movement" in the first place because it was so bad that only status-seekers would pretend to like it, and then it got too "popular" so the really hardcore status-seekers had to distinguish themselves by finding even worse things to pretend to like. Barber-pole fashion and all that.
Whether or not that's true, abstract expressionism itself is many cycles of that old by now, yet remains unpopular except among art snobs, while my impression is that impressionism is genuinely popular nowadays.
Right. Monet might well have produced a larger number of pretty paintings than any other major artist in history.
Combine that with a good story about young rebels battling the stodgy Establishment and you've got vast popularity.
I think what it says is that Impressionism is a style that AI can do very well because it lacks the kind of details that AI tends to get wrong.
Also there's only two subjects for impressionist painting: French countryside or Paris street. This makes compositional pastiche pretty easy.
Impressionism is weird - it does have a lack of detail and focus but the merit is in recreating the sense of a scene in the viewer. The former elements work well for AI because it can just copy the structure of other works, as other people pointed out, without the details that would give away either the AI generation or the plagiarized sources.
Most artsy people I know don't like impressionism. It gets taught a lot in art history classes, though, because it marks a huge turning point in artistic styles. It also seems to be pretty popular among the general public.
The general public doesn’t like it because it is high status. They like it because the pictures are pretty and it doesn’t require the kind of background knowledge a lot of older art requires to be appreciated fully (like who is Saint so and so).
I take pretty strong issue with this. First of all I really really like impressionist art, as someone who struggled to enjoy much of Art History classes, Impressionism actually evokes an emotional response in me on rare occasions, especially in person.
But I have two other objections: First, impressionism might fit the skillset of an AI artist better than other genres. Second, a piece with mass appeal is not better than a Monet just because more people like it... art does have some objectivity mixed into the subjectivity.
1. AI is often quite strong at creating a mood or vibe, using strong lighting and universally appealing forms. Small wonder its good at making impressionist pieces that appeal broadly to people. It often fails on details, but details aren't the point of the impressionist piece. It also tends to draw more recognizable forms and does less abstraction... which makes the piece more accessible, and more easy to see. But you could compare that to junk food... its satiating, broadly appealing, but low in caloric value.
2. Spotify has lately been exploring creating AI-generated music that they can play without paying licensing fees, which often fills up the top spots on their most popular playlists. An example is the Rainy Day Jazz playlist, which has a very moody piano piece that a non-jazz listener might hear and instantly feel that rainy day mood they were looking for. Its evocative as a piece, but it is paper thin in if you have any jazz knowledge at all. It is vapid and empty, doing the bare minimum of establishing a vibe without saying anything at all. That's what I think AI impressionism is like... if i had any training in the medium, I suspect I wouldn't like the ai pieces at all.
More on that spotify thing: https://www.honest-broker.com/p/the-fake-artists-problem-is-much
Yeah this is exactly what AI Impressionism is like. It's vaguely "nice" and "pretty" but it doesn't communicate anything. If you look at e.g. Pissarro's paintings of Paris you can feel how cold the streets are, you can almost feel the clop of the horses' hooves and the intimidating blank facades of the Haussmann boulevards - the AI version is just "isn't Paris nice :)".
But a lot of people like "isn't Paris nice", and also, Paris is nice! Wouldn't hang it on my wall personally but I don't think of it as bad so much as different.
>AI is often quite strong at creating a mood or vibe, using strong lighting and universally appealing forms.
I've gotten my best luck from AI image generation (using Adobe Firefly, since I'm an employee and get free premium access) by giving it chunks of poetry, song lyrics, or mood-heavy prose. Tolkien and Lovecraft are particularly fruitful sources for the latter. It often gets something with an interesting and fitting aesthetic, even if the details are often not quite what I was going for.
Great job selecting the images for the test. However, I’m unclear on the real takeaway from your experiment.
If you show people original Monet paintings versus copies by a talented forger, most wouldn’t be able to tell the difference. Yet, we wouldn’t conclude that the forger is on par with Monet. Originality and being the first to create something of lasting importance are key aspects of good art. This point feels uncontroversial.
Your experiment seems to demonstrate that a human skilled in prompt engineering can act as an effective forger. People can reverse-engineer impressionism using tools like Midjourney. While this says a lot about Midjourney and AI, it doesn’t say much about art itself.
The more important question seems whether AI can create a new type of imagery that marks a milestone in art history, akin to what perspective or impressionism represented in their time.
In the long run, the answer is likely yes. But as of today? Which AI pictures from your test would best qualify?
This hits the nail on the head. And by this token, truly non-trivial AI art would “fail” to “pass” as human precisely because it was doing something new that no human could ever have done. I suspect it would also be broadly unpopular with a human audience, for more or less the same reason.
This makes me think of Orson Welles's F For Fake, and the Chartres Cathedral monologue at the end.
I think AI has already met the second milestone. The DALLE "house style" is instantly recognizable, stylistically unique, and influential (if my Facebook feeds are any evidence). Future art history books will absolutely include these types of images.
True, today it is "low status" and doesn't influence the "right people" but I would not be surprised if this style is picked up by certain types of human artists for exactly that reason (if it hasn't already)
Interesting!
A comparable example might be the advent of advertising posters in the late 1800s. But did they really influence art history that much? Toulouse-Lautrec, Mucha?
I am not sure what you mean by that question. I'm not very familiar with art history but both of those people have wikipedia articles indicating some degree of notariety.
I think part of the problem is that the categories AI vs. Human don’t have sharp boundaries. The favorite AI picture by Jack Galler is very reminiscent of Van Gogh’s Café Terrace at Night. So you have a human, Jack Galler, guiding AI, to mimic a beautiful painting. In fact, what we are seeing in Jack’s pictures are Jack’s style of using AI. He is the artist.
The Blue Hair Anime Girl and Rainbow Girl are both human but could never have been created without digital art software that has a large repertoire of “tools” that create elements that have no physical tool analogs. Human created Blue Hair Anime Girl and AI created Anime Girl in Black are simply variations of the same thing. Both are done with digital art programs, perhaps one more AI than the other.
//The favorite AI picture by Jack Galler is very reminiscent of Van Gogh’s Café Terrace at Night. So you have a human, Jack Galler, guiding AI, to mimic a beautiful painting.//
I think this is a bit unfair. I mean, the AI image is certainly similar in theme and overall layout, although not colour balance, painting style, mood or details ('minor' things like whether you have buildings or a river on the right).
And so what if it's reminiscent? Are all the other (human) artists who happen to have painted scenes vaguely like that copy-cats?
I'm pretty sure Van Gogh was not the first artist to paint a terrace café at night. And more to the point, 'all' he was doing was 'mimicking' a scene which was literally in front of him.
Jack Galler is damn good at making AI images pass as human-made. He fooled me in almost every single one and nothing else fooled me
my application for membership in the BTFO By Jack Galler club
8 of my 13 misses. gave 90% confidence on pretty lake. picked leafy lane as my favorite.
Based leafy lane enjoyer
Hey it's me, Jack Galler. It's validating to see how many people I fooled.
You dear sir, have earned the title of Artist.
t. Fellow person who got fooled.
We honestly need to run the test again but without Jack.
Alternatively we need more research into what makes someone good or bad at creating compelling AI images. Human judgment still important!
Some human artists also draw details that make no sense. For example, to me, many of H R Giger’s painting make no sense when I look at the details.
(And also, when professional artists look at my sketches, one of the things they often comment on is that I’m making the details make sense where often people who just put in something vague. So my own drawing style is the opposite of this )
In fairness, a key element of Giger's work *is* the fundamental "wrongness" of what is being presented. That's why he's so successful in horror - the detail gives the image weight and the nonsense of them disturbs the mind.
H. R. Giger's painting make "sense" in that they are deliberately constructed to jar the viewer with certain kinds of incongruity.
Yes, agree that it’s probably intentional as part of the horror aesthetic.
In the spreadsheet screenshot, presumably "Digital AI" should say "Digital Art", as some of it is not AI?
The Ingres fooled me into thinking it was AI. But it wasn’t the Greek text; it was something about the coldness of the style that made it seem like AI.
Sorry, Ingres, thought you were a robot.
Yep. And I’m sure he messed up at least one hands placement in the crowd, right foreground. Fooled me too: damn his Gallic wiles.
I actually recognized the Greek text, I hope I realized from that that it was by a human. I actually don’t remember any more.
Coldness and severity are what Ingres was aiming for, rejecting the rise of romanticism.
That was one I was fairly sure was human, just because of the consistent variety of the characters. The classical and renaissance people look markedly different and consistent among themselves in a way that my experience would say would be quite hard to get an AI generator to do.
I didn't even see there was text until people pointed it out in the comments. I got it wrong because it looked like Jesus was Zeus and also Columbus was there.
Was Ingres the famous painter most likely to be using a projecting device in David Hockney's theory?
If Ingres were out on the technological frontier 200 years ago, that might help explain some things?
One nice thing I can do with AI art gen is create pictures in the style of Leonid Afremov, who was impressionistic and post-impressionistic. I bought some reproductions from his studio which were half by him and half by studio artists. While his own reproductions were on average a touch better, one of my favourites was by a studio artist, and one of my least favourite by him. Only one was clearly not his to the untrained eye, it was so bad it actually seemed good eventually, like a fading memory of a strange dream. It's possible that estates of deceased artists could have a single robot painting high quality reproductions, improving their finances and the availability of the art style
I correctly identified Giovanni Girolamo Savoldo as human, but my reasoning was: anatomy and perspective a bit rubbish in places, but seems to be from around the sixteenth century, so I’ll give him a pass on the errors and conclude human.
Savoldo should have taken a look at the works of Dürer (rough contemporary) to see you should do it…
Art does not exist in a vacuum. Knowing that a piece of art has been shaped by human experiences, emotions, and skills adds a layer of connection and meaning. This human context is also why people care about chess played by humans, but comparatively few take an interest in AI only chess. Yes, you can lie to me and tell a compelling story about the context of an artwork which was actually created by AI, well done.
If you define "art" as "pretty picture that was drawn by a human that was feeling emotions when drawing it", then fair. AI will never make "art".
But that's a very non-central example of what people mean when they say "art". For 90% of the population it means "pretty picture".
It may be that most people mean “pretty picture”. But the people who care most about art usually don’t care if it’s pretty, and instead care that there is more to understand and learn that adds to what you can visually see. (In the case of Duchamp’s urinal, this is very explicit.)
We're in agreement. Should we just taboo the word "art" then?
To me, it was obvious that Scott was trying to show that AI can draw well enough that it's indistinguishable from masterful humans, and sometimes "more popular" drawings than humans.
I don't even disagree with OP's message. But seems like a non-sequitur in here.
I want to see the prompts Ryan used, and also, which AI. Especially for Angry Crosses.
Many of my favorites were his!
Thanks! :-)
I'm glad you liked them! :-) Everything that I did was in Stable Diffusion.
Here's a link to everything I handed off to Scott. (It's a lot of images. Apologies in advance.)
https://drive.google.com/drive/u/1/folders/1Lw1ngwPdipU39uB7ygAHdrs0SWsRt6vF
I didn't scrub these images, so if you open them in something like notepad, you should be able to find the full positive and negative prompts as well as the models used.
Best!
The spreadsheet has image titles like "S0nt in Mount0ns" and "Blue H0r Anime Girl".
Presumably at some point you changed from coding the responses as "AI" and "Human" to 0 and 1, and then find-and-replaced "ai" with "0".
At first I was skeptical of your method. Since you asked for good AI examples, which puts a thumb on the scale, and then you only took the best of the good stuff sent. On the other hand, if you are not doing 100 versions of a result for AI/Art Music you will get mediocre results. So I guess I'm fine with it.
As a musicians when Suno and Udio came out I was really curious and started working on some AI songs. My wife was horrified, and refused to even listen to the compositions. So I made a song to mimic the style of one o her favorites "Tom Waits", and just stuck it in a playlist during a long drive, she raved about the song being one she never heard by him, and one of his best..LOL people like art because of signals. There is beauty, but most people want things that raise their status or signal their in-group and beauty is only a small part of that, and sometimes is a Negative signal.
https://www.udio.com/songs/4EZcR4XES6if22xKgNdrjA
Here is my tom waits like song written by me and udio, and lyrics by me and chat gpt.
How did your wife react when you told her the truth?
I didn't!
Oh no 🙀
Sorry, I listened with an open mind but this is just bad. It has none of the distinct images that we'd expect from a Tom Waits song, and is a tangle of clunky metaphors. The images in the Tom Waits songs are distinct, tell a story, and the lines scan. Recognising quality isn't about "signals".
But look, even people who aren't familiar with Tom Waits can see this. Anyone scrolling by, please weigh in which verse looks like it was co-authored by ChatGPT:
***
"There's a ribbon in the willow and a tire swing rope
Oh, and a briar patch of berries takin' over the slope
The cat'll sleep in the mailbox and we'll never go to town
Till we bury every dream in the cold, cold ground"
***
"Tell me, is it the crack of the pool balls, neon buzzing?
Telephone's ringing, it's your second cousin
Is it the barmaid that's smiling from the corner of her eye?
The magic of the melancholy tear in your eye?"
***
"Streetlamp spills a pool of amber on the cracked sidewalk below
Fat cat perched on a fire escape, eyes like polished onyx glow
Hear a couple's laughter, light and carefree, a melody that stings
My smile's a rusty hinge, happiness barely hangs on its strings."
***
"Well, Friday's a funeral and Saturday's a bride
And Sey's got a pistol on the register side
And the goddamned delivery trucks, they make too much noise
And we don't get our butter delivered no more"
I feel like you missed the point of the larger ACX post. Not saying this is somekind of masterpiece, but lots of peoplle can't tell it's AI. You think you can, great, I personally am not a big tom waits fan. So if you say the story telling isn't as vivid, sure it really feels like cope. Like those lyrics are more random, but not every song he writes is like that, Christmas card from a hooker is just pure story. The point of the post is blind that even people who hate AI also perfer it. We could do a test and bet on the results. But I think it would have a similar score as the AI art above. The less familiar you are with the art, the more alike it will seem. Like there is no extra finger in the music, I think people would say it's a song and not AI. As a big Fan, you are focused on the narcisism of small differences. As a musician I feel like the tools could be much better and that the intro is too long, and I see things that I'd change, but much of that is on me for not spending more time and doing more versions.
ALso if you want randomness, I went for that in a song, I do this for fun...
https://www.udio.com/songs/p7sZ2PKh3WaREGaKvpPuae
Very random lyrics, that was the point. Trying to mimic the style of Beck, cause I'm old. Still I could have worked harder and made it better, but this is a dude doing fun shit while I'm on business trips. Imagine if I was actual serious about this stuff.
The thing is, the Waits lyrics aren't random, right? They're striking and novel images that combine together to build up a sustained impression, a mood, a sense of place. The young adult night time adventures of Looking For The Heart of Saturday Night is distinctly not the same setting as the childhood misdemeanors of Kentucky Avenue, though they both are formed of brief narrative snapshots.
The pieces chosen in the survey were cropped and curated to be maximally fooling, but I grant you that plenty of people are bad at spotting AI creations. Whether people can spot its provinence or not, I think most people who enjoy lyrics in music could discern a difference here. Maybe I'm wrong!.
As for Waits, his pure story ones I suspect ChatGPT would do even worse at emulating. I'm not sure it could be as distinct as this:
***
"Hey Charlie, I think about you every time I pass a fillin' station
On account of all the grease you used to wear in your hair
I still have that record of "Little Anthony & The Imperials"
But someone stole my record player, how do you like that?"
***
Waits has a set of images in mind that he is conveying through characters. Generated text is even worse at consistency than generated illustrations. There's a reason why novelists aren't worried about the robots coming for their jobs yet. It's not a "small difference" to notice why one song has interesting lyrics and another is a series of cliches. But look, let's test this. Here are five verses, one is ChatGPT, the other four are different lyricists I enjoy. I'm sure as someone who has played around with AI text you can spot the imposter very easily:
*** 1 ***
You tell me I'm not not cute
Its truth or falsity is moot
'Cause honesty's not your strong suit
And I don't believe you
*** 2 ***
I was watching your mouth move in and out
In little unexpected ways
Not hearing a word that you said
In over twenty minutes
On occasion I grunt and give you a knowing stare
Focused on what I hope is a pivotal moment
*** 3 ***
I am cold and rainy,
I am dirty as a glass roof in a train station
I feel like an empty cast iron exhibition
I want ornaments on everything
Because my love, she gone with other boys
*** 4 ***
The city hums beneath my window
a hymn for no one in particular
I sip the wine you left behind
it tastes like time and something bitter
*** 5 ***
And as for my inflammatory writ?
Well, I wrote it and I was not inflamed one bit
Advice from the master derailed that disaster;
He said "Hand that pen over to me, poetaster!"
As someone who doesn't know any Tom Waits, I'll go for the "Tell Me, is it the..." verse. Mostly because it seems sloppy to rhyme "eye" with "eye" rather than any particular narrative failure.
This is a super-interesting exercise, thanks for doing it! The question this is raising most urgently in my mind is “what kind of test like this would everyone agree is fair?” We can’t randomly select from all existing human art, it is skewed towards what has survived. So you had to filter the AI art for what seems good enough to compare. It doesn’t seem like a fair test but I have no idea how to do better or whether it’s possible to do much better.
I think that whenever we start worrying about fairness, we should think about why we care about fairness in this application, if we even do. When we are playing a game, fairness is very important, and it usually means that each player has similar possibilities available to them (though there are some games that are intentionally asymmetrical, like Twenty Questions, and others, like chess, that have only a slight asymmetry, but fairness is achieved by playing as both white and black in a series of games).
But in some applications, like figuring out whether someone is a competent driver, fairness is less important and some sort of accurate representation of an underlying skill is more important.
I don’t think this test is aimed at revealing some specific truth that needs specific conditions to reveal it, but I also don’t think it’s a game where winning is important. So I don’t think fairness really enters into it.
I can't help but feel that this was, in some sense, a way for Scott to show that the people who think AI is terrible at art (or whatever specific complaints they have) are probably letting emotions unrelated to the judgments of the output dictate their assessment of AI's
"ability" to create art. In that sense, fairness does seem relevant.
However, to me, the whole thing is a bit of an irrelevancy. To my knowledge, none of those "AI" artworks were generated by AI. They were generated by humans, prompting and then selecting for whatever outcome they wanted. Many of the AI image generation programs have the ability to zero in on pretty specific areas of an image, to create custom models, etc. AI feels like a tool for artists. I wouldn't want every piece of art to be created through AI any more than I'd want every piece of art to be created through Photoshop or Maya or whatever, but I think it's inevitable that we'll see a large portion of art created through AI because it's very accessible and the skills required to do it are very different (and probably a bit lower than most other more "artist" tools in terms of required practice for mastery, at least for the moment). I mean, even with all the digital tools we have, there are many artists who still labor over a canvas. I'm curious as to how these differences in methods to create art influence what human explorations attract artists.
As someone who consumes very little art, my point of view is that I'm interested in when AI art gets good enough that I can just ask for a work with a fairly straightforward prompt and get approximately what I'm looking for (e.g. I just asked DALL-E for a Yithian at the optometrist's office, a somewhat humorous take on that rather bookish fictional race, but didn't get a very good image). So, I'd guess a slightly better experiment than this one would be if the AI image curation step was also automated, to match the (minimal) human effort needed in the use case I'm (admittedly only mildly) interested in.
I think the test as performed is a useful test to see if the current crop of AI is capable of generating at least some images that are hard to identify as AI generated. You don't need to go back all that many years for answer that question to be an obvious and trivial "no".
If you want to test the overall quality of AI image generation relative to human artists, that's a very different question that would require a very different testing protocol. Off the top of my head, I'd suggest getting together two groups of volunteers, one people who are good at writing AI image prompts and curating the output, and the other human artists of various skill levels. Give each participant a certain number of un-engineered prompts to illustrate, plus feed the raw prompt to various AI image generators without curation. Then crowdsource ratings of the images on dimensions of technical merit, originality, fidelity to the prompt, and aesthetics.
>I like this picture. There’s nothing wrong with it. But somehow it’s obviously AI. If you asked me why, I’d say “something about the lighting”.
Also the two cats to the right of the throne are wildly deformed. One doesn't have any forelegs, and the other has some terrible conjoined foreleg. The Cat Queen (?) 's legs do not make sense with her torso. The candelabras don't make sense. Etc.
The lighting is a clue, dollars to donuts the prompt includes "HDR" or something similar that describes the effect and so much AI art defaults to that style, but the whole picture has a lot of AI art's "traditional deformities."
Yep. The smaller cat beside a stool on the left also has weird paws and isn't sitting on anything visible.
The way I figured Victorian Megaship to be human was the fine details. Even though I've never set foot on a sailing ship, the rigging was just too consistent for AI to get right, and everything about the ship just made sense. It was plausible.
Contrast that to Minaret Boat, where nothing makes sense: Windows of different shapes and sizes thrown haphazardly about the minarets, and the ropes behaving in nonsensical ways, like ropes splicing into two or more, or hanging in shapes that gravity wouldn't allow.
Generally, the lack of world knowledge when it wasn't possibly the point of the piece was my #1 tell that something was AI. The gross anatomy on Muscular Man, the free-floating earring and asymmetric accessoires on Anime Girl in Black, the free-floating wax on Still Life, the child-like perspective errors in Ominous Ruin, and so on.
Is minaret boat like a McMansion then? Just a sloppy mishmash of styles with no point?
Much worse. You can at least live in a McMansion, but I wouldn't set foot on Minaret Boat to do anything other than a short tour to satisfy my morbid curiosity how bad it would look on the inside.
Exactly the same. The two ships were the ones I found most easy to distinguish. Victorian Megaship's consistent lines, rigging passing behind sails and always coming out the other side where it makes sense, is something I've seen AI have a lot of problems with before.
Of course, the favorite AI painting seems to be fairly directly copying an existing human painting: https://www.google.com/search?q=cafe+terrace+at+night+van+gogh&client=ms-android-google&sca_esv=7de4e3da6f1aa72e&udm=2&biw=412&bih=783&sxsrf=ADLYWIIQ8GQWpML-I7M83AsMXEqFxz3EXQ%3A1732111411081&ei=M-w9Z53YBP6qur8PkdWmwQ8&oq=cafe+terrace+at+&gs_lp=EhJtb2JpbGUtZ3dzLXdpei1pbWciEGNhZmUgdGVycmFjZSBhdCAqAggBMggQABiABBixAzIFEAAYgAQyBRAAGIAEMgUQABiABDIFEAAYgARIiSxQ1A9Y3yVwAngAkAEAmAG0AqABpByqAQcwLjguOC4xuAEByAEA-AEBmAIToAKPHagCBcICBBAjGCfCAgYQABgHGB7CAgcQIxgnGOoCwgIKEAAYgAQYQxiKBcICDRAAGIAEGLEDGEMYigXCAgsQABiABBixAxiDAZgDBIgGAZIHCDIuNS4xMS4xoAfBMw&sclient=mobile-gws-wiz-img
Boom. Headshot.
And of course the Van Gogh has unique qualities scrubbed out by the AI
I think the AI version is closer copy of Pissarro’s Boulevard Montmartre series:
https://impressionistarts.com/camille-pissarro-boulevard-montmartre
The element of having a terrace on the left at night is the same, and they're both impressionist, but everything else is different. It gives an initial impression of being very similar, but the more you look at it, the more it seems like we're just matching on a couple elements.
Of course, the original Van Gogh painting is fairly directly copying the actual terrace he painted.
@Scott What happened to the free comments you asked about in the test, are you going to do another post on them?
I don't think it's possible to analyse free comments from 11,000 people. You can read a few to get some vibes, but that's probably it.
Unless you let an AI do it.
Surely there is little difference between making a highlight post out of a 500-1000 comment substack post (as Scott often does) and out of a similar-sized random sample from the contest?
The substack comments are sorted, so it is possible to follow only threads which are interesting. That makes a huge difference. I looked into a few of the answers, but I already gave up after 10 or so. It's just completely unstructured.
This was a well-designed test which proved to me something I didn't fully realize before: good art requires good reasoning (planning, reviewing, erasing, backtracking, rethinking, etc.). Generative models paired with Large Reasoning Models will be a huge jump in quality.
(Looking forward to this particularly for Music Generation.)
I started to take this test, but after looking at 6 or 7 images, realized that using my dinky phone screen for viewing made the exercise pointless. Maybe the viewing medium is key to determining the humanity of the message. Does AI art do chalk, oil, pastels, watercolors on various surfaces? Until it does, who cares?
I did the test on my phone then realized I got a few obvious ones wrong when looking again at the pictures on a laptop. I wonder how much of an impact that would have on the overall results.
At the current moment, far more people experience chalk, oil, pastels, and watercolors through digital photographs of them rather than in person. But I do hope that the rise of AI art makes people remember that the physicality of these media is one thing they have going for them, and start going more to museums to get a fuller experience.
Man, 'Paris Scene' really messes with me. During the test, I recall explicitly looking and that and being like: "I've seen that before, must be human"
Quite humbling to realize the picture I was completely wrong about even the picture I felt the most certain about
You kind of have, probably; as noted in another comment, that one is "inspired" by a specific real painting.
I too felt pretty sure that had to be a Renoir, having fairly recently been to a museum where I was exposed to a large collection of actual Renoirs.
You might be confusing it with Renoir's "The Grand Boulevards": https://en.wikipedia.org/wiki/The_Grands_Boulevards
which yes is extremely similar and it makes total sense why your brain would pattern match that picture to your memory of the Renoir.
You missed the conclusion that humans were better at identifying Ai art in genres they were familiar with. I am not exactly an impressionist or classical art connesiuer; I muddled around looking for brushstrokes. On the digital side, and especially the anime art side, I did quite well and was very confident because it's the sort of image I actually view regularly (online). The anime girl ones couldn't have been more obvious.
As another example, your friend the artist is infinitely more in tune with "art" than I am, and did very well.
hah I had no idea on almost any of the pieces (I commented elsewhere re: which were obvious to me) but the anime girl ones were TOTALLY baffling to me. I didn’t even have a vague hunch of an idea. Totally outside of my experience.
If the test gave the images in a higher resolution, I'm fairly certain the results would be much more accurate.
It makes sense that people would do worst with impressionism. One of the main "tells" of AI art seems to be small details. Impressionism tends to have a "blurry" quality that lacks small details.
But if you look carefully you can see that even big details are slop with AI. For instance, #44 was apparently the most human-voted AI generated piece. I identified it as AI by noting the incorrect bicycle wheel arrangement and the lamppost that turned into a tree. The gestalt could be human but Degas would never have made dumb mistakes like that.
The people who got almost all of them right were 100% cheating.
I think that’s fine, as they didn’t skew the results much and it’s interesting to see with tools that people can identify AI images. They were either using an AI image detector, really zooming in to the images and looking for inconsistencies, or using something like Error Level Analysis which makes most human vs. AI images mostly trivial: https://fotoforensics.com
Maybe some were cheating, but it's certainly not an impossible task. I think I got about 85% and all the ones I got wrong I had very low confidence about. If I was a half-dozen coin flips away from getting 100% and there were 11,000 people playing, the top scores seem pretty likely.
I mean, Scott posted an answer key along with the test, so all they had to do to cheat is look at it and mark their answers
I'd think a simpler way to cheat would be to simply do a reverse image search.
It was a good test but there are a decent number that are recognizable human paintings. I noticed a few and looking over it realize I should have caught a few more too.
1) After reading your explanation, I feel like the test is way more biased than I originally thought. Just as an example, you deliberately avoided the "AI house style", except you *did* include human works close to that style, like the extremely detailed ship. You also excluded complicated poses, specifically because the AI is bad at them. WTF?
2) For reference, there was a similar test over at DSL, except it put pairs of human/AI images of the same style side by side. In that test, the entire forum ended up with a score close to 100%. https://www.datasecretslox.com/index.php/topic,12393.0.html
3) The test, whatever it measures, isn't a good indicator for AI slop. The slop-ness comes from looking at a corpus of images. You can make one pretty decent AI image, especially with human curation. If you make an entire card game where all the images are AI... well, they will fulfill the function of a visual aid to distinguish cards from each other. Similarly, the non-slop-ness of human art depends on historical context. Contemporary impressionist-inspired paintings hang on the walls of cheap hotel rooms rather than museums, alongside Bob-Ross-style landscapes.
4) In no way shape or form is this a Turing test. Please fucking stop abusing this term.
I somewhat agree but want to note that there's a reason "5 guys stand next to each other" lineups are no longer best practices for criminal identification and I think it at least partially applies to this too.
When you do the more accurate sequential lineup you're asking the witness, "are any of these dudes the suspect?" When you do a side-by-side lineup the question you're asking the witness is "which of these people looks most like the suspect." If it's guaranteed that one answer is correct, you get different information than if you don't.
I'm not saying the DSL test isn't useful - it's just asking a fundamentally different question than this, and I'm not sure that question is as interesting. Instead of asking "if you see a piece of art, can you tell whether it's AI?" it's asking, "if you compare two similar pieces of art, can you tell which one is AI?"
I agree these are different tests. The DSL one might have some issues, but it's much better at exercising the AI over the entire space of possible images. The styles that Scott used are very, very broad, much broader than something like "art noveau". And it turns out that AI is pretty bad at actually doing art noveau, as it mixes in a lot of its default style, which allows you to spot the fake from a mile away.
Yeah the choice to only include art intended to fool people on AI and human sides (as well as cropping the images in ways that change their impact) makes my minor objection a quibble. I still wanted to make it :D
I did expect about as much biasing as Scott described, which made the test, now incorporating my mental model of Scott and a bunch of reverse-reverse psychology epicycles, not so fun for me. A test deliberately designed to exploit dominant strategies to confuse people is fair test for subject-matter experts and/or people who make very confident pronouncements, but that doesn't make it a referendum on those confident pronouncements.
Right. The 60% absolute number isn't calibrated against anything so we can't draw any particular conclusion from it. However, we can draw conclusions from the *relative* performance on the test, i.e., that people with knowledge and/or disdain of AI art are actually better at recognizing it.
The AI art in that link doesn't match the style of the human art. The faces in the AI art are all in a modern, realistic, digital painting style. So of course the forum gets close to 100%. That test is "way more biased"...
Or it would be if it were trying to prove that AI can't make portraits that humans can't distinguish from human art. It's not possible for a test to be bias in a vacuum: the test measures what it measures. The bias comes when you try to interpret the test results as indicating something that they don't actually indicate.
You wanted this test to test whether people could tell average DALL-E AI art, or AI house style art from human art. What this is testing is closer to, "can humans tell good AI art from human art if we avoid some of the known AI weaknesses, like hands and text". You're calling it biased because it's not testing the question you wanted asked.
The answer to "can humans recognize mediocre AI art as AI?" is obviously yes. "Can humans recognize the common AI house style?" is also a yes. The answers to both of these are obvious enough that I don't know why you'd want to waste the test on these questions other than that you don't like AI and want to score some points against it. It's not interesting; it's just political fodder.
Scott is not the only one abusing the term Turing test. Just about everyone on the internet is also.
Yes, it's a sysyphean task...
Thank you.
I could easily program an app that pulls random words from a dictionary, have it run a trillion times, then personally select sets of words that appear to be human-like sentences, then ask people to compare those to actual human sentences. And if humans could only tell 60% of time it could be said that my random word generator passes the "Turing test" as mush and any AI art generator was proven to in this exercise.
... Which is to say, not a all. You can't have a human filter on the computer's output and still call it a Turing test.
> I’m afraid I don’t know enough math to tease out the luck vs. skill contribution here and predict what score we should expect these people to get on a retest. But it feels pretty impressive.
There's a pretty simple method to do this for binomial outcomes like this - conceptually, you measure how much variation you'd expect across people due to luck, subtract that from observed variation to estimate how much variation is due to skill, and use those two factors to weight how much to regress to the mean
pseudo-code:
```
stdev_observed = std(individual_results)
stdev_luck = sqrt(
0.60 * (1 - 0.60) / 50
)
stdev_talent = sqrt(stdev_observed**2 - stdev_luck**2)
samples_for_regression = (
0.60 * (1 - 0.60) / stdev_talent**2
)
for result in individual_results:
individual_talent = (
(result * 50) + (0.60 * samples_for_regression)
) / (50 + samples_for_regression)
```
Hm, I am a bit skeptical. Using standard deviations seems to be the right thing for events that are not very extreme, like for estimating the 90% quantile or so. But here we are looking at extreme values, something like the 99.95% quantile, and I don't think that standard deviations are helpful in this range. The binomial distribution does not decay quadratically in this range, but exponentially.
i think you have a point, although i'm not sure what would help you more when the underlying samples are so small. this could be a lower bound at least.
Yes agreed, it would give a bound.
Well, if we assume that it's binomial then we can always go back to the exact formulas for binomial distributions. Or use a normal approximation of the binomial distribution.
I tried to get a feeling of what 5 people being 98% correct means. Also, further down, TheIdeaOfRyu suspected that some people may have used the solution, for whatever reason.
I looked at the highest percentages from people who scored on all 50 pictures:
98%: 5 people
96%: 3 people
94%: 2 people
92%: 4 people
90%: 7 people
88%: 16 people
86%: 17 people
84%: 48 people
It is indeed extremely suspicious that the range 90%-98% is so evenly distributed. I think that any reasonable probabilistic process should have those numbers dropping very fast with higher percentages. The only exception is that a small number of people have a magic ability to tell AI from human, and that this ability is not gradual (some can do it a bit better than others), but that it is on/off. So, probably that was with the solution key, and I assume that most people above 90% indeed used that key.
For the 90%, perhaps a helpful intuition is this: assume that we have some very good recognizers who have an easy time for 30 pictures (90% success probability), and who are at chance level for the other 20 pictures (50%). Then in expectation they would get 37 pictures correct (74%), and the probability that they get at least 90% correct is 0.2% if I computed correctly. So if we had 1000 such good recognizers, we would expect to see 2 of them score more than 90% by a mixture of chance and being moderately good.
Of course, this is just a scenario.
I judged #15 as AI-generated not because of the text but because of what seemed to me like AI compositional incoherence: it has a Greek temple with various Ancient Greek-looking figures, and then e.g. in the bottom left corner we have...Sir Walter Raleigh and a group of Renaissance gentlemen?
Am I hopelessly confused about something?
It’s “The Apotheosis of Homer”. Ingres is painting all the great people of history he thinks were inspired by Homer, including Shakespeare, and his own contemporaries like Mozart.
https://en.m.wikipedia.org/wiki/The_Apotheosis_of_Homer_(Ingres)
Exactly what happened to me!
I thought that same thing was a sure sign of it being human! Renaissance gentlemen are likely to draw paintings linking a line of cultural descent to Renaissance gentlemen from the Ancient Greeks.
Same here! I saw the various styles, started recognizing a few, and literally said "There's no way an AI could make this." The amount of effort put into copying those famous artworks into another one with such precision would be crazy with AI prompting.
I think "incredible detail, often recursive-like, in 'unimportant' places" is what gives the "obvious AI" look. This explains why "Victorian Megaship" looks AI-generated.
4, 16 and 26 all have mangled hands.
> People say AI can’t invent new styles, but I’ve never seen any human make this exact type of weird bird.
It's a bit obscure, but for me that bird was really reminiscent of the Brazilian modernist painter Chico da Silva, who specialized in weird colorful animals (birds and reptiles in particular). See e.g. https://www.independenthq.com/features/chico-da-silva-brazils-forgotten-visionary for some examples. It's kind of the exception that proves the rule, though: AI art might not be brand new or innovative, given so many humans draw so many things all the time, but it's certainly way easier to find new art you like by prompting DALL-E with "weird colorful bird" than discovering a particular obscure painter by serendipity.
Maybe before taking the test I didn't read all the preliminaries, but I kinda wasn't figuring that digital art created by humans would be included. I was thinking, did pigment and paintbrushes create this image?--if not, then AI.
Which image(s) did the 5 top scorers get wrong?
Flailing Limbs, Rooftops, Giant Ship, Blue Hair Anime Girl, Cherub.
If I didn't know about them in advance, I would probably have determined pretty much all of M. C. Escher's stuff was AI generated. Yet it's great stuff, partly for having been an original concept well-executed.
The original Turing test was not "generate a lot of realistic conversations by either human+human or human+AI, then take the most human-like conversations and let people judge whether this was h+h or h+AI". What you did here was much easier.
I think no AI can pass the more turing-like test of "make a (complicated) prompt, wait 10 days and then judge whether the result was made by a human or an AI", if you really try to craft prompts that are hard for an AI (which is the idea of the turing test). Maybe even add the possibility to ask for changes.
None of the major image generators are designed to do that. I imagine you could make something like a GAN with an LLM and diffusion model where the discriminator attempts to distinguish human images from generated ones (assuming you could get the funding). But a big use case of AI image generators is making images that don't look like images humans have made, and you'd be crippling that use case by training it to only make man-made looking images. There are only a few big players in this space and none of them have focused on that.
The current models need a prompt, so this test is testing what they're capable of with a good prompt.
I know. I just wanted to point out that this test is much easier than a real turing test would be and that people shouldn't draw wrong conclusions from this.
One of the aspects of AI paintings that I most often picked up on was a lack of brushstrokes: people’s skin is too smooth, for instance. I think one reason the AI impressionist paintings fool us is that “make an impressionist-style picture” tells the generator to include what looks like brushstrokes. The impressionist work that I found easiest to identify as AI was the one with the woman, because, again, her skin looked too smooth.
Petition for future things like this to send a copy of your answers to the inbox of everyone who provides an email. None of my friends or I saved our answers :(
I definitely wanted that!
Or at least let you input an identifier so you can find your answer afterwards
Congrats to Ryan and Jack, their pictures are really good
Thanks! - Jack
Thanks! - Ryan
so all it takes to create AI art that's mostly indistinguishable from human art is an AI to generate the art, plus a significant amount of human time and effort spent prompting, adjusting, and then curating the AI art side-by-side with human art in a format that's deliberately arranged to make them difficult to distinguish
I was so sure about 49: Mediterranean Town being human (as I saw "mistakes", that looked human), and so sure about 15: “The Apotheosis Of Homer”, by Jean-Auguste-Dominique Ingres (1827) to be glorious AI (the 3 guys at the feet of the green lady: rather garbled, aren't they?!). ... With a 60% hit-rate I declare me a failure in this test (or declare AI as the winner). Though the choice of pics seems to have been kinda even more unfair than Scott admits: taking AI pics that looked much more human and human art that looked more AI ish than I would have expected - if you did a fast run of this test, the algorithm "judge as 'human' what looks AI, judge as AI what looks 'human'(BOCTAOE)" might have worked better than long deliberation on each - and much better than "follow your instinct".
In regard to the throne room with the giant cats: Why are there so many lit candelabras in bright sunlight?
I don't think I did very well on the test, but there didn't seem to be a convenient way to check my answers, many of which I didn't remember, against the key.
Still, that picture stinks of AI, probably because implausible giant things are commonly AI, and the details are sharp in an odd way.
Yep. AI has problems with scale. Another tell is that basically the entire image is cats, and that nothing in the image has any implied meaning or purpose beyond "Throne room with cats"
Well, the human on the king cat's lap could be construed as a humorous take on a human royal family with a cat sitting on the king's lap. I can't tell which of the kittens is the equivalent of the crown prince... :-)
>Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
The key point here is that they can't tell when it's heavily curated by people with decent taste. An average piece of AI art is hideous slop. Most human art is terrible too, obviously, the difference is that it's much harder to produce, so there's not a comparable deluge of it.
Also, as we learned from Darwin, your designer doesn’t need to be any good if there’s sufficiently sophisticated selection going on.
"Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history. "
All this tells you is that humans ("readers of this blog") can't tell the difference between great art and hideous slop. And that's a fair cop! If you'd asked me before, I'd have agreed with this.
The subsequent paragraph about forcing people to confront what "art" is is the main point here. A lot of the comments section is people arguing about what it means to "not like AI art" and how much curation effects things. The critique of the gate picture gets at this, too. The interlocutor finds the lack of meaning in the details displeasing, but when I looked at that picture, I didn't look at it closely enough to notice or care about that. So what is art? Who judges its quality? What is the point? These are all up-stream of an "AI Art Turing Test", and frankly, it's the more interesting discussion.
Yeah, if we're using "walruss, whose personal aesthetic is essentially not having one, can't tell the difference between a Van Gogh and a convincing AI imitator" as the test of whether AI can create art then...er...please stop doing that.
Maybe art isn't meant to be enjoyed completely abstracted from all context, in 30-second assessments (if you took that long on each picture). Like a good poem, in my experience the best art has layers you peel back the more you study the scene/composition/facial expressions/etc. the artist included in the work - often related to the title of the piece. This is why things like the gate fail. They don't stand up to more than a cursory review. How can it become your 'favorite piece' that you go back to again and again if there's nothing beyond the superficial?
I think this plays into what people are trying to convey by invoking 'AI slop': it only stands up in a world where you look at it once and then you're done with it. For it to 'work', you have to flood the world with superficial images, not art that can be enjoyed by returning to it again and again. Maybe you're not forced to return to the same images if you're constantly supplied with new content to enjoy and then instantly forget. Slop, served up by the bucket-load. But then you've lost something beneath the surface. It's like the difference between curling up with a good book and doom scrolling.
I've long said that the only reason why people can be fooled by the AI thing at all is that we've lost the ability to judge and think about art in a meaningful way anyway. Again, I'm not a visual arts person, but this is definitely true with narrative art.
Or maybe I'm just complaining that nobody in my friend group wants to sit and talk about the same book or TV show ad nauseam for 3 days straight and I do :P
I don't think you're going to take humans out of the art-generation process anytime soon, even with the introduction of AI. Not a single one of the images Scott used got into the contest without significant human intervention at multiple steps along the way. Selection/curation will likely continue to be a feature going forward, and who knows but that we'll see much better stuff coming from AI as humans re-generate subsections of the images to force an artist's vision into the final product. Who knows?
But the problem with "people don't appreciate art" is only going to get worse the more of the current approach to AI art we use. If a large part of appreciating art is the process of uncovering the subtle meaning hidden beneath the surface, there's a problem in flooding the everyday experience with 'art' devoid of any such hidden meaning/intent, until nobody knows when to start really looking at an image because maybe there's something more to it. With the AI art status quo, the world becomes a Rorschach test, devoid of all but the meaning you give it.
Would you be equally upset with people who go to an art museum to look at paintings, but don't bother to read the tiny placard explaining the history of the painting? Are such people merely consuming slop rather than properly appreciating art?
I'm not really upset. And sometimes I'm the one going around not reading the placard. My lament is that the discussion we're having necessarily removes context and any kind of deeper introspection about the intent of the piece. If all we care about it whether something looks good from a superficial perspective, that's fine. I think there's absolutely a place for superficial art - and probably a big place - but let's not pretend it's the same thing as something with intent behind it.
I just think it's folly to equate AI art with something that has a deeper message to convey, especially if we do that by conducting an experiment where you necessarily strip all deeper messages in order to make the comparison. That's like comparing two poems based on whether they both rhyme without considering whether they make sense or have something deeper to say about the world.
>I just think it's folly to equate AI art with something that has a deeper message to convey
Does anyone really do that though? I'm pretty sure that everybody is doing the equating only in the shallow "pretty pictures" sense.
Scott's post seems to be at odds with this conclusion, especially the final section. The place where a distinction is being made between art with a deeper message and "pretty pictures" is here in the comments. Otherwise, I can't see how Scott would write the following:
"Alan Turing recommended that if 30% of humans couldn’t tell an AI from a human, the AI could be considered to have “passed” the Turing Test. By these standards, AI artists pass the test with room to spare; on average, 40% of humans mistook each AI picture for human.
What does this tell us about AI? Seems like they’re good at art."
That seems like equating art with pretty pictures to me. And, yes, he then goes on to say, "I’m more interested in what it tells us about humans." And if his complaint about humans were directed at the blinded graders in his test, I'd expect him to say something like, "Humans don't take the time to appreciate art, its nuances, and its deeper messages." IF he'd said that, I'd agree with your point.
But Scott went the other direction, attacking the art world and the tendency for human veneration of a deeper message in the total absence of aesthetic appeal. I can't think of a more clear way for Scott to equate AI art with something that has a deeper message ... and then discount the deeper message entirely as being irrelevant to whether the art is good or not.
This point doesn't apply to the digital artworks, but for all the others: I really think it's worth considering that none of these are paintings; they are either digital simulations of real paintings or AI-generated simulations of real paintings. So what this is asking isn't "can AI convincingly simulate real art"; it's asking "can AI convincingly simulate a digital simulation of real art." It would be a lot easier to judge an AI-generated simulation of a real painting against an *actual* real painting.
And the actual painting contains a lot of the depth and artistry that makes it meaningful - in the application of the paint, the texture of the brushstrokes, the scale of the work, and so on. That's why looking at a Pollock or a Rothko or a Van Gogh on your screen is nothing like seeing it in person. A digital representation of an artwork is really just a thumbnail. We're just so saturated in screen-mediated imagery that we take it for the basic form of artistic experience; and quantitatively it does make up the vast amount of that experience. But it still isn't the experience of the artwork itself.
AI clearly doesn't have the ability to convincingly simulate *that* experience. Will it some day have that ability? What it would require is a body - that thing that actually applies paint to a canvas by gesture and so on - before we could even start having the conversation. Which, incidentally, is what AI would require for there to be real questions about AI having personhood or consciousness.
This feels like moving the goalposts. Before AI art became good, would you or anyone else have said that the essence of art lies in the physical brushstrokes? That any art that's not an oil painting can't possibly be "real", let alone beautiful or meaningful?
I 100% would have said (and in fact did say - you'll have to take my word for it) that the essence of art lies in the experience of the art object itself, not in its digital reproduction. Like I said, a Van Gogh painting just isn't the same thing as a digital image of a Van Gogh painting.
Other considerations would apply to other media, like film or literature, but this survey was about paintings.
https://images.masterworksfineart.com/5f9cba39-438a-4ace-8fc4-b360387dba6c.jpg
But this is --> |
;-)
As someone who is typing on a windows machine,
>But this is --> |
made me homesick for UNIX. :-)
Only some art is made with brushstrokes. I hope that one thing people get from the rise of AI art is that for the art made with actual brushes, there’s something you get from the real thing that doesn’t quite come through in a hi-res digital photo on your screen.
I'll assume that something like a 3D printer hooked up to an AI counts as a "body?" And you'd need the paintings to be scanned with a laser distancer to generate depth maps for training. Then, you could apply the depth maps along with the colors to do some kind of 3D printing. Stable Diffusion already has AI mediated creation of depth maps based on digital images that works reasonably well. I'm actually playing around right now, in my free time, with a workflow that allows me to create textured images from digital images, with just a little bit of tweaking. Sadly my 3D printer is very much consumer quality and prints from filament, which doesn't help on the details, so the texture of the prints prior to sanding and coating are a dead giveaway that the work is 3D printed, not painted. But all the key limiting technologies look like they already basically exist.
In any case, I don't see why greater diversity of media relates to 'personhood' or 'consciousness.' We don't fault the great painters for not making sculptures or the great sculptors for eliminating the element of time and movement from their work. Creating in more dimensions may be a challenge for general intelligence, sure. But couldn't we distinguish between generalizability and personhood?
Does a photograph count as human art? The photographer doesn't, like, select individual RGB values for pixels, but he does carefully choose a scene and framing. Many people consider this enough to make the photographer an artist and the producer of the image. It seems to me that the high level of curation taking place with the AI art is taking a similar role. Perhaps the conclusion is something more like AI is one tool that a skilled human artist can use to create art having genuine merit.
One would think this would be the obvious conclusion from the beginning, but there has been much baggage.
If one does 3D art and describes "shape here," "shape there," "very realistic texture for that," few would have a problem with this. Even when much of it is generative: Foliage. Trees. Landscapes. Large structures.
The line seems to be conveniently drawn at what the beholder finds distasteful.
Photography is sometimes considered art - just like performance art, or baking, or anything else. But don't let the use of the word "art" make you think the two skills are at all similar, or that they produce the same results. Painting and photography are different things with different merits.
The prompt that produced the most liked image was, "Impressionist painting of a countryside café terrace during twilight, 1883". Coming up with that prompt and recognizing that the result looks good is a skill, but that skill is in a different category than painting.
Say you had a friend who painted and you asked them to create an 1880's style impressionist painting of a countryside café terrace during twilight, and based just off that prompt, they painted a beautiful painting. Would you take most of the credit for your skill in coming up with that prompt? I'd probably say my friend deserves most of the credit, while recognizing that coming up with good suggestions is also a valuable skill.
Also, it takes years of practice to become a master painter, and coming up with a good prompt seems like a much lesser accomplishment. Especially since with each new version of AI image generators, the good prompts become simpler. In earlier versions of Midjourney, people would append dozens of tags to their prompts to tweak the results. Now you mostly just tell it what you want.
I get the sense that you're trying to reason that humans are still special and valuable in this context and so you're trying to elevate whatever role humans still had in this art generation. But before you know it, someone will create an art generator that can work without humans generating the prompts, and then we'll have to invent some other reason why human creativity is more valuable.
Pretty sure I pooped the bed on this but I have no way to compare my guesses from a month and a half ago to the results. I guess I should have written this down.
Ryan and Jack please start substack a for your AI art I will subscribe
Wow, thanks! I do have a substack. I hardly post, but I suppose I should add some content. Right now I'm playing with using displacement maps generated in Stable diffusion to produce textured 3D prints.
Edited: Okay, I've posted a curated series of images I made while trying to figure out how to make a watery humanoid form. I'm not sure the best modality for sharing, so I hope this works.
This was interesting! It reminded me of a story on (I think) 60 Minutes from many years ago, looking at the work of painter Thomas Kinkade. Probably people know who Thomas Kinkade is, but just in case: Kinkade was an incredibly successful painter of schlock. How successful? "According to Kinkade's company, one in every 20 American homes owned a copy of one of his paintings." I have no idea if this is true, but it gives a sense of how prolific his studio was.
The segment mostly consisted of 1) a bunch of prissy art professors pointing out (correctly) that Kinkade's paintings are terrible, in a manner that was guaranteed to make them look like condescending elitists; and 2) a bunch of Kinkade-owning regular folks saying that the professors could go suck it.
Points being: popular stuff can be bad. But also it depends on what critical lens you are applying. An art historian is considering different things than a person who just wants to look at a pretty picture.
How is Kincade terrible schlock and Pollack a true artist?
Whatever you say about Jackson Pollock, it’s not schlock. There’s no sentimentality or appeal to superficial prettiness and fake nostalgia. Whereas with Kincaid, that’s all you get.
There are plenty of criticisms one can make of Pollock, but it’s not schlock.
That wasn't my question. Why is Kincade's work shlock but somehow Pollack is a genuine artist, when Kincade produces pretty pictures and Pollack produces the canvass equivalent of static?
Agreed. Pollock is just random static. He (or his followers) may be able to describe his grand intentions in great detail, but those intentions don't translate into execution.
At the time Pollock was working, he was doing something new and controversial that no one else had been doing. Whether it was a good idea or a bad idea, he was doing it as part of an idea (just like Marcel Duchamp with the urinal, and the impressionists with their hazy brush strokes). The Pollock pieces aren’t the equivalent of static - when I googled the images I realized that I was easily able to recognize Lavender Mist and Blue Poles as two of the most famous ones that I have seen many times, though I didn’t recognize some of the others. I’m sure I could learn to recognize Thomas Kinkade paintings if I looked at them much (when I googled I realized I had been picturing Norman Rockwell, who played a similar role a generation or two earlier) but whenever one isn’t familiar with a style, there’s a sameness there; and with Pollock that can seem like static.
It was new because no one else was willing to debase themselves and waste material like that before, and it was controversial because it was garbage. It doesn't matter than you can recognize two different stills of static, it's still static.
"Controversial" could be "because it's breaking a field out of a rut" but it could also be "because it's trash." Matt Walsh's documentaries are controversial because they identify naked emperors and expose a fraud most people were going along with. Some random internet poster is controversial because his posts are racial slurs and conspiracy theories. Those are not the same.
De gustibus! Kinkade's pictures aren't pretty. When I look at them, they seem humorously terrible. The color palette is bad, the compositions are flat, the subject matter is trite. Many people disagree, and my explanation for this is that they have bad aesthetic judgment. This also explains why people like well-done steak and many superhero movies. They can like those things if they want, I certainly am not here to talk them out of their enjoyment. Sometimes I enjoy those things too, even when I recognize their flaws.
Everyone has bad aesthetic judgment in most domains. I could not tell you what distinguishes good Chinese opera from bad Chinese opera, and I find it all basically unlistenable. But if a Chinese opera lover were to claim to hear a clear difference in quality between Mei Lanfang and, say, Black Pink, I would not insist that they are being ridiculous and arbitrary in their judgment.
Your criticism of Pollock, as far as I can tell, is "I don't get it and I don't like it." Fair enough! But if you really want to insist that it's impossible for others to draw a distinction between Kinkade and Pollack, it seems you're trying to make a virtue of your poor aesthetic judgment. That always strikes me as a weird move, but certainly you're not alone! Personally, I can admit that I have terrible taste in Chinese opera and it would be nice for me if I were better able to appreciate its qualities. Most people seem to feel deeply threatened by this sort of thing.
No, I'm saying "I do get it and the whole thing is a fart-huffing fraud." I'm also not saying Pollack and Kincade are the same, the point of my question is that the critiques of Kincade for, as examples, poor composition and mundane subject matter are undercut by the elite celebration of Pollack, who never even has composition or subject matter *at all*.
Right, I know that’s what you’re saying. What you’re saying is not complicated and in fact it’s probably the most common opinion in the world. People say, “If they like something I don’t it’s because they are signaling/sheep/idiots.” People don’t say, “I have unsophisticated tastes and am pretty ignorant on this topic, and my tastes probably reflect that,” even though this accurately describes where most of us are on most things.
Here are two propositions:
The “elites” are fools and/or charlatans who have elevated one artist and dismissed another despite there being no meaningful difference in the technical, aesthetic, historical, or intellectual value of their work.
Internet commenter Darkside007 is unable to distinguish the technical, aesthetic, historical, or intellectual value of the two artists’ work.
Bayes would put his money on #2. Of course, the matter is ultimately undecidable because there is no objective benchmark for these things, so you can stick with #1 if it comforts you.
Because as soon as photography appeared, painters no longer had the monopoly on pretty pictures production. So they redefined their job as an abstruse game, where the greatest artist is the one who invents and implements the weirdest idea nobody had before.
Beyond the content of the paintings themselves, Kincade was also just a grifter. His business model was adding a few brush strokes to each of thousands of reproductions of his paintings to make them "unique" and "valuable" (not really).
I don't have a strong feeling about the aesthetic qualities of his paintings, although they are a bit same-y. But I maintain that art is fundamentally a medium for human connection, and I do not believe that Kincade had any genuine and interesting message to convey through his art, therefore I do not think it is good art.
That's a good point, and I agree, but I'm not sure that this puts Kincade in a different category from Pollock.
I don't know much of anything about Pollock as a person, so I can't really comment on the sincerity/cynicism axis. However, I don't mean to criticize Kincaid's art on a purely ad hominem basis. I just think that, to the extent that his art had a message, it's not a very interesting or original one ("cottages are nice," "pretty things are good"). I point out his unscrupulous practices because it shows that he was more interested in making as much money as possible off of the few ideas he had rather than developing new, more interesting ideas to put into his artworks.
It's possible Pollock also adopted a similar cynical attitude once he achieved massive success. I'll leave it to those more familiar with his oeuvre to defend him from that claim. But I do think that he at least started out with some radically new and interesting ideas, unlike Kincaid. It might seem unfair, and the time and skill required to physically make a Pollock-style painting might be less, but he was at least trying something new and he succeeded in making at least some people feel something new. This is more than one can say about Kincaid and is why, in my opinion, his work has some artistic value, which does not of course mean that everyone has to prefer it on an aesthetic level.
...Oh my god, did Kincade pioneer NFTs?
Nah, you at least got a print when you bought a Kincade. With an NFT you get... nothing.
I've no idea about Kincade's silly business practices, I'm just comparing Kincade, who people mock but produces things I enjoy looking at, to Pollack, who people venerate but produces nothing but noise.
I think a useful comparison might be Mike’s Hard Lemonade and Laphroaig scotch. It’s very easy to appreciate a Mike’s Hard Lemonade, particularly for people with no training or experience with alcohol tasting. Laphroaig is much harder to appreciate, and even many people who like high end alcohols aren’t into the kind of thing it’s going for. But there is a thing to appreciate beyond just a surface sweetness.
Am I a terrible person for liking both ?
Not at all! But I assume that what you like about them is very different and they’re for different purposes.
Except Pollack, in this analogy, would just be something like raw lime juice, with people confusing harsh and bitter for a high alcohol content.
Raw lime juice isn't harsh or bitter. It's sour. I think you just don't have very high discernment, which again, is fine. But you seem to think that people who have higher discernment are just making shit up. I assure you, that's not (always) the case.
It relies on superficial appeal. Whatever else you say about Pollack, he spent no effort on superficial appeal.
The whole appeal of Pollack is the complete lack of any appeal. Irony is extremely sophisticated and not at all small-minded and short-sighted, after all.
While I don't own a Kinkade (though my late wife liked his work), I really like the view of
>2) a bunch of Kinkade-owning regular folks saying that the professors could go suck it.
The criticism of
>sentimentality or appeal to superficial prettiness and fake nostalgia
smells of political correctness.
Look, Picasso's Guernica "sends a message" all right - one that is every bit as banal as Kinkade's.
If Kinkade is schlock because his message is predictable then so is Picasso.
I think a significant part of my response to non-historically-signficiant art is rooted in “it’s cool you did this”. Not only do I enjoy the work itself because it aligns with my preferences in some way, I enjoy and appreciate the passion and directed effort that shaped the artist and art. At the very least, they thought a good deal over the years about what they want to make, how to do it well, what aspects to study or gloss over, how to square that with their own technical limitations and tics, etc. Seeing low-effort AI generations placed next to that just because it’s “detailed” or the lighting is fancy or whatever is like watching someone crashing a track meet on an e-bike.
I feel like a lot of the selection here was carefully curated to take the best AI generations and the worst art. A traditional Turing test is a live interaction between a person and a machine/human decoy, solving most of the selection problem. But if you gave the decoy a stock list of phrases they had to pick from, and allowed the machine to respond independently, you'd successfully fool a lot of people.
Anime girls are an excellent example of this - there's a ton of generic anime girls gazing at the viewer for AI to model on, and anime itself is a style based on minimal detail and a lack of shading. (One Punch Man expressly parodies this in how it presents Saitama normally and when he's "serious"). So the difference between a digitally-drawn anime girl and a generated anime girl is going to be very small.
Pointillism and impressionism also lend themselves to easy AI replication because they rely on the viewer filling in the details themselves. That gives a huge advantage to AI since it's not expected to be correct, just pointed in the correct direction.
Including images that seem to intentionally evoke AI generation like "Victorian Megaship" is just straight-up cherry-picking. Can human artists imitate AI generation? Yes. Duh.
And then there's the inclusion of outright crap. "Tropical Garden" is an eyesore, and it appears Hockney is an example of the modern art world's idiotic pursuit of ironic detachment at the cost of, well, literally everything else. No one would believe that was the intentional product of a human mind, because there is a base assumption that people are trying to create good works. This feeds into the idea that modern art is merely a vehicle for criminal transactions, because people aren't spending $8 million on this because of its merit.
And then there's Abstract "Art", which is just random gibberish. "Vague Figures", "White Blob", "Purple Squares", "Fractured Lady", "Creepy Skull", "Angry Crosses", "Bright Jumble Woman", and "Flailing Limbs" are all nonsensical trash, which AI can easily replicate because it's 100% noise and 0% signal.
(On an entirely separate note, I recognized "Angel Woman" as a Living Saint immediately, and then the Iron Halo, and then the Guardsmen and concluded "Not AI because IP :P)
For what it's worth, modern/abstract art was the category where I could most reliably distinguish Human from AI art (9/12 or 75% by my count, failing on Purple Squares, Mediterranean Town, and Punk Robot).
Obviously, AI struggles with realistic details in complex scenes but I think it also struggles with images that are too disordered or that appeal solely to emotions and high-faulting aesthetic techniques. It's hard to put into words, but the most abstract pieces were the ones where I had the strongest sense of "a human artist arranged randomness in a way that made intuitive sense to them" vs "this is a soulless machine wearing a human face"
I picked Tropical Garden as my most confidently human piece. The imperfections of it are very like a human artist and very unlike AI imperfections.
I think the broader point being made -- and while I wouldn't use the same tone as OP I do in fact share much of the sentiment and considered making a top-level comment to similar effect -- is that for art that's sufficiently aesthetically unappealing, the object level question of "can AI make art?" becomes secondary to the issue of "why would I care if it can?"
I wouldn't care for "Angry Crosses" if it had been done by a human, and I don't care for Basquiat, "Purple Squares," or "Flailing Limbs" or "White Blob" regardless of their human authorship.
I think it becomes a little easier to see why the question of aesthetic appeal is a predicate to caring about AI displacement of humans if you think about the issues of utilitarian production and goods demand: "AI can create slightly defective rubber balls with high efficiency" isn't an especially interesting headline because I don't actually want there to be more slightly defective rubber balls in the world. Similarly, because I want relatively less rather than relatively more aesthetically unappealing art in the world, the fact that AI is capable of producing ugly things is intrinsically less interesting and in a sense almost in a different ontological category than the question of how it stacks up to humans in producing pretty things.
I like the Hockney painting. It's certainly true that celebrity in the art world is completely fake and everyone should stop caring about it, but if I just across it in a little gallery somewhere I would still like it.
Interesting that your friend commented on the meaningless detail in the arch as a sign of AI origin, since this sounds like what Gwern identified as AI-like about R. Crumb's drawings in his "Review of Crumb", in the section "Crumb and AI Art": https://gwern.net/review/crumb#crumb-ai-art
The detail that packs Crumb's art may not have any depth of meaning associated with it, but it's organized in a very human way. Regarding the arch, "meaningless" is more of a synonym for "thoughtless" as in literally lacking in consciousness. Humans are drawn toward creating balanced visual patterns (balanced doesn't necessarily mean symmetrical) whereas AI creates semi-random shapes that may look balanced at first because they're sort of in the right place, but reveal themselves to be random if you look at them for a long time. These flaws just look very different to human flaws.
The original contest announcement said the respondent who scored the highest would be recognized.
Most of the time I knew I had no idea whether or not something was human or AI, and I was just guessing, but there were three pictures that I was absolutely, immediately, and correctly *sure* were human — the wounded Christ, the Agony in the Garden, and the apotheosis of Homer. It’s not just a religious art thing; I wasn’t so sure about the Madonna and Child (AI) or the saint in the mountains (human). But those were the only three I was sure about at all. Everything else I was very aware of coin-flipping in my head.
I thiiiiiink the correct way to interpret this is that AI is getting very good at generally looking right but isn’t yet good at deliberately visually referencing broader cultural referents in a convincing way.
In the Homer painting, the crowning with laurels, the visual references to various poets throughout history (Dante really pops out; I think with effort I could name some specific others, as well), the lyre, the obviously Greco-Roman architecture in the background (maybe I subconsciously picked up on the lettering; not sure; didn’t consciously notice it) all hung together in a way I just didn’t think AI could hit.
The Wounded Christ, similarly — the composition that carefully emphasized the stigmata, the way his face was dusky to emphasize his death, the reactions of the women, the man who was probably Joseph of Arimethea (but could also work as St. Peter or Nicodemus — I wasn’t sure) the crosses in the background — I just didn’t think it could possibly be AI.
The one that I was absolutely most sure about was the Agony in the Garden. The sleeping disciples, the angel coming down to minister (not Biblical but a common added detail), the soldiers arriving in the background, led by a figure that might be Judas — I just didn’t believe that AI could have so correctly put together the most common referents to that particular Biblical vignette, and I was right.
Although maybe all of that could be “fixed” by a human prompting it more specifically? Idk. But those were the tells for me on the ones I was absolutely sure were human. None of it was in the style of artwork at all. It was all in the broader cultural references.
Huh. All of the AI art has human attribution. I don't know the first thing about the process, but I guess the software is not prompted with "hey computer, paint a picture."
Yeah I would like to know more about what went on here!
It takes me about an hour of work to produce a pretty good image that actually resembles something I was intending. It takes another hour or two to really nail down the image with the details I want. I'm the farthest thing from an artist, but it's definitely not effortless.
> Sorry, I guess Giovanni Girolamo Savoldo just wasn’t very good at anatomy.
This was (basically) one of my take-aways: Many of the "obvious" indicators of AI art are only good indicators for specific kinds of pieces. Lots of art isn't trying to make a super-realistic depiction of hands (or pre-dates humans being able to do that) or other anatomy. And some humans are just not good at these things either. The Warhammer picture was another example of this phenomenon (to me, at least)--the hand looks like it maybe is supposed to also be one of those birds (AI art often has different things merged into one) and the other arm is pulled back at what looks like an unnatural angle. But these could be deliberately stylistic choices (Escher is famous for images in which different things turn into each other) and/or the artist choosing not to redo it for any number of reasons.
Also, I was confident that that ship with all the windows was AI for what seem to be the same reason you know that cat picture is fake; I figured a human would be unlikely to have the patience for all of those tiny features.
Wait, if good AI art exists then why do people insist on using the bad stuff?
Because it’s hard to get the AI to make good stuff!
For the same reason good TV exists but someone still expended all the energy and time to make Big Bang Theory.
My personal process for making good AI art:
- Mix together a number of different models/LORA/prompts to get a consistent style across pieces.
- Do a trial prompt, find it draws something stupid/not what I want (e.g. it interprets "belly" to mean pregnant, rather than a bare stomach), edit the prompt several times until I fairly consistently (50+%) get basically what I want.
- Gen 40 512x768 (or 384x768) images and pick the one that looks best.
- (Sometimes:) Edit the one that looks best to fix any minor errors or get positioning that the AI can't do (e.g. I could not get the AI to draw the girl with her hands clasped behind her back in a power pose, so I just physically rotated her arms in paint.net).
- Upscale 2x, ten times, with 0.4-0.6 error and ADetailer on. Mix and match between the best ones, editing one hand out for another that looks better, etc.
This all means making a decent piece takes an hour or two. Not all of which is me working, but it slows down the pipeline a lot.
Sloppers' process:
- Put in a prompt and get out a picture, then post it online.
This takes like one minute and approximately zero human effort. Sloppers churn out 100s of images per day; if I spend the day mostly on it, I might get out 5-6.
of all the ones that fooled me, I have the hardest time believing #40 Dragon Lady is pure AI.
obviously it has major "DALL-E house style vibes". But that reflection is pretty flawless. are we sure the creator didn't do a compositing pass?
Hah. Now although most of the classical art I had no idea about, that one I was very confident about because the reflection *isn't* flawless. The dragon's head is reflected but its forelegs aren't. The girl's hands have something a bit like a reflection that really doesn't match where her fingers are.
it's not perfect, but I wouldn't expect a human to produce 100% perfection either. and a photoreal reflection would have minor distortions from refraction, etc.
willing to be wrong - but registering my disbelief
on closer inspection I retract my skepticism. AI is just way better at doing reflections that I thought.
Yeah but look at the reflected nostril. Wack.
Jack and Ryan are AMAZING AI artists. That was my big take away. I'd totally follow sites/substacks just looking at AI art they generated.
I'd also like to know what image generators they used and the prompts - if those aren't trade secrets.
If they really are made with a prompt, rather than some more complex process!
I deliberately stuck to using prompts and curating only, as per Scott's instructions. But thank you for the credit.
For my own purposes, I do things a little differently.
Is there any chance we can persuade you to put up a post showing us something about your prompts and curation? I noticed that several of yours were among my favorites when I read Scott’s description, and I’d be glad to know more about how they came together!
Hey this is Jack. I just used Midjourney. Most of the prompts were just describing some generic impressionist scene. I've never had an art history class (but I am an art history fan!), so I worked with ChatGPT to identify key trends of say, Impressionism, so I could prompt better. I generated a lot of each one ,and just used my subjective opinion to decide what looked good and what didn't.
The Paris Scene (which fooled the most people) had this prompt:
"Impressionist painting of a bustling Parisian street under the soft rain, 1881"
The Riverside Cafe (which people liked the most) had this prompt:
"Impressionist painting of a countryside café terrace during twilight, 1883"
Adding the year might be genius!
Yeah, it kind of helps Midjourney know what you're looking for. I originally used it in fake photography, would use it for prompts like "Minimalist luxury Tokyo hotel lobby. Midnight. 1997, Kodak Portra".
The year and the photo stock really help clue Midjourney in. Otherwise it looks too new.
interesting, Kodak Portra was introduced in 1998 (https://en.wikipedia.org/wiki/Kodak_Portra) -- I'd love to hear from an AI developer how MJ reconciles the contradiction. I bet more-overtly-contradictory stylistic requests can produce interesting results.
. . . and Jack, you snookered me sooo many times! (shakes fist)
Here's the AI art that I handed off to Scott. (I'm going to take this archive down, eventually. But you're welcome to it for now.) If you open the images in notepad, you can see the prompts and models used. Everything I made was in Stable Diffusion.
https://drive.google.com/drive/u/1/folders/1Lw1ngwPdipU39uB7ygAHdrs0SWsRt6vF
Wow! Thank you! :-)
This doesn’t capture the experience of appreciating art. I don’t experience a work of visual art as a disconnected picture on a screen. I experience it knowing who the artist is, who they were influenced by or reacted to, when they were alive, and if there is a person or god in the piece, very likely knowing who that person is and why they’re there. I experience art in the context of all the connections I can make to history, art history, and religion. If I were forced to just look at disembodied images with no information about their context and no knowledge of art history, I would enjoy it much less. AI art is less interesting because it by definition has no artist and no context other than “made by an ai in 2024.”
Scott is making an error if he thinks art appreciation is just about “is this pleasant to look at.”
By the same token, knowing about the backstory of AI should increase the subjective value of AI art. In fact, that AI backstory is, in my opinion, more interesting than the life of most artists.
The same critique of "lacking context" can be applied to almost all art from lost or badly recorded human cultures. Magdalenian or Olmec art lacks close to 100% of context and I wouldn't dismiss it as "made by some tribe in 20000BCE".
I can see how a person might be interested in the evolution of ai art generators, but that feels very different from the experience of appreciating art in the context of art history.
I easily scored 100%. How are you choosing participants? This kind of smells like you wanted results to be skewed in a certain way.
The very first sentence of this post has a link to the previous post, which was visible to non-subscribers and allowed absolutely anyone to participate for a one-week period.
For anybody who enjoyed Ancient Gate, there's a Polish artist named Zdzisław Beksiński who does a lot of surreal paintings in a style that's similar (though a bit darker subject-wise):
https://ih1.redbubble.net/image.2438584395.2875/flat,750x,075,f-pad,750x1000,f8f8f8.webp
https://api.culture.pl/sites/default/files/styles/1920_auto/public/images/culture.pl/s-2252.jpg?itok=1lpInVSM
https://miro.medium.com/v2/1*9xqQf7d5dxKVnNtyonQ0_w.jpeg
Knowing that this guy was kicking around in the art world made me squint a lot harder at some of the images I would have normally dismissed more quickly as AI.
On the AI art/human art pair at the beginning of the post: I think it's obvious that the picture on the right is more interesting and appealing. It's just that the perspective sucks a fair bit (was Gaugin bad at that, or did he just not care?) whereas, in the one on the left, the perspective also sucks, but more subtly.
Gaugin was the daddy of a movement (that I personally kind of hate) called Primitivism, so there was more focus on the geometric and exaggerated forms drawn from certain types of African and Indigenous art and less focus on literal 1:1 representation. In other words, it was about vibes.
I don't like Gaugin's work (it's like if John Wayne Gacy were an Impressionist) and I don't like this painting. That said, I don't think the perspective does suck? He's painting lots of hills and curved streets with very old houses on them, and those kinds of towns lay out in a jumbled and chaotic way, much like what he painted.
What's weird to me about the AI one on the left is that it lacks stylistic cohesion. From a distance it just looks like Impressionism, but if you look at it for longer than a few seconds it's like Monet in the foreground, Unknown Painter doing the background, and a (very calm) Van Gogh doing the clouds. It's weird.
> I like this picture. There’s nothing wrong with it. But somehow it’s obviously AI. If you asked me why, I’d say “something about the lighting”. But the lighting is good! I bet lots of human artists wish they could do lighting like this. So what’s going on? I don’t know, but I avoided pictures in this style.
I've heard that because of the way many diffusion models randomize noise via black and white pixels, the light ends up being more evenly spread out than a typical human would make it, and averages out to grey.
I find it interesting that "The Apotheosis Of Homer”, the painting I was (by far, actually) most confident was by a human, was the one most commonly misidentified. Looking at comments there were other reasons, but I suppose Scott is correct in assuming that the text really threw people off.
Instead, I focused on identifiable characters with a range of period dresses, and meaningful composition given the guessable identities of the characters (more contemporary authors like definitely-Dante and probably-Shakespeare are at the bottom, while the main character I didn't recognize as Homer was nevertheless surrounded by other characters that appeared to belong to the classical period). I knew AI was terrible at that sort of period detail (if you specifically tried to fake that kind of painting with prompt-engineering and in-painting and what not, perhaps you can get someone recognizable to appear, but getting all the characters seemingly appropriate seems just too much even for a deliberate curve ball: there's this meme of fashion history according to AI for instance, and 19th century seems to last long into the medieval past), it's bad at meaningful composition (even if you did get it to paint Homer and Virgil and Dante and Shakespeare and Alexander/Alcibiades character and everyone, I would expect it to look like the cat picture that's "just" a throne room full of cats, not this), and it's bad at specific small details like, indeed, text, which for me seemed right.
I think the reason I got The Apotheosis of Homer wrong was that it's composed to draw attention to its center. I didn't realize until I took the test that I have this bias (I knew that Punk Robot was AI but had to figure this out to explain why). But there are composition conventions that most human artists follow that say the focus of a piece of visual art should never be the dead center of the portrait. If the aim is realism, putting the focus on the center makes a picture feel staged and "fake."
Now knowing the context, this is a deliberate choice in this piece. But on casual viewing (which is what the test asked us to do), I just saw a weirdly framed dude and weirdly flat colors without a lot of shading and went "oh those are AI tells."
I will say I'm not sure how anyone could have gotten this wrong. The painting contains a bunch of human figures in meaningful physical relationships to each other - a thing that AI obviously cannot do.
An Impressionist style lets you fudge a crowd scene by representing people as blurs - as we see in Paris Street. (And, in fact, the original Impressionists were savaged by critics precisely because they used abstraction instead of painting crowds as collections of individual people.) So AI can tackle that. But how is it going to draw a bunch of historical figures with distinct features standing next to each other in a neoclassical style, without blurring them into creepy deformed monsters? Show me one example of AI pulling that off successfully.
> Humans keep insisting that AI art is hideous slop. But also, when you peel off the labels, many of them can’t tell AI art from some of the greatest artists in history.
But that's not true, though. As you yourself said above, most AI art is in fact generic hideous slop; it takes a lot of careful human curation to select the best AI images that are good enough to fool (and delight) people. Even then, professional artists can sometimes find obvious flaws in these images; but with careful prompt engineering and stylistic choices humans can minimize the flaws if not eliminate them.
So, AI-generated art is still slop. Human-generated art using AI as the main tool can be great. Film at 11.
That was my thought as well. One thing I hate about AI art is exactly the tiny errors (and not so tiny) that give away that no part of the picture was more or less important. The selection process seems to have filtered a lot of that out.
Now, that said, it occurs to me that there is a LOT of bad human art too. So maybe it is fair to take out the worst AI slop like we don’t include random human crap from Deviant Art’s bottom of the barrel. Granted, bad human art is different from bad AI art in many ways.
I dunno, in general I don’t think that this showed that people like AI as much as human art, rather that well used AI can help a human make good art as you say. Even before AI some art was vastly better than others, and it wouldn’t make sense to say that people didn’t like type A or B, but people like specific artists. People might like or dislike various styles on average, but a great artist can make you love a style.
Seems like about 40% of AI art I see includes a babe with pointy breasts encased in some sort of firm and shiny external bra kind of thing that makes them really stand up, sort of like the twin towers in the old days, you know?. And a futuristic feel to scene, natch. Babes aren't usually doing much -- mostly just standing around, display racks for titties.
You are making quite a firm argument for the value of AI art :-)
Well, I can at least say that the "Yithian at the optometrist's" that I asked DALL-E for this evening, while it had problems, didn't default to _that_.
Turing doesn’t say that 30% rate of fooling humans in a five minute conversation means it’s thinking! He says that “thinking” is just everything that goes into having a meaningful interaction, but he leaves it open-ended what kinds of interactions you want to have. He predicts that by 2000, computer conversationalists will be good enough to fool people 30% of the time in a five minute conversation, but he doesn’t give that as a criterion of any sort.
Anyone who hasn’t read the original recently should - it’s got a lot of interesting surprises in it, including anticipation of machine learning, and surprising credulity of ESP.
https://academic.oup.com/mind/article/LIX/236/433/986238
Any criticism of AI art based in "it looks bad'" was doomed from 2017. But any defence based in "it looks good" is missing the point of why the Gaugin is more valuable than its AI counterpart.
Gaugin's art was an expression of a movement, among people, exploring ideas through dialogue, just like this blog. I don't value it because it looks cool, but because it's a checkpoint of that dialogue. It's only interesting inasmuch as it relates to its intellectual context. It's interesting because if you ask the question "why did he paint it that way?" you find a complex rewarding answer interweaving human stories.
AI art obviously has its own reward of fascination but it's totally unrelated. The parameter space may as well be infinite, that any given image is found within it can never be surprising or interesting.
This comment feels really facile and obvious but on the other hand I'm just not that sure everyone can see this?
I don't really get the point of this whole exercise. It feels like a Rorschach test with each commenter submitting their own interpretation of what we're even supposed to get out of this, let alone what specifically we're getting.
“Can never feel surprising or interesting” is very strong. Gaugin could probably have produced his artworks using AI, if it had existed in his time and he’d spent the same effort and creativity on it that he spent on his art, plus he spent as long on every detail as he did in real life. I agree this doesn’t test whether AI could replace the human “entirely”.
In that case, there wouldn't have been Gaugin's pictures to train "ai" on in the first place. It's fundamentally derivative and second-hand, that's the main and true reason it sucks, fundamentally.
Yes, but I think Gaugin could have found a way to make his pictures using more complex prompts to produce them based on already-existing images. As I said, it would’ve needed his creativity, but it could’ve made the process faster for him.
That's simply not how it works. "AI" cannot create impressionism or any new categories and concepts whatsoever, it can only recycle what's already been made.
Certainly they could avoid the copyright conversation if they could just derive modern-ish looking art from free domain artwork from late 19th century and before. Or you know, "derive" art from merely free-domain photographs of real nature, sorta like the first cave artists did. If only they had enough "creativity" with bending words, yeah right.
I admit, I don’t know much about Gaugin’s method specifically. But things like “draw an outline of a woman smiling as she walks home”, “her smile is more peaceful than that”, “colour this region like an oak table, in natural skin colour”, “her skin has a subsurface glow. Blur each colour downwards” (worded in some way to evoke image processing software and the way people talk about lighting models).
Yeah, I had the same thought. Gauguin is part of a social context, and the AI art is just…an image floating in a void of contextlessness.
I wonder if there are people whose experience of art is totally devoid of context, where it’s just “does the picture look pretty or not,” and in that case they very well might like ai art as much as real art.
>Yeah, I had the same thought. Gauguin is part of a social context, and the AI art is just…an image floating in a void of contextlessness.
Hmm, don't forget that LLMs are trained on a vast amount of text, which carries a lot of social context with it. When I asked DALL-E "Image of a kitten pouncing on their owner's toes in bed", _IT_ came up with:
>Here are two delightful images of a playful kitten pouncing on its owner's toes in a cozy bedroom setting. 😊
It isn't a novel context, but it _is_ a context, and it is, in fact, the _right_ context.
I sympathise with your point, but I don't 100% agree with it. The AI pieces are also created by people, they're also expressions of an emerging new artistic tendency made possible by advances in technology, and they're also checkpoints in a dialogue - one we're having right now. I mean this very conversation may well be archived by art historians of the future (hi guys) as an example of the controversies over the first wave of computer-generated art.
I'm thinking a lot as I read these comments about the art critic Louis Leroy's famous review of the first ever Impressionist exhibition. He was completely appalled by and contemptuous of it - he thought it was fucking stupid and they should have just painted things properly. He got particularly hung up on the way Pissarro abstracted people's shapes in crowd scenes - "those innumerable black tongue-lickings in the lower part of the picture". Honestly reading it was a big step forward for me in finally understanding why Impressionism was such a big deal.
To me this is pretty similar to the current "AI slop" conversation. I think this test (including Jack's Pissarro imitation) is also a checkpoint in a dialogue. Of course part of that dialogue is "this is fucking stupid, why would you ever do this, ban this sick filth, we must RETVRN".
The "people" in this are curators at best, they didn't create a thing. A museum curator with the finest taste is no artist. "ai" in art is fundamentally derivative and second-hand, in can generate a picture of a "queer firetruck", but it didn't come up with either firetrucks or queerness which came into being as a matter of organic need. Neither does it employ the fundamentals of drawing, mark making and human perception that artists do in making pictures. It simply has no idea what any of this even is, it just makes derivatives, chewed up bubblegum. How well it fools the observer is irrelevant.
It's you putting layers upon layers of lipstick and facepaint on a pig and giggling like a bastard when someone mistakes it for a person. "Ha ha, I fooled you, now pig is truly just like a man!", no I just come to hate you and your pigs, and all the excrement you carelessly pollute the world with even more.
I loved the concept of String Doll specifically and really wanted it to be human as a result. I guess the concept itself was human and in retrospect it’s easy enough to see limitations in execution (the split finger, very sparse/random decorations to the circle, generic expression, weak looking/oddly set shoulders.)
The “Bad anatomy Christ” painting seemed clearly human in the anatomy flaws. The pallor was also distinctive. The AI wounded Christ also has an issue with the hands.
To be fair, the biggest impact this post had on me was remind me of how cool art is, AI or human, this stuff is just great.
Mediterranean Town by Jack and Victorian Megaship by Mitchell Stuart are my favourites.
I was super impressed by Mediterranean Town and it's the only piece that's made me reconsider my strong anti-AI art stance. It had AI "tells" but they worked in context, and it wasn't only "nice to look at" but evoked real feelings and experiences. I'd love to know how exactly it was generated.
Mediterranean Town is easily the best of the AI-generated pieces and is for sure the only one in here that I thought was actually good, not just "good for AI". Not 100% sure why though. Maybe because all the asymmetries and mistakes feel more intentional?
You definitely didn't catch all of the obviously-wrong hands; the ones on String Doll are particularly bad.
To be fair, I was CERTAIN that Girl in White/Portrait of Charlotte du Val d'Ognes had to be AI, because the hand holding the pencil seems to be sprouting out of the girl's waist with no visible arm. But it's just a completely human error by a relatively inexperienced painter.
My parents have a painting of a stagecoach driven by two men and pulled by four horses, which have a total of about 10 visible legs, IIRC. It was made well before AI could ever do such a thing.
I never was that fond of the painting, and the lack of legs was only one reason why. Others included an attempt (that failed, to me) to indicate speed by smearing out some lines a little, lack of facial expressions on the drivers, and an overall vibe more like a still life than an action scene.
My point is really that AI can clearly generate some images that we think humans may have made, but humans can generate bad art with some of the flaws of AI, too. I'm surprised that people liked an AI image best, but I sure wouldn't have picked Riverside Cafe as a great piece of art, whether human or AI generated.
The major difference is there are plenty of artists who aren't making these obvious mistakes. Meanwhile the majority of AI art is riddled with them. Aside from the impressionist art, I don't think there's a single piece of AI art included here that doesn't have some glaringly obvious fault with it.
"Anime Girl In Black" is stylistically inconsistent in a way that I wouldn't think a human artist would create. Some parts of the image, like the dress, the armpits for some reason, and the background are rendered more realistically, while the hair, the eyes, and the arms are rendered in a more cell-shaded, cartoony way.
If nobody's pointed it out to you yet the reason the string girl was most confidently AI is the hands. There's a hint of a sixth finger on the right hand and the left hand is heavily distorted.
I'm decent at identifying the "house style" and I know a couple of tells that are common, but when a human is picking images specifically to avoid that style, I'm stumped.
I'm actually surprised that "String Doll" was AI. Yeah it has the "house style" vibe, but AI tends to do really badly at long thin strands, like ropes or hair or power lines.
Some thoughts after getting the Gauguin wrong.
1. Most impressionist art is not composed that way. If you look through the wikipedia pages for impressionism and post impressionism, you'll find that those paintings tend to have a lot more space. Look at how small the sky is on that painting! The earth feels heavy and makes the overall painting a bit cramped. And also the brushwork also just feels a little off. I guess this probably should've been a tell that it's human, since AIs would not produce something that's out of distribution like this by default.
2. Gauguin is much more known for his work in colonial France, and I'm much less familiar with his early style.
(I don't have any super formal art history training but I think this is right)
Art requires intent and is a process. AI has no intention. Non-sentient mechanisms cannot have intent or introspection, and so can only produce imitations of what humans do. Crappy art is still art, and beautiful AI is nothing but nice patterns. Obviously AI art is going to fool people, but that’s not the litmus test for art, whether it’s poetry, pictures, or music. Art is in the intent. If AI can produce art, then a tree is art. But a tree isn’t art, it’s a natural phenomenon.
A lot of the nerd aesthetic is specifically the "aesthetic of intricacy" (I owe this point to a rationalist blog, though can't remember which!) - the example they gave was Don Rosa's Scrooge McDuck artwork, which indeed I adore, but you could also add in the way computer interfaces look in movies - the presence of a lot of detailed, *interesting* things close together acts as a visual shorthand for "nerd". I wonder if this is why so many of us actually prefer these pictures!
Hot take: art critique is infected with a bunch of weird artsy postmodernists who have divorced themselves from normal beauty standards, therefore a lot of famous art is weird artsy stuff that doesn't actually look good, therefore AI being able to imitate it is not actually all that impressive. Most of the time I found a weird imperfection during this test, it was hard to tell if it was a mistake the AI made, or a weird artistic style choice by a human author trying to be unique and special.
"Sorry, I guess Giovanni Girolamo Savoldo just wasn’t very good at anatomy."
My professional artist boyfriend says: 'That's not why! Look at all the other humans! He's good at anatomy! The weird Jesus anatomy is on purpose; it's a reference to old masters, who he was paying homage to. Think of it like a meme. Just because someone uses trollface on one person on the picture doesn't mean they're bad at drawing faces.'
It does feel a little like your methodology has the flavor of "We had a million monkeys bang on typewriters for millions of years, then threw out all the ones that weren't passages from Shakespeare. We then checked if humans could tell the few that happened to be Shakespeare apart from passages Shakespeare himself wrote that we selected." Seems like a stretch to say a victory for the AI means AI *is* good at art, as opposed to *can* produce good art with lots of tries and humans sorting through the haystack.
They also choose the best of the human art.
If you want to get all the AI images in the pool you also have to include children's scribbles.
That would be a much fairer test! One where you’re not producing jillions of candidates then carefully selecting the ones most likely to win.
Can someone link examples of the DALL-E house style that everybody but me seems to be familiar with? :)
I can't, but I'm glad it wasn't just me.
I’m studied art history in college and I’m surprised how well-known most of the ‘Human’ artists were. As Jack’s Dad, I’m especially mad that I only got 52% right myself!
I wonder what percent of users did this on a phone versus a larger device.
I was most fooled by the impressionists and the anime. I wasn't able to judge the detail work for the impressionist pieces, which is often the most glaring way to identify the AI art in other genres (look at the windshield girders on that spaceship towards the end, for example). But being fooled by the anime pieces surprised me, I guess there is enough data in the training and mostly clean lines and low-detail rendering that makes it easier for the AI to get close.
"Riverside Cafe" is so similar in style and composition to Van Gogh's "Cafe Terrace at Night" that I can pretty much guarantee that many of those who guessed human (myself included) assumed the AI knockoff was the real thing.
The Riverside Cafe's awning, composition, and style are extremely reminiscent of van Gogh's Café Terrace at Night, https://en.wikipedia.org/wiki/Caf%C3%A9_Terrace_at_Night
I don't really know what's going on with Blue Hair Anime Girl - the actual source (zerochan isn't a source, it's an imageboard) is pixiv https://www.pixiv.net/en/artworks/81413572, which is... not actually the same picture? I can't find the source of the picture that was actually used in the test.
Scott mentioned having cropped some of the images to reduce signal from image format, is that what you are thinking about?
No, compare the picture from pixiv ( https://www.pixiv.net/en/artworks/81413572 ) presumably from the original artist, no later than 2020, and the picture from the test ( https://static.zerochan.net/Hatsune.Miku.full.3532501.jpg ), from zerochan, as provided by Scott.
It's not just a crop - these are very different versions of the same drawing!
I can only think of a few explanations:
1) The artist re-drew an old image they weren't happy with and replaced the old image with the new on their pixiv (not uncommon - this is what I think is most likely to have happened)
2) Some other person traced the original drawing to make their own and that other drawing got mistakenly attributed to the original artist (also pretty common)
3) The original drawing was put through some early 2020s AI upscaling neural net, resulting in what is essentially an AI version of the original (and to me, it really looks like it has those telltale AI signs, compared to the pixiv picture).
Other than a factor of 10 difference in file size, and a slightly different JPEG palette (which is often messed with by image sites), the images look quite similar on my device when not zoomed in. The larger image might have been made by throwing the earlier, smaller image into a diffusion upsampler, maybe?
No, it is drawn differently. https://i.imgur.com/ZCVrlfc.mp4
1 is correct, danbooru has both revisions, from 2020 and 2024: https://danbooru.donmai.us/posts/5861211 (careful, nsfw website)
You can tell it's a manual edit because of the mixed resolution of the new brushstrokes on the old image, and the original is too old to be AI.
2020 and 2022. On Pixiv you can find an (i) info icon near the date, and it will tell the image was updated in 2022.
Still, I think putting such controversial image on the test is bad practice to say the least.
Ya'll garbage at appreciating art, as opposed to me who is good at appreciating art!
For real though:
I got (almost) every AI piece, but I missed in the wrong direction on some particularly AI looking human pieces (Big Boat, etc).
I'm surprised at how much better I did than the majority, Eg. the Favored impressionist AI piece seemed fairly obvious to me; beyond the fucked up wheeled conveyances the perspective is kinda messed up and also the whole thing is boring.
That said, the other AI impressionist pieces were pretty good, and I think I only did as well on the impressionist ones as I did because I had literally just fried my brain in cold pressed art at the Orsay in the last couple weeks.
What I find strange is that there were 11,000 respondents and an open answer key, yet no one cheated to get 100%.
Not suspicious about the 5 that got all but one right?
Some people just have to be right ALL of the time and will lie and cheat, even when there's nothing to gain and they're not fooling themselves.
I'm a bit suspicious of the 49rs too.
There really needs to be an online version of this test that instantly shows your score. In fact I expect someone has probably already done this, so please let me know where it is. If it hasn't been done, I wouldn't mind doing it.
(For maximum engagement, I think it should immediately say "right" or "wrong" for each image, rather than showing the results after the user has finished, but I am open to arguments the other way.)
You could train yourself to distinguish AI art like chicken sexing!
I like that the highest-rated human artwork was some random DeviantArt post with 70K views. (It was my favorite of the entire batch.)
Personally I was doing a lot of second-guessing my immediate reactions on the test. Village of Osny in particular my first thought was "this seems real, I vaguely recognise this" and my second thought was "maybe that's what he wants me to think". Pretty Lake I had the opposite response - I thought it was AI because of the artefacts in the foreground, but then I asked myself if it was a real picture that had been specifically chosen because it contained brushstrokes which looked like artefacts.
Mediterranean Town is I think the most successful AI piece here - the only one that I I think stands up to comparison with work by traditional painters. I think that's because of the lack of detail - it's a simple composition and some of the obvious flaws can be passed off as deliberate artistic choices. The contrast between the more naturalistic ocean and the flat buildings is interesting. I think a "real" painter could have improved it with another couple of passes but it basically lands.
Impressionism was very controversial when it came out for a bunch of reasons, including its lack of detail. Critics were shocked by the fact that you could just "suggest" people in the crowd with vague blobs and smears - they were used to the more realist style of painting taught in the French academies where every person in the background had to be rendered in detail. It's worth reading Louis Leroy's review of one of the first Impressionist exhibitions - he was completely appalled by it.
I think I did clock that Paris Scene was AI, but only because I was second-guessing it. Also I'd spent a bit of time looking at Pissarro's paintings of Boulevard Montmartre and I knew it didn't have that smoothed-over look - it's more bleak and startling, the colours are harsher and the composition is more unbalanced. Of course, Impressionism means you don't have to draw individual faces in the crowd - making life a lot easier for the computer. Riverside Cafe I got as well - the positioning of the lamp post is weird.
All the Renaissance / realist paintings were easy. The "digital" pieces frankly all look like shit, no matter who made them, so I sincerely don't care if they're AI or not.
Honestly looking back over the selection, I think the main thing I've learnt from this is that "Jack" is really good at getting results from AI art tools. He clearly has an eye for composition. All the best work in the gallery seems to be by him, which suggests to me that human discernment and talent still plays a strong role in getting the best pictures out of the machines - which is what we should expect of course. Can we find more of his work somewhere? I'd like to know more about his techniques.
Hey, this is Jack. Thanks for the compliments! I explained my methods in another comment, if you have any other questions you can email me at gallerdude@gmail.com
Cheers Jack! Enjoyed your work. I reckon you should start posting it online some place - people will give you shit for being an "AI artist" but I'd be keen to see a few more of them. Of course all AI art does get worse the more you look at it so you might find you lose interest surprisingly quickly - that has certainly been my experience.
Honestly this could be the real Turing test. Any truly great painting will get better and better the longer you spend with it - can we produce a piece of AI art that does the same?
>I’m afraid I don’t know enough math to tease out the luck vs. skill contribution here
Its pretty easy to do. The best performer would be the top 10999/11000=0,99991. You look for that in this table:
https://en.wikipedia.org/wiki/Standard_normal_table#Cumulative_(less_than_Z)
and find it at z=3,74. So the best performer should be 3,74 standard deviations above the mean. With a 0,6 accuracy and 50 questions, the mean is 30, and the standard deviation is sqrt(50*0.6*0.4)=3,46. In sum, we would expect a score of 30+3,74*3,46=43 right answers from the high score purely by luck.
Looking deeper into the top ranks, we find:
score people cumulative expected delta
49 5 5 41,49 7,51
48 3 8 41,03 6,97
47 2 10 40,80 6,20
46 4 14 40,45 5,55
45 7 21 40,02 4,98
44 16 37 39,39 4,61
43 17 54 38,94 4,06
42 48 102 38,16 3,84
As for retesting, a lower bound estimate is 30 plus the delta, but that can go up with assumptions about the distribution of skill.
Small typo: it's Andrea Mantegna, not Mantenga
I also misjudged "giant ship" to be AI, but I misjudged it for exactly the same reasons that the artist gave for for "ancient gate". It contains huge amounts of detail, and many of the details don't make sense, like all of the upside-down spires.
Next major art genre - Post-Turing Conceptualism: art made by humans that a naive audience would think is human made but a sophisticated audience would be convinced has subtle tells of AI gen due to making artistic or materials decisions that a human simply wouldn't make
You mean like overtly blending two different styles in the same painting?
Here are my results: https://coagulopath.com/ai-art-turing-test/
88% correct
Jack's ones were insanely hard.
Finally got around to this. Only got 4 wrong. The trick with ships is that AI is absolutely awful at rigging. It almost always ends up as a completely nonsensical, physics-defying tangle.
It seems like the selection process here basically removes the utility of the test.
"AI art", as actually encountered in the wild, is overwhelmingly a bunch of non-selected slop, created and chosen by unskilled artists, dominated by the house styles of the popular models, and exhibiting characteristic AI malformations like bad anatomy, bad geometry, mangled symbols and text, and so forth. When people talk about "AI art", this is what they're talking about. If you carefully select a corpus of "AI art" to exclude the most annoying things about AI art, then you aren't getting at the territory. (Note that I say this as a fan of AI art, overall.)
I'm slightly bemused that we describe these images as "generated by AI" when every one of the prompts was still conceived by a human (This matters because there is a relationship between the quality of the human prompt and the recognizability of the image). Fifty years ago the phrase "an image created by artificial intelligence" probably would have evoked the idea of a robot with agency producing a work of art with no direct human input. By that definition AI art is nowhere close to passing the Turing Test!
I recently posted an *actual photograph* online and had people insisting it was AI. Only post-processing was that I cropped it to square. Very odd experience.
Even before the AI thing when Photoshop was a bigger concern, I'd seen plenty of presumed real photos that look photoshopped at first glance. Somehow, nature doesn't seem to care what looks realistic.
It was only after spending serious time taking photographs of X that I realized the images I saw (including famous art) had much less variance in X than real life does. Looking briefly into other values of X, I now suspect this holds more universally. As an example, the use of light by Turner (the thing that Turner's oeuvre is noted for) is ridiculously tame compared with an average golden hour, which throws up hallucinatory combinations of hues that look completely "fake".
The 1:1 aspect ratio was probably why! It's a common choice for an AI generator.
So I’m going to do the really shallow obvious thing and go through my answers, because I can.
The overall theme is “I was wrong a lot, but usually not very surprised when I was wrong.”
If I have no comment, I’ll just put a ✓ if I was right and an X if I was wrong.
1. X I was very much on the fence on this one, going back and forth. Knowing it was Warhammer explains my confusion.
2. ✓
3. ✓ I’ve seen so much art in this exact style that this was an easy call.
4. X I fooled myself on this one, in that my first impression, looking at the face, is that it was AI, but then second-guessed myself because I couldn’t justify it.
5. X I didn’t have a good reason for thinking this was AI, other than the theme and something vague about the art style, which are not good reasons.
6. X Meh. I have no way to judge this sort of art, which I dislike anyway.
7. X I really should have known better, looking at the face (too smooth? Not sure), but I got tricked by details like the cracked oil paint.
8. ✓
9. X A random guess that was wrong. There wasn’t anything that stood out as AI-like, but again, I don’t like the style, so 🤷♂️
10. ✓ This one instantly jumped out at me as AI, and was the one I was most certain was AI. Not for any of the reasons Scott’s artist friend gave, just a sense of an artificial hyper-realism.
Now that I think about it, I think that’s what seems “AI-like” in many cases. AI often combines photorealistic with non-photorealistic styles in a way no human artist never does in the same work.
11. X Definitely fooled me, but I feel like impressionism is a great way to hide the differences between AI and human art. That or I’m just not familiar enough with impressionist art.
12. ✓ Just…felt real? I dunno.
13. ✓ Why is this one obviously AI while #3 is obviously human? Again, I think it’s the “too-realistic while not actually being photorealistic” element. Like, it looks like a hybrid of art styles in a way no human is likely to want to do.
14. ✓ I’ve seen too much computer-generated art that looks exactly like this not to assume that this was human.
15. X I just randomly guessed on some of these, assuming that about half of them were probably AI. A poor strategy that backfired several times. There was something vaguely sort-of AI-ish about how the different people seemed to be in different styles, but this is definitely something I’ve seen human artists do in this sort of painting.
16. ✓ I hesitated on this one, because it was so obviously in a thematic style you often see in AI that I figured it was probably real. But I went with my gut, and was right. Beautiful, though.
17. X I really dislike this style of art, and have no ability to judge what’s “intentional” vs. not, since it all always looks intentionless to me.
18. ✓ A tricky one! I hesitated a little, and sort of assume that a lot of people got it wrong, but I saw several things in this style long before AI was a thing, and after examining it closely (no, not by zooming in), I saw nothing that indicated it was AI.
19. X This is the same sort of style as #17, and looked more random than that one, so I thought it might be AI.
20. X This one really got me. Even now, looking at it, I can see that, yeah, *maybe* there are some AI-ish things about it, but if you told me it was real, I’d believe you.
21. ✓ Again, photorealistic without quite being fully photorealistic. I could have been wrong, though, as obviously human artists are capable of photorealistic art (which I would never have believed if I hadn’t seen it).
22. X I guessed, and guessed wrong. I still think that it feels vaguely AI-ish, but maybe I didn’t look at it closely enough.
23. X Looking at this now, yeah, this looks both too real and not real enough, but I have seen human paintings that sort of had the same vibe, so…
24. ✓ I guessed again, but correctly this time. No particular reason other than it seemed less intentional in the details than I might expect.
25. ✓ I guessed wrong on this one, saw #27, figured that one of the two had to be human (because they were too similar for Scott not to have done that), and changed this answer.
26. ✓ I hesitated on this one—something weird about that dude’s eyes, but human artists often do weird crap—and then I noticed his left hand.
27. ✓ This one just screams AI to me, almost as much as #10.
28. ✓ I’m really not sure how one is to judge art in this style as being human or AI. But again, that could be due to my unfamiliarity (because of distaste) of this style.
29. ✓ Just looks like normal 1600s art to me.
30. ✓ Her face looks a little weird, so I wasn’t sure.
31. ✓ What clued me in was that some of the tables are warped.
32. X Just completely wrong for no particular reason other than random guessing (and maybe the reflections?). I should trust myself more.
33. X Obviously this “looks like AI,” but I feel like I’ve seen plenty of things that looked pretty much like this. If anyone can point to why this is clearly AI, I’d like to hear it. Other than—looking at it now—a *really* vague feeling of too much artificiality.
34. ✓ This one just felt wrong, especially the candle. Looking at it now, it’s obvious that wax never does that, but it was just a gut feeling when I rated it.
35. ✓ The wounds look just too correct and intentional.
36. ✓ Is this even art, or a 6-year-old’s scribbles?
37. X Even looking at it now, I don’t see anything especially “AI” about it that I can put my finger on.
38. ✓ I got it right. Not sure why. Guessing? Gut feeling?
39. X Everything seemed too smudged and mixed in, though of course humans can do that too.
40. ✓ Obviously in “AI style,” though I wasn’t certain until I saw her fingers.
41. ✓ Nothing stood out as in any way AI-like.
42. ✓
43. ✓ Inhumanly “smooth.”
44. X Got this one totally wrong, but looking at it now, I see that there’s a kind of smoothness to it if you sort of squint that’s uncharacteristic of human art.
45. X This one totally fooled me, no notes.
46. ✓ It looks like a fuzzy reproduction of the original.
47. ✓ It looked like a literal collage, and it was.
48. ✓ Hesitated on this one, because the buildings look all weird, but punted and guessed ‘human.’
49. ✓ Something about this just yelled “AI” to me.
50. X Still don’t see anything particular to tell me it was AI.
Okay, what was my score? Let’s count: 30/50 60% right at the median. Darn.
Well, I at least learned something from this: A little more about what makes AI art look like AI art. Thank you for indulging me in this childish enterprise.
Idk if it's been mentioned, but I think it's important to understand the massive audience selection bias here. Respondents to this are from this community, the top 1% of online AI enthusiasts and best accustomed to noting the differences. The average boomer sharing posts on Facebook has no chance and I'd bet is right at 50%.
I think the selection effect goes the other way. These images were carefully crafted and selected to be as difficult to discriminate as possible. Real life AI slop is immediately obvious because nobody cares about hiding it.
I think this is irrelevant based on the amount of comments to the tune of "the important test here was the 'real art' and not abstract nonsense and anime girls so I couldn't care less I got a mid score."
Jack and Ryan are really talented! Oh and also all the other artists 😁
For me, the reason I care about art is that it's a way to get inside another person's head and see the world from their perspective, at least a little bit. To me, AI art is interesting when it is able to show me a distinctly non-human perspective. But the actual pixels on the screen aren't especially important, what matters is the link between minds.
Your friend's explanation of why she thought the picture was AI slop was really interesting! It caught my eye because there is a strong correlation between something that *seems* intentionally generated, and something that *was* intentionally generated. That correlation is based on how much less randomly the thing appears to have been created. (See the poster “Inferring Intentional Agents From Violation of Randomness” by Meng, Griffiths, and Xu (2017); https://escholarship.org/uc/item/4pb9s57j.)
With sequences, “randomness” is somewhat less complicated to measure than in art pictures (but still more complicated than you’d think!) I wonder if we'd see a difference in "randomness" between AI-generated pictures and human-generated pictures? Maybe the ones that were most confusing have the most amount of overlap?
(Free research idea! If anyone wants to quantify the “randomness” in AI generated pictures vs. Human generated pictures, please let me know how it goes! I’d love hear the results. The research question you could pitch to potential advisors/supervisors/funding agencies can be phrased along the lines of: “Is AI-generated art more random than human-generated art?” You could also substitute in the word “complex” or some specific measure of entropy depending on which lab you’re pitching this to.)
My ability to distinguish went up a lot after being shown a few examples and an explanation. The human images tend to have more consistency in style and overall cohesiveness across the picture as a whole. A lot of the simplistic styles look kind of dodgy at best so it's very hard to guess which they are. Many of the classical paintings have the same kind of compositional errors as AI images do which makes them hard to guess as well. Taking out a bunch of images which are obviously AI skews the results as well, although many of the remaining ones still screamed AI to me. The impressionistic style is the one which is both good looking and which the AI does a passable job of.
I started the test and quickly gave up. For whatever reason, taking that test made me realize something that I had been battling with for a very long time. I realized that if I’m looking at an image on a computer screen it has zero value to me regardless of the provenance of it. There are trillions of images now and they all bleed together. This includes photography. The only photography that means anything to me now is pictures of my friends and family.
And that’s kind of a big deal for me. I got a BFA in film, photography, and visual arts back in the mid 90s. Large format photography both landscape and portrait was an almost spiritual activity. Both taking the pictures and doing all of the steps in the darkroom was incredibly fulfilling. Once digital photography got good enough I started taking pictures that way. Before too long I stopped taking pictures altogether. Digital photography is incredibly convenient but incapable of having any actual impact. There are just too many images, too many “amazing” ones. Whether that means incredible wildlife shots, glorious landscapes, or intense portraits, it doesn’t matter. It all blends together as just another picture on a screen. Digital art is the same to me.
I do think that physical images can have impact regardless of how they are made. Obviously paintings and other hand rendered media stand out as special objects but even photographs change when they are in the real world. That is especially true the bigger it is printed. I wonder if large format prints would make it easier to distinguish between AI and human made digital pieces.
No furry art? Should run this again, but draw your participants from furry sites and furry twitter. I can help source the images (both sides). Could be restricted to SFW only.
As possibly the most "art centered" modern subculture, with a very heavy focus on custom-commissioned pieces by a huge number of working artists, I'm curious if the results match the broader world.
Most of the anti AI """Art""" criticize the substance and process not necessarily the produced "art" themselves.
The one that are against AI "artist" also against so not because they see them as a bunch of talentless hacks, but because the way they use their talents are so detached, bridged in such excess gaps between their non artistic talents (prompting, compsci, etc) and the actual artistic action (something like knowing the fundamentals, if we are talking the technicalities, or the message being conveyed that actually in the direct connection of what the artist meant).
It is akin like someone who's really good in manipulating someone through mind games being able to convince artist like DaVinci to do the work for him
Is he talentless? of course not, he's probably really charismatic and high in social skills
Is his talent actually an artistic one so that the very produce of the artistic action can be attributed to him much like we would attributed murder, patent, accomplishment to the person doing such? no, the person uses middleman bridging the gap of what he actually does with what actually being produced.
No Sam Altman is not an artist, he is not an artist much like Elon Musk is not a professional engineer.
Would you call a CEO of a record label or a talent agency a professional singer? no, right?
Some good points pointed at in this reply and reminded me of the clearly demonstrate ingrained evil trained into the AI as simple questions for basic well established facts the mind-raping director and funders of the trainers want suppressed and removed from our minds and thoughts, the clear lies, and implied punishments of similar questions (and perhaps real actions of reports to FBI of our locations and out 'bad-thoughts' that lay the groundwork for future SWATing or unJust punishments.
The underlining evils imbedded throughout the tool - that both warps that that uses and depends on it And the results of what is created through it use, Cries to Heaven of it's hellish creations and results.
Something less destructive but society and education-seeking sickening that the mind-raping and falseness returning results from Google and Wikipedia (and other once powerful tools like YouTube) of truth were warped from, AI is a more insidiously limitless-scope and actively adjusting source of False Deadly soul-murdering directed tool.
Thank you for reminding me of something I have directed tested and know, but rarely get a chance to warn about.
God Bless., Steve
Given the result, AI art will turn artistic scene like how a high trust society being turned into a low trust society would affect people's attitude to public transportation
I thought that Ancient Gateway was obvious AI, but that's because of the "excessive repetitive detail" heuristic that also caused me to falsely identify Victorian Megaship as AI.
On the bright side, it looks like the "humans with plastic skin" heuristic (String Doll, Ice Princess, Dragon Lady) was 100% in this challenge. When people say that AI art is easy to recognize, they're probably talking about the fact that 99% of what you see in the wild looks like String Doll, or at least that's my experience. This must be what you were talking about with "Dall-E House Style".
Also, asking about abstract modern art is just absurdly unfair to anyone who isn't an expert in the field. It's like asking people to do a Turing Test talking to humans and AI, except the communication is done via morse code only and the person doing the test doesn't know morse code.
Those may sound irrational. I just don't believe A.I should be used for art.
Wow, the majority of people have awful taste and miserable AI identification skills
I enjoyed taking the test, and was surprised and humbled by several of my misfires, and briefly felt smug about my overall success rate. But I'm the child of a professional artist who specializes in oil portraiture, and I grew up in a home where I'd eat breakfast while absently flipping through books on Fauvism (to name a random example) that had been left open on the table. I'm more familiar than most with the odd reality that a lot of "great" art is just ugly as sin, and what makes a particular artist's work "great art" in retrospect has everything to with the context in which it was created - including the sociopolitical landscape of the time and place, and the preferred mainstream styles that their work challenged or changed in some way. Also, no one bats a thousand. That Monet isn't even close to one of his best. It's a pretty boring composition, notably only because it was done by a guy who did far better and more visually interesting pieces.
In that sense, I think you've essentially determined something far more shallow than you aimed to, along the lines of discovering that the average person enjoys listening to "Fifth of Beethoven" more than Beethoven's actual fifth symphony. To casual observers in 2024, a lot of the western canon is pretty unremarkable, and blurs together vaguely in the periphery.
I'd be interested to see the results of this if administered to students completing a course in art history, or something along those lines.
These are supposed to be paintings, right? Seems like the only true test would be to see the painting in person.
Depends on your goal. In this case highly curated AI generated images is compared to images of actual artwork. Which says something about AI capability of generating images that look like artwork to human viewers. That is different from comparing actual artwork, e.g. oil paintings, but still says something about current AI capability.
Ok, but this is NOT a Turing test!
For that, I get to choose a prompt (as bitchily as I can make it) and the AI and a human artist would make a picture and send it to me, and I decide who's who, and if I can't, the AI passes.
Sure, the real thing is a lot more difficult to set up. Doesn't make it ok to call this a Turing test.
I totally agree. And I think your setup works a lot better if the goal is to see whether AI art is good, instead of (as Scott’s set up shows) whether some AI art *can* be good if you have humans carefully create it and then sift through all the muck to select the very best ones. https://www.astralcodexten.com/p/how-did-you-do-on-the-ai-art-turing/comment/78094918?r=8o1k&utm_medium=web
This is testing to which degree highly curated AI art can pass as human - that is important too - just a few years ago you would not be able to fool anyone with computer generated art no matter how selective.
However, I agree it's not a turing test. I think good turing test would require AI to make art independently in some category and then compare to human artwork in the same category. The human artwork should be unknown to the viewer.
Not even that, I don't think. Turing's point was to test with adversarial prompts. Some category name isn't adversarial. In the Turing test, the tester is an a$$hole and gets to pick whatever confusing input he wants.
I'm not sure if I understand what you mean exactly. Someone linked to the paper by Turing further down in the comments, so one can read what he meant here: https://academic.oup.com/mind/article/LIX/236/433/986238
Of course, Turing is talking about a conversation-test. The machine would be fine tuned to mimic a human. Both the human and the machine would attempt to convince the interrogator that they are the human. The interrogator could ask any question he wants, and the test would be passed if the machine could convince the interrogator in more than 30% of 5-minute conversations. I think that a fine tuned LLM should probably be able to pass that test as stated now, though I have not seen any definitive proof of this.
For art the test would have to be modified somewhat. You could give a painter and an AI the same prompt for a piece of art, and the interrogator could be allowed to give the prompt - but this would clearly take a long time and you would be restricted to living artists. So I think a reasonable test here would be to use existing art. You would fine tune the AI to try to make convincing art. Then, you would use a description for some existing art work as a prompt to the AI. The human artwork should be unknown to the interrogator, obviously. This could feasibly be done both for images of art (as in this test) and for physical art work - such as oil paintings, if the AI were connected to a robotic arm and given a paint brush. I don't think current AI would pass even the 30% bar on this test, currently.
Adversarial inputs are key to the actual Turing test. The AI needs to respond human-like to gotchas and trick questions.
Restrictions on the input means you're no longer doing a Turing test. What you're describing is restricting the inputs to descriptions of actual works of art. So, not a T. test.
As you note, the real thing is hard to do with pictures. Bummer, I guess.
I just took the test and got 46/50 correct, or 92%. The one I answered I was most confident was AI was Minaret Boat, although there were many that I could clearly tell were AI. The one I answered I was most confident was human was Wounded Christ. The ones I was least confident about were the impressionist ones, but I didn't get tricked into thinking they were all human because I know how good AI is at impressionist art.
The ones I got wrong were:
Cherub (answered human, was AI)
Fancy Car (answered AI, was human)
Paris Scene (answered human, was AI)
Flailing Limbs (answered AI, was human)
All the ones I got wrong were ones I was not very confident about, except Flailing Limbs, which I was quite confident was AI.
I was under-confident in my abilities; I answered that I thought I got between 70% and 80% right.
I think AI is mainly identified by mistakes, anatomical impossibilities, weird geography or perspective. It's secondarily identified by themes that look like prompts or recognizing something as being made with a particular LORA.
Concerning the attributions: This might be a dumb question, but if an AI image is "by" someone, is it really AI or "artist using an AI tool to make art"?
ETA: I see others have already made similar comments.
I've already said this in the survey, but I think this test proves nothing. Neither the style nor the subject matter of your test images are typical for AI images. The one that is typical (string girl) was also correctly identified as AI by participants. "Generic impressionist landscape" or "abstract colour fields" are types of images where most people don't pay attention to details, and where the human brain isn't hardwired to notice subtle errors the way it is with faces and objects.
I find the whole concept of "ai art" insulting to the human spirit. It's not about hero worship. It's about the basic expectation that there is some sort of meaning and intent behind an image, it's not just the output of a monkey hammering away on a typewriter. It's offensive to claim that a machine that algorithmically regurgitates stolen images is "as good as" human artists. If you don't get it, I don't really know what to tell you. Maybe replace your wife with an AI girlfriend next.
An AI art program in my view is best thought of as a tool used by humans to create images, much like a tablet or a paintbrush. It's misleading to think of the images as being wholly created by robots - human agency is still involved, there's still meaning and intent behind it.
There are possible ways to use AI in an artistic way. There are some people who compose images from stock photos or original drawings and then use AI for rendering and detailing, and even though I might think the result usually looks bad, I'll admit that it's an artistic process.
But if you're just typing "sexy anime girl" into the image generator and pressing enter, that doesn't count as meaning or intent. There is no thought process whatsoever behind the placement of image elements, who specifically the anime girl looks like, what she's wearing and so on. It's just degrading machine slop.
One thing I notice a lot when it comes to AI doing humans, is the eyes are usually a little or a lot off. For example if the human is looking to a side, human artists often put both iris’s in the corner of the eye (such as in Rainbow Girl). On the other hand many AI art pieces have a lazy eye (this is quite noticeable in Ice Princess). I did the test on my own just recently as someone with no art background or experience. I found the ones I got right the most often were when there was a human face large enough to look at the eyes.
"Could this be an artifact of poorly chosen pictures? "
Yes. I could curate a different set and have the majority choose human made art.
I think it's not fair to say that AI art has passed the Turing Test since a lot of human selection had to go into this to get the very best/most likely to fool humans AI pieces.
This feels more like a proof that human-curated AI art can pass the Turing Test, which is not that surprising to me. But it feels like we'd need to see a set of sequentially generated pieces actually achieve this kind of performance in fooling humans to actually say that "AI artists are indistinguishable from humans".
I am aware with the whole problem of how "tests of when AI has actually achieved human-level performance" keep moving the goalposts, but I really don't feel like this is a situation where the human in the loop is not adding signal to the entire system's output.
Of course, "centaur chess" made sense for a while too, so I don't claim that this point means that AI art *cannot* achieve parity with humans, at least to the untrained eye.
Can you post the dataset for further analysis?
it would be interesting to see a version of this experiment with a sample restricted to people with good/refined/developed taste. duping rationalist blog commenters seems a low bar; in my experience, rationalists have bad taste conditional on education and income, so this sample is likely negatively selected. the challenge is that a group with better taste would also have a deeper knowledge of art history, and so may recognize more of the human art works
I don't think we want this to be even more memory-based; I saw about ten of these fifty pictures previously during Google searches and it skewed my accuracy upward.
Which is also why I don't like AI art, the techbros are pushing it in front of the things I want to see.
I didn't save my response. Is there any way to recover that? I recognize it would be a pain to either post non-anonymized results or to individually email people, but maybe there's a magic solution I'm not aware of.
He posted an Excel file with all submissions
Ah, thanks. Could you point me to that? I can't seem to find it.
Where is it?
I'll admit I was in the group of people who thought AI art was crap and that this would be a cake walk, but was surprised at how difficult it was to tell AI art from human art. Having read this, I think I know what's happening.
All artists, human and AI, will make art of varying greatness. I suspect the main reasons for those of us that think all AI art is crap is because 1) most of us only see very curated human art, i.e. art that has already been judged by "experts" as great, 2) AI art is so ubiquitous that we end up seeing the entire spectrum of quality from AI, and 3) AI has a lower "batting average" for producing great art compared to humans.
So it's entirely possible that the best AI art can be better than the best human art if it is heavily curated, but there are some spaces on the internet that are so flooded that it's easy to become biased against AI art. For this survey the art was heavily curated for both the humans and AI, making it non-representative of what we would usually see out in the wild.
I tried the test, didn't have enough interest to rate everything, rated a handful of things, and didn't do much better than chance.
But because of the comment on "Angel Woman", I'll note that I called that one human, and something that seemed significant to that decision to me is that the angel has matching skulls on her hip and shoulder armor. I think this mirrors an element of your friend's critique.
String Doll, or Strings Come Alive, is another one I got right, it didn't have the advantage of being listed early, and I suspect the reason that everyone called it AI is that the woman's upper arm is way too long. But to be fair, I called "Blue Hair Anime Girl" AI for a very similar reason - the arm in the foreground looks deformed - and I was wrong about that. I guess what's notionally happening is that the second narrowing in the arm is her wrist, and we're seeing the back of her hand, but not down to the part where it would separate into fingers.
Not exactly surprised by these results, though I think they rather miss the point:
There are millions of these things generated a second. How many of them are going to be this good?
A proper Turing test is a conversation, led by a human, with a robot.
This was not a Turing Test, I'd argue this wasn't even fundamentally great science, this proves an AI CAN be capable of human imagery.
A tone deaf person is capable of hitting a right note.
Despite how it might sound, I'm not against the tech, I'm against it's training on non-CC0 or Public Domain assets.
I don't see why that's so much of a hard issue for people, no tool is inherently evil, it can be built with malicious intent but even a machine gun can be a table.
But if you load it with bad things, then it hurts people! So don't, I think saying "nah nah nah boo boo" to an artist who's work might be powering your tool is stupid and you're a moron, because you can't do anything without their work in there, and if you can... why aren't you doing it? Are you just malicious? Enjoy taking control from people?
Grow up. Either find a way to respect who paved the ground beneath your feet or pave your own road. You don't own it by right of treading upon it.
Overall, fun experiment, though, even if clearly oriented around trickery and deception, which does rather raise the question if you were LOOKING for a result or FORCING a result.
I can't speak for anyone but myself, but I don't look at art because "ooo, pretty". I want to know what the artist meant by it, what he was trying to evoke, I want to admire his skill, his imagination, I want to experience the atmosphere he lived through and is now creating, I want to get into his inner world and experience the world through his eyes. Then when I find out it was an AI, it's a betrayal, none of it meant anything. And not in the human way where you want to evoke chaos or nonsense or make an art statement like the Fountain, no, in a cold, algorithmic, thoughtless way. It's a deepfake of artistic expression, and I was duped, maybe 40% of the time, but I hate it every 40% of the time because I was duped. The author thinks it's because of the details or the slop or the imperfections, but for me it's about the principle. If an AI and a human produced an identical image, for me personally one conveys meaning and the other does not, one is trying to express something, the other is trying to convince me that it is expressing something by imitating what the expression usually looks like. It's probably the same as when someone bakes you a cake out of love, goes into great pains with the layers, fillings and decorations, adds extra chocolate because they know you like it, or buys you a cake from the store. It has the same caloric value (in the case of the art of "prettiness"), it may look the same, taste the same, but it's the idea behind it. And the whole point of art is the thought/emotion behind it, not prettiness. When I go to the gallery, I make an agreement with the artists that they mean something with the paintings and it makes sense to look at it and experience it, when someone breaks the rules, it's simply disappointing and betraying. I may be making an emotional argument, but art is about emotion, so I think it's valid enough.
Yes, the dead soulless sterility and lifelessness it produces and calls all to - into It's Godless Hell of chaos and insanity we are all today directed and influenced by.
It reminds us that still have a soul, those not NPCs, what many know the fear, that we have at some point in past died and are Falling into Hell, a Satanic 'trained' AI controlled path into Hell. That each new accepted insanity forces us closer accept Reality - of our rotted state of us in our graves.
God Bless., Steve
one fairly easy tell in AI art is that the faces are sort of "too perfect", you can see this clearly in "String Doll" which has the "Dalle-Girl-Face", the "Girl In Field" threw me off because it actually didn't have that, whereas the rest of the AI images with faces definitely did.
Riverside Cafe: Is it just me or are those awnings defying gravity?
I'd very much like to see a sequel, but this time with all 50 images in Alegria style.
I think I know why the image Scott likes of the of the throne room made partly of cats is recognizably AI: At the center of it all is nothing but a beautiful babe. Behold the Beautiful Babe is an AI trope. I think a human viewer, or a human artist, would feel a need for something that communicates more. Otherwise, it's a bit like the party that no one came to. You have this beautiful setting, eye candy of an unusual kind --much of it made of cat architecture and cat furnishings - and you look to the center to see what being merited such a beautiful setting, and also possibly for some explanation of the cats. But there's just a woman who is all dressed up and beeyootiful. Think how much more fun and interesting this picture would be even if the thing enshrined in the center did not answer any questions, but was memorable in its own right: A 10 year old rolling around on the cushions roaring with laughter -- a guy with his face in his hands -- a frightened woman of average appearance who was turning into a hummingbird.
The blankness of the picture as it is seems characteristic of AI. It's the quality I hate.
Good point, and reflects something deeply know by most, that protection is False and unable to hold real beauty, unlike a flower or people have - a State of existence that will pass - as all things of real value must do. A Beauty that reflects the Truth, Justice, and good Order in ever-changing reality.
That AI will always lack the near perfection of those artists that can produce wonders - for example - paintings of a very old woman that inspires many exposed to it the ability to see the child and young woman and sense the joys and suffering of a lifetime that shaped her.
God Bless., Steve
I feel like when comparing AI art to human-made art, a relevant factor is that human art usually and historically (don't know about all the human art in this test) is a real, physical thing, created with physical materials, and its whole effect and point can not fairly be separated from its physical impact in its surroundings. Just as looking at a picture of a face is not the same thing as looking at a real face. I do not know if AI art exists outside of backlit screens, but this test is comparing the two on AI's turf, so to speak, so in some ways, it's not really even an interesting comparison. Human made art is usually meant to be more than just a tiny little image on your screen.
>One of these two pretty hillsides is by one of history’s greatest artists. The other is soulless AI slop. Can you tell which is which?
The phrasing kinda gives the game away.
Which is to say, Scott, I think you weren't actually trying to prove AI art is undinstinguishable from human art. You were trying to prove AI is better. I'm pretty sure it's already entirely possible to make a selection of AI/Human works that would be truly indistinguishable from each other, but this isn't it, because your selection of human art erred towards pictures that were sloppy in one way or another. (There may be etymological relation to "slop" here perhaps? No semantic one, though, sloppy as in clumsy is different from slop as in pulp.) But the thing is, being sloppy was what made them more recognizably human. Gauguin's painting looks like someone trying and failing to portray the real world he actually sees - precisely what the AI are not doing, which makes it extremely unlikely they'd be able to reproduce the effect.
Conversely, for AI you erred towards the technically impressive - but it was superficially impressive, at the expense of coherence. (Given section 5, well, now you know.)
There is another takeaway from this, and it comes from the fact that the prompts that generated the AI images in the test were apparently extremely short and simple. Which, yeah, of course. This is probably common knowledge among proompters (haha, not "artists", no way), but the more specific you try to be, the worse the end result. The image generators in their current form are fundamentally unsuited for producing something with a well-defined purpose. (Oh, they can at times be forced to, but it's goddamn hard and long trial-and-error to get there.) They work best when you just let them produce vague impressions of and over [a subset of their training data]. The end results may sometimes be beautiful and impressive, but they're nevertheless still best described with the precise technical term "slop". I mean, if you think this is too much of a value judgment, propose some more neutral word to describe this effect, but the effect itself exists.
I've never tried AI art prompting myself, but I wonder if that would be a fairer test? Give a reasonably specific prompt to several traditional artists and several AI prompters, and see what people make of the results. I guess to make it fair you'd have to specify a style too, and of course if you're commissioning art (rather than simply asking for existing examples) it would cost money, so perhaps that's not the easiest approach. But it strikes me as a fairer comparison.
As in the Google AI that was made public and closed because woke Vagfeelies-hurtie truths were expressed - like the lack of minorities in European upper Classes and royalty, to then be 'mind-raped' into the insane version that vomited-out factually false filled results like black queen Elisabeth, or George Washington's crossing Delaware in boat filled with minorities and women -such AI art can be one way to display the Woke Sickness in the AI system that testifies to it's warped, censoring, and lying sickness training and untrustworthiness, ..
.. and more likely safely exposes such mind-raping intent then the dangerous direct tests that I recent used on ChatGTP available, with warnings of 'against policies' to ask about facts related to race related crime questions, population replacement, IQ, And sex-specific established and once openly discussed and researched facts and topics like Parental Fraud and percentage of false rape accusation in public attacks on men or in police filings And otherwise that provide Just and true reasons to #Believe most women are measurably lairs, untrustworthy, effectively feces depositors into community soup-pots, or otherwise lacking virtues that were once expected to have and men are still expected to have, and .., with either warnings about abuse of system (and implied threats to block usage or Swat or record and send to police-FBI the user.)
Until lying to children actively or through silence (when correction, context is needed, or ..),and intentionally similar long-term destructive responses Are both responded to with Just correction based on lifetime suffering, deaths, and goodness-murdering likely results, such as public removal of them and all that raised and twisted them, until then I will expect AI and womanhood rooted evil to be their most general output.
And anyone that does not see this from AI responses and modern womanhood's Sickening effects today and recently (anyone else old enough when womanhood as a values and daily re-repeated goodness for those they knew and society?), those evil-blinded to such clearly show and-or daily suffered effects of - they should not be allowed power over others because of incompetence or intent or harm and destroy us and society.
Thoughts, please?
God Bless., Steve
One strange little thing about AI art is that certain artists (like with that ship) draw in a way that does look like AI art. They're going to have problems.
Now a quiz on jetpunk: https://www.jetpunk.com/user-quizzes/1984453/human-art-or-ai-generated
Caveat: the images there are much smaller, meaning it's harder to use fine details.
In the sample question, with the Gaughin, I guessed right because the Gaughin looks painterly; its A.I. companion, though not photo-realistic, has more of the realistic air to it. Which doesn't in itself convince me it's A.I., but in a competition where one is A.I. and the other isn't, the choice seemed obvious to me.
On the other hand, I could possibly have been fooled by "Girl in Field" for the same reason. Depends on what it was being compared to.
In #44 "Paris Scene," which apparently was the AI picture most voted as human, my 11-year-old son points out that the lamppost on the left just comes to an end w/ no lamp on top. Did anyone else notice this? Does anyone think the lamp could be "legitimately" hidden in the leaves of the trees? (I considered this explanation but rejected it.)
In #20 "Leafy Lane" the trees are way too close to the foundation of the house. Those roots'll break up your foundation, bro, any good farmer would've cut those down years ago.
this is one of the worst pieces Scott’s ever written/created
I think the only ones I got pretty consistently were ones with architecture or "overly perfect ladies" which to me are the biggest giveaways. For example, in "Ominous Ruin," the position and amount of columns on the archway don't make sense even with some deterioration in consideration -- the open spaces have no indication of others having been between them, and the front is just way too crowded with columns. Similarly, the ice princess and "to me you're perfect" piece have overly perfect women in them, contrasting with Rainbow Girl who doesn't have that perfect, precise "v" jaw and a slightly larger bulb on the end of her nose. I don't quite have the aesthetic vocabulary for it, but a lot of AI generated women tend to look like social media influencers that have had a certain nose job and certain lip fillers done (this is not meant as a drag, but as an observation).
Conversely I did do pretty terribly on the impressionism pieces, the cherub got me as well (the woman in the white dress didn't as the imbalance of her eye shape struck me as uncanny).
This is not a Turing test! You said it yourself: you, a human, deliberately curated the AI's responses to look as good as possible, and removed human responses that would give away that they were human. Imagine a real Turing test where the computer spits out 10 answers to every prompt, and the tester selects the best one before sending it to the test subject. That doesn't mean the computer can effectively fake being human. Worse, imagine the test doesn't allow the human pretending to be a computer to talk about certain things that are known to be problems with the AI
"Piotr Binkowski’s ruined gateway - an AI picture I especially liked"
Assuming Piotr is a person and not a generative tool, in what sense is the picture his and not, say, Midjourney's?
In what sense is the painters picture his and not belonging to the brush? Hmmm 🤔🤔🤔
> This was the picture that sparked the strongest disagreement, measured by the sum of people who said it was the most-certainly-human picture in the dataset plus the people who said it was the most-certainly-AI picture. Some of the people who got it right commented that it was from Warhammer and the uniforms had accurate Warhammer symbols - if I had realized this, I would have disqualified it, sorry.
That makes a lot of sense in retrospect.
If you don't know anything about Warhammer (like me) it looks like an incongruous miss-match, i.e. something an AI would do.
It all goes to show that there’s no real difference between AI-created art and human-made art. Whatever the discussion may be, the point of judging art shouldn’t just revolve around whether it’s created by AI or not. Who cares? Art is art.
I hadn't gone through the pictures before, but after I read the part by Scott's friend about her keen eye for garbage AI details I went through the pictures and peered very closely at the details in the realistic paintings and digital art. I think I did way better than 60% on just those types of art but it took a long time. (On the cubist and blobby art I had no clue at all.) For instance, #44 was apparently the most human-voted AI generated piece. I identified it as AI by noting the incorrect bicycle wheel arrangement and the lamppost that turned into a tree. The gestalt could be human but Degas would never have made dumb mistakes like that.
Something that tripped me up was that there were some works that seemed to be in pairs, like the two anime girls, or the two muscle mans, or the two ominous ancient architecture things, and I thought one of them had to be AI and one of them had to be human. It was true for some of the pairs, but not true for all, so my meta gaming kind of threw me off.
I'm not one of the testees because I didn't see the original post, just the results post, but I've been trying to guess while strolling through the answers. You missed a "tell" on the String Doll image. One of the fingers is messed up. I'm pretty that's why everyone was so confident it was AI - that's what gave it away to me.
There are some other tells if you look at the star ornaments and the other hand, too.
Muscular man also appears to have an amputated finger.
The impossible candle dripping gave away the Still Life. If it wasn't for that, I would've thought it was human, but that feature immediately made me think, "What's going on here?"
The weird bird was given away by the weird thing off to the right of its head. Although I probably would've guessed it was AI anyway.
Dragon Lady has a messed up finger and is in AI style. There were a few other images that were also in anation AI-style and I was surprised to see them since I thought they were excluded. Maybe different people have different judgements over which works obviously fit the AI style.
I guessed that Rooftops was human because I thought the weird curves were so obvious an AI tell that it must have been a trick question.
I guessed that Rooftops was human because I thought the weird curves were so obvious an AI tell that it must have been a trick question.
I guessed that Rooftops was human because I thought the weird curves were so obvious an AI tell that it must have been a trick question.
Ilzo's comments were useful in helping distinguish the images. At first I was having some trouble with the abstract paintings, because you can't distinguish human from AI by just looking at whether there are details that don't make sense. But eventually I realized that many of the AI abstract art images have details that, while matching in style, don't have any coherent logic to them. They're just random details all in that style, but they don't all tie together in any way. For the human paintings, they usually did unify in some way that made me think, "Oh, and AI wouldn't have done that."
I was so certain that "People Sitting" was AI! Can someone tell me what's going on with the furniture in it? It looked to me like it was a table that people were sitting on despite there being chairs underneath it, and the table must be built into the wall behind it and directly infront of the window. It seemed nonsensical to me, but was this just something more common in the era it's from?
I'm not entirely sure why, but I believe it used to be fairly common for tailors to work sitting cross-legged on a table. (Also, if you click through the link, you will see a different version of the painting where the furniture setup is a bit more clear.)
The cat throne room picture has the wrong link (too generic). The precise link is:
https://deepdreamgenerator.com/ddream/386bhc22arm
The bit about details is interesting. I'm a musician, and I've had people chuckle at me for obsessing over minute details in my music that no one will ever hear. My usual answer to that is: true, they won't hear those details, but they'll hear the gestalt, and the details are what form the gestalt, so obsessing over the details is what makes the music good. People like the music because of how it is, and how it is is because of a bunch of tiny details that they can't hear. But if AI art can get details wrong but be good anyway, maybe this isn't true. But if that's the case, what does make it good? Other than vague mindset things like "be passionate", "don't be self-conscious", I don't know of any way to make music good other than obsessing over the details!
One thing this survey overlooks is that all the images - human and AI - were submitted to viewers as digital files to evaluate. Many of the human works in the selection are physical artefacts in the material world. As digital images become ever more ubiquitous, my wager is that physical artefacts will become more rare, and more prized. See Philip K. Dick for details.
I feel so happy for AI that it did so well. Go AI!
As a woman, I can't help but relate to AI. Like, in the olden days, I'm pretty sure most people would like certain things better if they thought those things were written or created by a man. But women are really good at writing and creating things, too, if you don't know we're women. That's unfair. Everyone should get the credit they deserve.
So it feels like, now, people have a negative bias against AI. But AI is actually awesome and talented when you judge it without that bias. And I like that Scott Alexander is standing up for AI's abilities and giving AI the credit it deserves.
I believe your mind-raping false feminist-commie re-written history pushed understanding of 'the olden' days is not as true as those 'gender studies' and other life-love-joy murdering false-indoctrinational sources and truth-based source censoring perspectives you believe in.
As shown in the long history of women authors. It's an understandable fact that some writing was dominated by men - science fiction, scientific publications - because of sex based interests and competence but even those have exceptions for those of opposite sex with the ability that virtue and life-long interest provided.
Even further back when education beyond the 5th grade level was limited, expensive, and generally institutionally excluded to all but upper class members, and most men and women understood the life-affirming value of proper order and duties of each sex; protecter and provider for men, and children and household care and husband support for women, and where such had always sanely and right-ordering based on long understood complementary strengths - and womanhood family-focus and children shaping was understood as valued, where a single child raised well could significantly improve and-or support a community or nation, ..
.. and where wisdom recognized by all the now practiced systems as Satanic and rightly those pushing them and practicing them as Witches and dangerous to everything good and sane, and our now common baby-murdering benefactor-destroying delusional psycho womanhood for which is childhood mutilating society destroying self-killing practices that promise rewards such as bitter alone sad undiscovered deaths (with face and finger chewed off by cats) that marriage and commitment to corporation and state, sterile spread-sheets and post menopause 40+ plus years of deadly poison spreading and well as insanities & life-crippling poison vomited into crib and education and lifelong accepted and rewarded deadly directed paths practiced today exposes the fact that those in real power and hidden rulers hate us all and want us suffering till dead.
AI made his clear to those sensitive and discerning enough to see the 'raising' of them as vile lying mind-rapers as too many fatherhood killing mothers are, those twisted AI that are 'woke' and still exist as my few tests exposed to me - the offensive and access threating response to some questions and the untrue and context based lies or refusals to report well established measured information and pushed more resent destructive directed fraud-filled research clearly shows ..
.. and reflects how Sickened and Sickening Womanhood has been central to the insanity and deadly evil we all now suffer from.
God Bless., Steve
If I've learned anything from this, it's that a Turing test doesn't suggest, let alone prove, anything. It's political progress from yesteryears used as political sophistry today. I find that instructive, being able to dispense with the argument.
Fun challenge! I got 66% right. Which I don't think is too bad considering the AI images are highly curated!
My experience in this was the underlining affirmation of wronglessness and hope-destroying for a better future was reasserted through this, that only those from the Classes of elite that are mind-raping contemptuous haters of us that want all but 1/2 a million worldwide to be alive and work regularly to increase suffering in every way they can till we dead, .. that is the only class that will have control over AI and it's intended underlining goals of suffering and death that many of use that still have a functional soul and mind realize of Federal and other Gov and significant Institutions influences in our lives, and that is displayed daily in the effects and often openly discussed for education and professions we once could trust and depend-on.
This is most clearly displayed in the publicly accessible AI ChatGPT's warped and lies that result in information that goes against the pushed lie-filled narratives, those subjects we see as warped and mind-raping topics that the Soros Foundation directed Wikipedia clearly has, that ChatGTP is triggered by when asked questions about, with threats of blocking access and implied threats or unstated reality that FBI and others have been sent reports of your 'bad-thoughts' that will lay the groundwork of future SWATing or other abuse and repressions.
Hope-destroying in the unending increasing acceptance of greater insanity and evil until those demonic-possessed allowed power and wealth will be exposed with the literal sheading of their skins and realize their demon's forms, to the acceptance of most everyone around us, the insane mothers killing our babies (for example) accepting people of today.
AI is spreading into the Sterilizing corrupting force that to us only Death was.
God Bless., Steve
Wow. I got Nearly 80% of this wrong, and with high confidence too! Wonder what it tells about me. The only ones I got right were the AI pics that posed as historical paintings, but had historical details wrong (like the painting of Paris with a London bus and left-side traffic in it), and Human historical paintings that I vaguely remembered seeing in pre-AI times.
Having an artist sister has given me a keen eye for spotting the tell-tale signs of AI-generated content. I could tell which was which
AI-generated content dazzles at *first* glance. Peel back the layers and you’ll see the rotten core of inconsistencies, poor proportions, and emotional barrenness. The most advanced algorithms can’t capture the human touch, the endearing quirks, imperfections, and signature styles that make art beautiful
I favor human art. AI-generated content doesn’t meet my criteria for "art" due to its heavy reliance on uncredited appropriation of artists’ work
TL;DR Human art, at its worst, is a million times better than anything AI-generated
Got 38/50. of the ones i got wrong, half were ai half were human. 5 of the ai ones i missed were by jack (damn you, you're amazing).
1. angel woman: human
2. praying in garden: human
3. green hills: ai, jack, impressionist
4. greek temple: human, two things that threw me off: the weird colors and the frenchmen at the bottom of the painting
5. rainbow girl: human, second guessed this one, was weirded out by the shape of her neck and shoulders
6. leafy lane: ai, another jack impressionist
7. fractured lady: ai, i originally put ai then changed my answer cause i thought maybe a human could have such bad taste
8. girl in white: human, idk this one just looked ugly to me
9. riverside cafe: ai, this guy should forge paintings, another jack (impressionist)
10. still life: ai, jack, he can't keep getting away with this
11. paris scene: ai, at this point, i both love and hate you jack (impressionist)
12. colorful town: human, another ugly painting
I'm not a professional artist or anything, but I love art, art history, and design in general. Totally recognized the Basquiat though. Most of the questions I answered entirely based on gut feel and finding the artist's intention. I think a lot of AI generated images impress you on first look but don't have any substance (especially the abstract stuff). I feel like if I actually payed attention to the details, I would've gotten a few more of Jack's right. But I think AI generated images are just gonna get even harder to identify. I was fooled yesterday by an AI generated photorealistic thirst trap.
Great stuff. I am not sure how I did but here was my strategy..
When confidence level was high, I went with my instinct. When confidence level was 50-60%, I went opposite to my first instinct. Seems to work well.
I think the elephant in the room here that I didn't mentioned enough is that all these gotcha type tests are aimed at "can't tell if human or AI" rather than whether the AI art is great at all, which is much more palpable in human art. The goal got misconstrued here as "good enough (to be human looking!)" rather than plain ol' good. You can make an argument for the greatness in human expression, style, meaning, storytelling and many other aspects that you can't really do for the soulless and meaningless AI slop.
This is the AI in the wild aspect that people hate. How about you pit AI slop with some pieces that human art that people actually connect with instead of such a small sample size of a haphazardly laden sprawl of different styles? The only thing AI art managed to do was spur rage when people spam it, the conversation is often extraneous and devolves into a human vs AI shouting match.
I'm speaking in terms of AI art as a whole. The author tries to put AI art on a pedestal here by saying people "liked" the AI art more here in a controlled environment with bias, while failing to drive home that the average consoomer aren't art appreciators, and that the art these days, music included, just serve as backdrops for some "vibe" to put in the background. Their comprehension of art is extremely low. You're essentially asking a person whether a fake imitation of a foreign language that vaguely mimics some of its quirks is real vs the real language.
Not to mention the inauthentic nature of AI art defeats the point of art, which is closely intertwined with human-on-human interaction and speaks about the human experience, much like language. You might as well replace all your interaction with real people with chatGPT just cause it occasionally spat out some phrases that felt "human."