Where the AI Slop on Facebook comes from

Remember my last post about the flood of weird AI-generated images on Facebook? Well, buckle up, because I've discovered the shocking truth behind these bizarre posts. Thanks to an eye-opening article by 404 Media last month, I now know why our feeds are drowning in this AI slop – and you won't believe who's actually paying for it.

404 Media did a deep dive on "Where Facebook's AI Slop Comes From" (paywall unfortunately, but $10/month for their quality reporting is worth it) and made some fascinating discoveries on an entire network of accounts creating all sorts of of AI images, from the weird and surreal to depictions of poor people and children in starvation, or disaster situations. All of this in the name of profit.

I've encountered a number of these posts depicting army vets with prosthestetic limbs holding up a birthday cake, or extremely malnourished people holding up a sign "why doesn't anybody love me", or a photo of a landscape that is an optical illusion - where you're supposed to see Jesus if you squint your eyes. I'm sure you've also seen stuff like this.

How it works

These 'creators' set up Facebook profiles and pages, and fill them with posts generated with AI generated images. There are a number of generative AI models these days, including many that are accessible for free, including Microsoft's AI Image Creator, or Stable Diffusion which you can run on your own computer if you have a decent GPU.

This makes generating hundreds, or thousands, of images trivial, compared to doing it by hand using image editing tools like Adobe Photoshop.

Monetization

How is it that they are making money from this? Well, Facebook pays them. Facebook has something called the "Creator Bonus Program" which rewards content that garners high engagement, in other words, content that goes viral.

Facebook pays somewhere around $3 to $10 per 1000 likes, which doesn't seem like much, but when you do it at scale, it all adds up to hundreds to thousands of dollars. These scammers are also based in primarily in India and Southeast Asia where cost of living (and income) is low.

There are several YouTube channels with tutorials on how to exploit this Facebook program, by generating images using prompts like 'poor people thin body' to create images that evoke a strong emotional response, or aligns with your values and beliefs, or just plain silly and ridiculous that you just can't help but clicking on the 'Like' button and sharing it with your network.

They also use bots and click-farms^[1] to artificially boost engagement, so that Facebook's algorithm is more likely to show it to users who don't follow those accounts. From my experience, they also use Facebook Groups to garner followers and likes, which also influences what gets shown in your feed.

These YouTube influencers and scammers use Telegram groups to coordinate their efforts, share images and prompts that have been successful and tips on what works on gullible audiences.

Meta does not seem to have the means, nor the inclination to moderate this. There have been layoffs from Facebook's content moderation team in 2023 and Meta's Oversight Board earlier this year. Meanwhile, in that 404 Media article, Meta says these AI-generated images are not in violation of its policies.

This does not bode well for us, as the number of AI slop posts will only increase, and our feeds will be drowning in these low-quality, low-effort images, to the exclusion of actual posts from friends, family and content creators we care about.

The rapid improvement of generative AI models also means these images are getting much more realistic and harder to distinguish from real photos, which creates mistrust in ALL photography. This is very troubling as the spread of misinformation and disinformation will only increase.

Conclusion

I am shocked but frankly not surprised that all these content farm accounts generating AI slop are doing it for money, just that a few months ago I did not know HOW they were being paid, when in fact Facebook themselves are to blame, and these scammers are exploiting a loophole.

So the next time you see a AI generated image in your feed, from an account you don't follow, be extremely critical of the content and the motives of the person posting it, and DO NOT click on 'Like' (or any other reaction for that matter) it and absolutely DO NOT share it on your own feed. Even commenting "This is AI!" is still engagement - though I'm on the fence on this, because other ignorant people need to be warned.

On the other hand, support your creators making authentic art. No matter what their medium of choice is, photography, painting, sculpture, writing, music.

I have further thoughts on Generative AI, and how it relates to the creation of art, stay tuned.