Marketing and model collapse — Kelford Labs Daily

Have you heard about “model collapse”?

It‘s the idea that that machine learning models (including Large Language Models like ChatGPT) gradually degrade as they are trained more and more on the outputs of other machine learning models.

Basically, if you train an AI solely on the outputs of other AIs (to the neglect of human-made content), eventually the model will degrade, or “collapse”.

This was a big topic a few years ago. It prophesied the end of “scaling laws” as AI labs ran out of human-made content to train on. The fear was that “synthetic data” would lead to model collapse, instead of steady improvement.

But, for AI labs, the good news is that so long as human-made content does not go away, and is accumulated alongside AI outputs, model collapse seems to be theoretically avoidable. Even as the use of synthetic data increases.

What this means, though, is AI labs and the models they train crave novelty. They crave, they depend on, human-made content to fuel and feed them.

And they may naturally, based on their incentive and reward structure, tend to eschew content that is clearly AI-generated.

This is all to say that, if you want your marketing content to enter the “training data” for search engines and AI models, it may benefit you to focus on human-made content. Or at least human-curated, human-edited.

The idea is to generate novel tokens. Things the models haven’t seen before or have only seen rarely, instead of feeding them back their own tokens that they have an incentive to ignore.

The benefits of churning out AI-generated content may be real, but they may also be extremely brief. And in the meantime, we might be teaching search engine crawlers and AI algorithms to ignore us, to even avoid us, if they don’t want their training to be tainted by their own outputs.

When we help our clients create content that’s augmented or accelerated by AI, we always begin with their own words and then hand-edit the text, introducing disfluencies, abbreviations, contractions, and grammatical eccentricities.

We sand off the uncanny veneer of AI writing and apply a human touch.

If you’re using AI for your content, I recommend considering doing the same.

(As always, this newsletter is 100% organic, human-made).

Reply to this email to tell me what you think, or ask any questions!

Kelford Inc. shows you the way to always knowing what to say.