Are we capable or reliable? — Kelford Labs Daily

In the world of AI and large language models, there’s a lot of chatter about the “capability-reliability gap”.

(Here’s a link to a talk on YouTube about the topic if you’d like to learn more)

Basically, LLMs can be extremely capable. They can perform amazing, sometimes superhuman feats of what looks like intelligence.

But they are simultaneously extremely unreliable. They can perform amazing things sometimes, occasionally most of the time, but never all of the the time.

Many (including me) suspect that this gap will keep most consumer-facing “agentic” AI systems, which perform real-world actions autonomously, from gaining mainstream traction for at least a little while.

You don’t want an AI doing real things in the real world with real consequences on your behalf if it gets it wrong a meaningful (and perhaps irreducible) amount of the time.

But here’s what this has to do with you and me:

Are we marketing our capability or our reliability?

Let’s face it, anyone can pull off a miracle every once in a while. But it takes real quality, real expertise to do great every single time.

Like that old Joel Raphaelson quote goes, “People do not choose Brand A over Brand B because they think Brand A is better, but because they are more certain that it is good.”

Sure, we’ll always want (and should want) to show off our capabilities and just how amazing we can be.

But we shouldn’t neglect showing off, and demonstrating, our reliability.

The processes, the principles, the purpose behind our work and why it works so well, all of the time.

Some buyers are looking for a miracle, but most are looking for certainty.

We need to demonstrate reliability so they can have it.

Reply to this email to tell me what you think, or ask any questions!

Kelford Inc. shows experts the way to always knowing what to say.