Scott's Notes

UGC is a terrible benchmark because it measures visibility, volume, emotion, and manipulation better than it measures truth.

It can be fake. It can be bought. It can be coached. It can be polluted by competitors. It can be written by people with no expertise, no context, and no accountability.

Yet AI systems often treat this mess like a research file, then repackage it into a polished answer that sounds neutral, informed, and trustworthy.

That is not verification. That is reputation laundering.

The problem with UGC is not just that it can be wrong. The problem is that it can be wrong at scale, repeated across platforms, absorbed into search engines, scraped into AI models, and returned to consumers as if it were consensus.

A fake review is bad. A fake review converted into an AI recommendation is worse.

This began as a normal business question. I asked an AI tool to help me find a competent, affordable, knowledgeable, and honest marketing consultant.

What I got back looked researched, structured, and useful. Then I asked where the confidence came from. That is when the whole thing started to fall apart.

This whole exercise was 'prompted' — no pun — by the experts telling dealers how AI search works. How AEO, GEO, and so forth operate. They talk like they know.

They don't. They are fishing in the same stocked trout farm as the customers. The pond only has trout in it. If you are fishing in a trout farm, you are not going to catch a bass.

That is the mistake. The experts cast a different line and think they pulled something different from the same pond. They didn't. They caught the same trout the customer caught.

The problem is not that they are charging for the trout. The problem is they are telling dealers they caught a bass.

I have worked with several of the names on the list. I no longer work with them. I will leave it at that.

I mention this because it matters when a confident, ranked recommendation is presented as research. Experience is not abstract. It is personal, and it carries a weight that a model scraping reputation signals cannot replicate.

This site is a record of what an AI tool said and a critique of how it said it. It is not a claim that any named vendor is dishonest, incompetent, or should be avoided.

The point is the failure of the recommendation process: circular sources, unverified forum posts, and confidence without evidence. The named vendors are incidental to that larger point, even though the AI itself named them.