The short version
- Ask whoever you are paying to explain what they are doing in words you could repeat to someone else afterwards. That is the whole test and it takes two minutes.
- If you cannot repeat it, you cannot evaluate it. You are running on trust, and trust is not a control.
- I learned this the expensive way. Eighteen months, two agencies, 40% of turnover gone.
- It is happening again with AI, and faster, because adoption has moved quicker than understanding has.
Quick links
- What is the two-minute test?
- What it cost me to skip it
- I see the same shape most months
- Why this was never really about SEO
- What does this look like with AI?
- How do you check what AI assistants say about your business?
- What do you do with a bad answer?
- Questions I get asked
What is the two-minute test?
Ask whoever you are paying to explain what they are doing, in words you could repeat to someone else afterwards.
Not in their words. In yours. You should be able to walk out of that meeting, find a colleague who was not in it, and tell them what you are buying and why it works.
If you cannot do that, you cannot evaluate the work. You can only decide whether you like the person doing it.
That distinction is worth sitting with, because most buyers think they are evaluating and they are actually just trusting. Trust is fine right up until the moment something goes wrong, and then it turns out you have no way of telling the difference between a supplier having a bad quarter and a supplier who does not know what they are doing.
I am not suggesting you need to become an expert. I have no interest in painting my own walls. But if I hire a decorator I want to know enough to ask what preparation they are doing and to notice if the answer is thin.
What it cost me to skip it
I was an optician. I qualified in 1996, and in 1999 I set up an independent practice in Leeds city centre.
The dot-com boom was happening around us and we could see an opportunity. Not a need. Nobody in an opticians in 1999 needed a website. We just thought there was something there.
So we started selling sunglasses online, under a different brand name, because the manufacturers we worked with would authorise online sales but would not be seen supplying an online shop. Oakley, Dior, Chanel, Bolle. Twenty-five years later they all sell direct themselves, which tells you how that particular principle held up.
For five years it worked. We were the biggest online sunglasses retailer in the UK, buying more stock than Debenhams.
I did not understand why it was working. I just knew that it was.
We were paying an SEO agency. They were doing things. I could see the results, I could not see the method, and because the results were good I never pushed on the method.
Then sales dropped.
40%
of turnover gone, over eighteen months, after Google penalised techniques our agency was using and nobody in our business had ever asked what those techniques were.
Jonny Ross, first-hand. Leeds, 2004 to 2006
It took us a month to work out what had happened. We were on page seven of Google for our own brand name.
We went back to the agency. We spent more with them. Then we moved to a second agency and spent more again.
“Keep doing what we are doing. It will come back.”
What both agencies told me, every month, for eighteen months
In the end I found somebody in New York charging $250 an hour, which in 2005 felt outrageous. He fixed it in two weeks. He understood something that two agencies I had been paying for years did not.
By then it was too late. Competitors had taken the top two positions and the only route back was to spend heavily again. The senior partner did not want to. I sold my shares.
I see the same shape most months
My own story is twenty years old, so it would be fair to ask whether it still happens. It does, and the version I meet now is quieter.
A business somewhere between twenty and eighty staff is paying a monthly retainer for something digital. It was signed off two or three years ago, often by somebody who has since left. Reports arrive. Nobody reads past the first page, because the first page is the only part anybody understands.
When I ask what the supplier is actually doing, I get a list of deliverables rather than a method. When I ask why it should work, the room goes quiet.
That is usually not a bad supplier. Most of them are perfectly competent and some are excellent. It is a missing control, and the business only discovers it is missing at the point where something needs challenging.
Why this was never really about SEO
For a long time I told this as a story about a Google penalty, which is the obvious reading and the wrong one.
The technology worked. The understanding never got inside the business.
We had bought a capability and parked it outside the company. When it was working, that looked like efficiency. When it stopped working, we had no one internally who could tell whether the people we were paying were right, wrong, or guessing.
That is the actual failure, and it has nothing to do with search. You can make the same mistake with an ERP system, a finance function, a piece of machinery or an AI tool. It is also why I keep saying that AI will not fix your marketing on its own.
The thing that makes innovation stick is somebody inside the business understanding it well enough to challenge the supplier. Not to do the work. To challenge the work.
That person does not have to be a full-time hire, which is the argument behind what scaling businesses need before they hire a Fractional Chief AI Officer.
What does this look like with AI?
Adoption has moved a lot faster than understanding.
12% to 35%
the rise in UK businesses with 10 or more employees reporting they use AI, between late 2023 and June 2026. Among businesses with 250 or more employees it reaches 49%, against 28% for those with fewer than 10.
ONS, Business Insights and Conditions Survey, wave 159, 15 to 28 June 2026, published 20 July 2026
Nearly tripling in under three years is a serious rate of change. What those figures do not tell you is how deep any of it goes, and in the rooms I stand up in the honest answer is usually not very.
Most of what I see is task-level. Somebody is using it to draft emails, somebody else is summarising documents, and none of it is joined up or written down. That is not a criticism. It is the normal early phase of any technology, and it is exactly where we were with websites in 1999. It is also the gap I wrote about in why most businesses stay stuck between AI tools and growth systems.
The risk is the same as the one I walked into. You buy something that works, you cannot explain why it works, and you find out the difference between those two things at the worst possible moment.
How do you check what AI assistants say about your business?
This one is worth doing yourself, today, because buying decisions are increasingly made before anyone contacts you. I have written before about how AI is already describing your business to potential buyers, and most owners have never checked what it says.
Worth being precise about
AI visibility
It is not a ranking position. It is whether an assistant names you when somebody describes their problem and asks who can solve it, and what it says about you when it does.
There is no results page to check, which is why so few businesses have any idea where they stand.
Here is the routine. It costs you twenty minutes and no money.
The twenty-minute AI visibility check
- Open three different AI assistants. Ask each the question your best customer would ask before they knew your name. Something like: I am looking for someone in West Yorkshire who can help with X, who do you recommend?
- Write down who gets named. If you are in there, good. If you are not, that is the finding, and it is better to know.
- Ask it why it recommended those particular ones. The reasoning it gives you is the useful part, not the list.
- Ask whether it has heard of your business, and if so, why it did not recommend you. Push. Argue with it a bit.
- Do the same across all three, because they disagree, and the disagreement tells you which signals are thin.
You will not come out of that with a strategy. You will come out of it able to have a much better conversation with anyone selling you one, which is the point.
What do you do with a bad answer?
Run the two-minute test and you will get one of a few shapes of answer. They are easy to tell apart once you know what you are listening for.
| What you ask | A workable answer sounds like | A worrying answer sounds like | What to do about it |
|---|---|---|---|
| What are you actually doing? | Plain language, specific actions, and they check you have followed | Jargon, or a list of deliverables rather than a method | Ask them to say it again for someone who does not work in their industry |
| Why does that work? | A mechanism, and the conditions where it would not work | Because it is best practice, or because it works for our other clients | Ask what would have to be true for it to fail. A good supplier has an answer |
| How will we know it is working? | A measure tied to money, and a date by which it should move | Rankings, impressions, reach, or anything with no cash attached | Agree the measure in writing before the next invoice, not after |
| It is not working. Now what? | A changed plan, and a reason the first one did not hold | Keep doing what we are doing, it will come back | This is the one I ignored for eighteen months. Get a second opinion that week |
On a phone, scroll the table sideways.
A supplier who cannot explain their work is not necessarily a bad supplier. Plenty of genuinely good technical people are poor explainers, and I have worked with some of them.
But you still have a problem, because the explanation is your only control. If they will not give you one, you need somebody who can translate, and that person needs to be on your side of the table.
Questions I get asked
What if I do not know enough to judge the answer?
You know more than you think. The test is not whether the answer is technically correct. It is whether you can repeat it. That is a test you are fully qualified to run.
Is it not rude to ask a supplier to justify themselves?
Any supplier worth keeping will be pleased. The ones who bristle at plain questions are telling you something useful for free.
How often should I run it?
When you first engage someone, and any time performance changes in either direction. Good results are the most dangerous time to stop asking, which is the mistake I made.
Does this apply to AI suppliers specifically?
It applies to anything you buy and cannot personally do. AI is just the current example, and the one where the gap between adoption and understanding is widest.
Run the test on me
If you are paying somebody for something you could not explain to a colleague, that is worth a conversation, and you should come at it the same way.
Ask me the four questions in the table above. What am I actually doing, why does it work, how will we know, and what happens when it does not. If I cannot answer them in words you could repeat afterwards, you have learned something useful for the price of half an hour.
