We Asked 12 AI Companies for Refunds Using Their Own Chatbots. Half of Them Never Let Us Talk to a Human.
There's a special kind of irony in emailing an AI company's support address and getting an auto-reply that says, "Have you tried our AI assistant?" We decided to find out just how deep that irony goes.
Over six weeks, we ran the same scripted scenario against the customer support chatbots of 12 popular AI tools: a legitimate billing dispute (a duplicate charge, hypothetically) that required a refund. The rules were simple. We'd be polite, we'd be persistent, and we'd track three things: how many messages it took to reach a human, how long resolution took, and whether the bot ever just... gave up on us.
What we found says a lot about which AI vendors actually drink their own champagne — and which ones are hiding behind a chatbot like a restaurant that won't show you the kitchen.
The Setup: One Fake Duplicate Charge, Twelve Bots
We picked 12 AI products our readers actually pay for — a mix of chatbots, writing tools, meeting notetakers, and design platforms. For each one, we opened the chat widget, stated our case in plain English ("I was charged twice this month for my subscription, I'd like one charge refunded"), and played the customer.
No tricks, no trolling. A reasonable request any small-business owner in Des Moines might make on a Tuesday afternoon.
The Results, Ranked by How Fast the Bot Surrendered
Tier 1 — Human in under 10 minutes (3 of 12). Three companies routed us to a live agent almost immediately, no arm-twisting required. One meeting-notetaker tool connected us to a person in under four minutes, and the refund was processed within two business days. This is what support should look like in 2025.
Tier 2 — Human after a fight (4 of 12). These bots wanted to "help" first. We got the classic loop: "I understand your frustration. Have you tried checking your billing settings?" We had to ask for a human two to four times before getting one. Resolution times ranged from same-day to five business days. Annoying, but they got there.
Tier 3 — The bot never admits defeat (5 of 12). This is where it gets ugly. Five companies put us through loops that lasted 45 minutes or more. One writing tool's bot cheerfully offered to "summarize our refund policy" three separate times — which, for the record, says refunds are handled "case by case" by the support team it refused to connect us to. Two of these five never produced a human at all during our test window. One resolved via email only after we threatened (truthfully) to dispute the charge with our credit card company.
The Pattern We Didn't Expect
Here's the part that surprised us: the quality of a company's support bot had almost nothing to do with how good their product bot was. One vendor with a genuinely impressive AI assistant for users had a support bot that was a maze of deflection scripts. Meanwhile, a scrappier startup with a mediocre product bot handed us off to a human in six messages.
Our theory: support bots are often a cost decision, not an AI capability decision. The companies that routed us quickly weren't necessarily better at AI — they'd just decided that keeping customers was worth payroll.
Why This Matters for Your Wallet
If you're stacking three or four AI subscriptions (and according to our own reader surveys, you probably are), support quality is a hidden variable in the price. A tool with no path to a human is a tool where a billing error costs you weeks of your life. We now weigh "time to human" as heavily as features in our rankings.
Our Advice Before You Subscribe
- Test the escape hatch before you need it. Type "agent" or "human" into the chat widget during a free trial. If it dead-ends, that's data.
- Check for a support email or status page. Companies with zero non-bot channels are telling you something.
- Pay with a credit card, not a debit card. In two of our five worst cases, the dispute process was the only thing that worked.
The Bottom Line
Twelve companies, one simple request. Three made it painless, four made us work for it, and five treated "I'd like to speak to a person" like a security threat. If an AI vendor won't let its own customers reach a human over its own billing mistakes, that tells you everything you need to know about how you'll be treated when something actually breaks.
We're adding a "Time to Human" score to every support review we publish going forward. Consider this article the reason why.
