Matador's GuideGeek hit 98% travel-AI accuracy with live human takeovers and RLHF

Matador Network was a travel publisher with no plans to become a tech company when its development team got access to GPT in late 2022, just before ChatGPT launched. Within months it shipped GuideGeek, an AI travel planner travelers message on WhatsApp. Accuracy was about 85% — meaning roughly one in six conversations included a miscommunication or hallucination, which Klopp calls a fun gadget rather than game-changing tech.

The fix was organizational. The media platform attracted a critical mass of users, whose corrections gave a basic reinforcement signal. On top of that, Matador hired five people, most multilingual, to put RLHF 'into high gear': to date they have monitored 1.5 million user-AI conversations, catching subtle misses, manually taking over conversations to get them back on track, then flagging and categorizing each issue for a backend fix that improves an entire category of questions.

A separate system categorizes and reframes questions before they hit the model — 'this weekend' is translated into actual dates so old event pages stop causing hallucinations, and 'Airbnbs in Vancouver' followed by 'close to Yaletown' is rolled into one query. Under the rewriting layer sits a plugin network: when a question's intent is categorized, the bot pings partners for flight times, hotel pricing or exchange rates and feeds the result to the LLM. Klopp's stated rule for gaps: a friendly 'I don't yet have that capability' beats a hallucination while the plugin backlog catches up.

Treat your media audience as the training pipeline: launch at 85%, let reviewers take over failing chats live, and turn every save into a category-level fix

Live takeover turns every support failure into labeled training data, so the same mistake never has to be fixed twice by hand.

The media audience supplied conversation volume at critical mass, which is what makes human review of 1.5 million chats worth staffing.

Reframing questions attacks hallucinations at the source — most bad answers were answers to the wrong question.

Real-time partner data by intent means the bot answers from current sources instead of a model's memory.

Admitting missing capability keeps accuracy honest while the product area is built out.

Accuracy is an ops problem before a model problem: a human takeover rescues the user and labels the data, rewriting the question beats a bigger model, and 'I can't yet' beats a hallucination.

By January 2025 Klopp reports 98% accuracy reached over 15 months of RLHF, more than a million users, partnerships with major travel brands and awards — figures self-reported by the CTO in his column, with no independent audit in the piece.