We run showcase storefronts of our own products, real sites wired to the real system, so we feel problems before a client does. This month, one of those showcase real-estate sites taught us an expensive lesson: its AI assistant had stopped answering. Not answering badly. Not answering slowly. Answering nothing, to everyone, in complete silence, while every monitoring light stayed green.
The cause was almost funny. We had connected a live MLS data feed to the site, and the assistant's briefing dutifully grew to include every one of them: 4,989 published listings, each with its address, price, beds, baths and link, stuffed into the instructions the AI reads before every single reply.
That briefing came to 506,188 tokens. The model underneath has a hard limit of 200,000 tokens of working memory in its standard configuration (Anthropic publishes the number). We were handing it two and a half briefings' worth of paper and asking it to speak. It declined. And because the refusal happened deep inside a streaming response, the visitor saw a chat widget that simply never replied.
The instinct that causes this is a good instinct
Every business owner who gets an AI assistant has the same first thought: give it everything. Every listing, every FAQ, every policy, the whole services binder. It feels like diligence. An assistant that knows everything can answer anything, right?
Here is the problem with that instinct, and it has nothing to do with token limits. A receptionist who has to re-read the entire MLS binder before answering the phone is slower and more expensive on every call, and one day the binder gets too thick to lift. Long before our showcase site went silent, the same design was quietly making every conversation cost more and drift more, because the model was wading through thousands of listings to answer "what areas do you serve?"
The token limit did not create the flaw. It just finally made the flaw visible.
What we build instead now
The fix we shipped is a design rule, not a bigger bucket: the assistant carries highlights, and bridges to everything else.
| Give it everything | Bridge design | |
|---|---|---|
| Listings in the briefing | All of them, forever growing | A curated set of current highlights |
| "Do you have anything on Maple Street?" | Guesses from a stale wall of text, or dies trying | Says honestly what it carries, links the site's own search, offers an agent follow-up |
| Cost per conversation | Grows with your inventory | Flat, small |
| Failure mode | Silent collapse at a hard limit | None inherent; the honest handoff is the design |
The assistant on that site now carries its top listings with a plain statement that these are a selection, points visitors at the site's full listing search for everything else, and offers to take contact details so a human agent can answer the specific question. Which is what a good human receptionist does with a thick binder: she does not memorize it, she knows where it is.
And because silence is never an acceptable answer, the chat widget itself got a rule of its own: if a reply fails for any reason, the visitor sees an honest "I'm having trouble right now, please use the contact options on this page" instead of a dead screen. An error a person can see costs you a little pride. An error dressed as a working chat costs you the lead.
The quiet failure is the one that gets you
The part of this story that should worry a business owner is not the token math. It is that the failure was invisible. The server returned success codes. The widget loaded beautifully. Every check we would have written two years ago said the system was healthy, while every actual visitor got silence and left. We only caught it because we run adversarial audits against our own properties, where someone's whole job is to behave like a real visitor and distrust every green light.
So if you already have an AI assistant on your site, from us or anyone, here is the test that costs you ninety seconds: open your own site in a private browser window tonight and ask your assistant three real questions. Not a demo environment. The live site, the way a customer meets it. You are checking for the failure your dashboards cannot see.
What to ask a vendor before you sign
- What happens to the assistant as my data grows? If the answer is "it knows all of it," ask what happens at ten times your current size. Listen for whether they have thought about it at all.
- What does a visitor see when the AI fails? The only good answer is a specific, honest fallback message with a human path. "It doesn't fail" is the worst answer on the market.
- Can I watch a failed conversation land somewhere? Captured leads should be visible to you even when, especially when, something upstream misbehaved.
We build the AI front office for real estate teams and local businesses this way because the assistant's job was never to be an encyclopedia. Its job is to be the fastest possible bridge between a person with a question and the human who can actually help them, and a bridge that collapses in silence is worse than no bridge at all. You can watch the design in action on our live demo site, or read how we think about an AI that speaks for your business.
Give your AI less to memorize and more to be honest about. Your visitors will get more answers, not fewer.
Sources
- Anthropic, model documentation listing the standard 200,000-token context window. platform.claude.com
Frequently asked questions
How much information should an AI assistant be given about my business?
Enough to answer the common questions completely: services, areas served, hours, pricing approach, and a curated set of current inventory or work examples. For everything past that, the assistant should bridge, pointing to your site's own search or pages and offering to connect the visitor with a person. Volume of knowledge is not the goal; a reliable answer or a fast handoff is.
Why did the AI fail silently instead of showing an error?
Streaming chat responses report success as soon as they begin, so a failure that happens mid-stream can leave the visitor with nothing while every server metric looks normal. A well-built widget watches for that case and shows an honest fallback message with a human contact path. If your vendor cannot describe their failure message, they probably do not have one.
Does connecting an MLS feed to an AI assistant work at all?
Yes, and it is worth doing, but the feed should power the website's listing pages and search, while the assistant carries a curated selection and links to the rest. The wrong design pipes the entire feed into the assistant's instructions, which grows your cost per conversation and eventually breaks it outright.