Content design
Overview
Extra Day Grace Period gives checking clients until the next business day to cover overdrafts—one of the most common and emotionally charged reasons people contact their bank.
Problem
Overdraft questions made up a disproportionate share of chatbot volume—over 3,000 queries in the analysis window alone. The existing responses were static, couldn’t adapt to different account types, and often deflected to “call us.”
Goal
Design AI-powered chatbot responses that handle ambiguity gracefully, personalize by account type, and resolve overdraft questions without a phone call.
Analyzed 3,000+ chatbot utterances and found that vague queries (“extra day grace period”) outnumbered specific ones 10:1—meaning the response had to handle ambiguity as the default, not the edge case.
Redesigned conversation flows to route by account type and query specificity, covering edge cases like Clear Access Banking and non-checking products that the previous system missed entirely.
Authored grounding documentation, FAQs, and source content for AI-generated responses—repurposing legally approved .com content to accelerate compliance review from weeks to days.
I started where most content designers don’t: the data. Before writing a single word, I analyzed thousands of real chatbot utterances to understand what clients were actually asking—and where the existing system was failing them.
Utterance analysis
Categorizing 3,000+ queries as vague or specific revealed a critical insight: vague queries like “extra day grace period” (2,838 occurrences) outnumbered specific ones by 10:1. The existing system treated vague queries as failures. I reframed them as the primary use case—the response needed to handle ambiguity gracefully, not punt to a phone call.
Conversation flow
I redesigned the conversation architecture to route clients based on account type and query specificity. The previous flow had a single path; the new one handles edge cases—Clear Access Banking accounts, non-checking products, clients who are already enrolled vs. those who aren’t—ensuring each person gets contextually relevant information instead of a generic explainer.
Content iterations
I explored three approaches for the FAQ response and three for the overview—from low-lift repurposed pages to scannable accordion patterns. Rather than presenting a single recommendation, I gave stakeholders options with tradeoffs: speed to ship, scannability, and information density. This built alignment faster than a single “here’s my solution” pitch.
FAQ response options
Overview response options
Option 1: Disclosure text
Option 2: Bulleted format
Option 3: Card with illustration
The final design pairs a concise overview with a detailed FAQ, both written in plain language and structured for scannability. The chatbot adapts its response based on account type and query specificity—something the previous system couldn’t do at all.
The overview screen gives clients what they need in one glance; the FAQ page lets them go deeper. Every response is grounded in legally approved source content, ensuring accuracy without sacrificing conversational tone.
Chatbot overview response
FAQ detail page
To power AI-generated responses, I authored grounding documentation built from content already published on the bank’s .com site—a deliberate strategy to fast-track legal review. I then evaluated outputs both manually and with AI, providing structured feedback to model trainers that directly improved response quality.
Source documentation
I restructured existing .com content into grounding documentation for the AI model—giving it accurate, legally pre-approved source material. This cut legal review time significantly and ensured response accuracy from day one.
AI evaluation
Across two testing rounds, I evaluated AI-generated responses and identified systemic issues: repetitive phrasing, overly chatty language, weak product awareness, and unnecessary “call us” deflections. My structured feedback to model trainers drove measurable improvements in subsequent rounds.
Current Production response
Response generated with AI based on source document
Original source
Revised source document
Results
Evaluated 349 AI-generated responses across 14 intents over two testing rounds.
My three intents—Extra Day Grace Period, common overdraft reasons, and manage overdraft—achieved 100% accuracy, no-harm, and completeness scores.
Identified four system-wide issues that improved response quality across all intents, not just mine.
Reduced default “call us” deflections by restructuring grounding documentation and consolidating narrow Q&A pairs.
Takeaways
Grounding documentation is only as good as the source content it’s built on. Investing in clear, structured source material paid off directly in AI response quality.
AI evaluation at scale surfaces patterns manual review misses—but human judgment is still essential for tone, empathy, and edge cases.
Presenting stakeholders with multiple options (plus a clear recommendation) builds trust and speeds alignment—especially in regulated environments where everyone has opinions.
This project is under NDA
Enter the password to view this case study
Incorrect password — try again