Building an AI Patient Chatbot for Urgent Care with n8n, GPT-4, and Langfuse

The first week our chatbot was live, Langfuse — the tracing tool that records every model call — flagged a response that listed outdated holiday hours. Stale schedule data the bot was about to hand to real patients. Nobody had complained yet. Langfuse paid for itself in the first week when it caught that response. We could see the full trace: the exact context that went in, the exact answer that came out.

The model was the easy part. The hard part — the part that earns you the right to deploy at all — was auditability and the context we fed the model. We built the bot around the three questions every urgent care patient asks — “What are your hours?” “Do you take my insurance?” and “How long is the wait right now?” — and instrumented every interaction so we could catch errors like the holiday-hours response.

The problem at the front desk

Our front desk staff were context-switching all day — checking in the patient at the counter, picking up the phone to repeat our hours, answering the same live-chat question for the fourth time that morning. The questions were predictable. The answers were fixed. Staff time was going to repetition instead of the people standing in the building.

We wanted it to answer common patient questions accurately and instantly, send complex questions to a human, read live wait times from our queue-management software, and leave a full paper trail for quality monitoring. The paper trail wasn’t an afterthought. In a clinic, “the bot said what?” is a question someone has to answer on demand.

Three parts

The system has three parts: n8n (a visual workflow tool that orchestrates the steps), GPT-4 (OpenAI’s large language model that generates the answers), and Langfuse (the observability layer that records everything).

Context quality mattered more than model quality. GPT-4 gave mediocre answers with mediocre context; with accurate, current clinic data, it was excellent. The model was never the bottleneck. The data we handed it was.

curl https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "max_tokens": 10,
    "messages": [
      {"role": "system", "content": "Classify the patient query into exactly one label: hours | insurance | wait_time | medical_advice | other. Reply with the label only."},
      {"role": "user", "content": "Do you take Blue Cross?"}
    ]
  }'

The obvious alternative — send every message straight to GPT-4 — dies on cost and latency, and it leaves no cheap place to refuse a medical-advice question before it ever reaches the model. The two-stage bucketing solves both: the cheap GPT-4o-mini classifier runs first, and only the buckets that need real reasoning touch GPT-4.

Why n8n

We chose n8n because:

1. Visual workflow design — non-technical staff can understand and modify flows
2. Self-hosted — patient data never leaves our infrastructure
3. Extensible — easy to add new tools, APIs, and decision branches
4. Version controlled — workflows are exportable and auditable

What a message does

Each patient message follows this path:

1. Message intake — Patient sends a question via web chat
2. Context enrichment — System pulls current clinic hours, wait times, and service availability from Clockwise.MD (our real-time queue-management system)
3. AI response generation — GPT-4 generates a response using clinic-specific context and conversation history
4. Langfuse trace logging — Every interaction is traced with input, output, latency, and token usage
5. Escalation check — If confidence is low or the query is complex, route to human staff via Teams (Microsoft Teams)
6. Response delivery — Patient receives the answer in under 5 seconds

Langfuse: seeing the whole exchange

Healthcare did not require a different technology stack. It required a higher bar for reliability and auditability. A wrong answer about clinic hours is an inconvenience. A wrong answer about services or insurance could send a patient to the wrong place at the wrong time. Every response had to be auditable, and we had to catch quality problems before patients did.

We could have used n8n’s built-in logging. That would have produced raw log lines, which are fine for one failure but not for a question like “show me every session that burned more than 800 tokens.” Langfuse gave us structured traces. The combination of n8n for orchestration, GPT-4 for intelligence, and Langfuse for observability gives us the confidence to deploy AI in a clinical setting while maintaining the transparency that healthcare demands. The model answers the phone. The traces make sure it answers correctly.

Langfuse gives us:

• Full conversation traces — exactly what context was provided and what the model generated
• Latency monitoring — track response times against the <5 second SLA (our target response time)
• Token usage tracking — cost per interaction (tokens are the chunks of text the model bills by)
• Quality scoring — flag responses that may need human review
• Session replay — review full patient conversations for training and improvement

What we monitor

From our Langfuse dashboard, we track:

• Response accuracy — Are answers factually correct about hours, services, insurance?
• Escalation rate — What percentage of queries require human intervention? (target: <10%)
• Patient satisfaction signals — Follow-up questions that indicate confusion or frustration
• Edge cases — Novel questions that the system hasn’t seen before

Showing those traces to clinical staff built confidence faster than any demo. People trust what they can inspect.

What we measured

After deploying the chatbot across our clinic network, the numbers looked like this:

I want to be honest about those qualifiers. Response time under 5 seconds — verified, we hit it. The 95% accuracy and 10% escalation numbers are targets we’re actively measuring toward, not results we’ve certified. I’d rather say “monitoring” than claim a number we haven’t earned.

The biggest win is outside the table. Front desk staff can focus on the patients standing in front of them instead of answering the phone to say, “Yes, we’re open until 8 PM.”

Where it refuses or hands off

The system is designed to fail gracefully:

• Unknown questions → Acknowledge the limitation, offer to connect with staff
• Medical advice requests → Firm redirect: “I can’t provide medical advice, but our providers can help when you visit”
• Emotional/urgent situations → Immediate escalation to human staff
• Multi-language queries → Spanish support in development

The medical-advice branch worried me most. We did not ask the model to judge risk. We hardcoded the refusal. If the cheap classifier returns medical_advice, the message never reaches GPT-4. There is no model call and no chance of a confident answer to a question it should not answer.

We launched narrowly, with only hours, location, and insurance queries. We added wait times, services, and booking only after validating the previous domain. Shipping the whole thing at once was tempting. We did not.

What’s next

• Appointment booking integration — Let the chatbot actually schedule visits, not just answer questions about them
• Insurance verification — Pre-check coverage before the patient arrives
• Multi-location support — Scale across our clinic network with location-specific context
• Post-visit surveys — Automated follow-up to capture patient feedback