This is a JigsawFlux project. JigsawFlux builds open-source tools for health tech, humanitarian response, and crisis management — in places where "cloud-native" is not an option and IT budgets are measured in grants, not headcount. That context imposes hard constraints on every architecture decision: portability, cost, and freedom from vendor lock-in.

The frameworks here — LangGraph, CrewAI, and AutoGen — were chosen because they meet those constraints. They are open source, actively maintained, and run entirely on hardware you own. Alternatives like Microsoft Semantic Kernel or Amazon Bedrock Agents are capable, but they introduce hard dependencies on specific cloud ecosystems. That trade-off doesn't fit the JigsawFlux model.

Each framework is also the natural implementation home for a different family of agentic patterns — which is the other reason for this grouping, and why this is Part 1 of a two-part series. Part 2 implements those patterns directly.

The shared task​

Every framework runs the same pipeline: a Researcher agent uses a DuckDuckGo web search tool to gather facts on a given topic, then a Writer agent consumes those notes and produces a structured Markdown report. The topic in all runs was solid-state batteries.

python run.py --framework all --topic "solid-state batteries"

This symmetry is deliberate. Same task, same model (claude-sonnet-4-6), same tool — only the orchestration framework changes. That isolation means the telemetry at the end reflects framework behaviour and overhead, not model variation.

The full source is at github.com/JigsawFlux/comparing-agent-frameworks.

LangGraph: the stateful graph​

LangGraph models execution as a directed graph where nodes are functions and edges are routing decisions. State flows through every node as a typed dictionary — you define what it contains, and reducers control how it gets updated. ^[1]

The researcher loop is cyclic by design. It calls web_search, gets results back via the tool node, and loops until it decides it has enough information — at which point the conditional edge routes to the writer.

The state schema makes the data contract explicit:

class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], add_messages]
    topic: str
    research_notes: str
    final_report: str

The routing logic is a single function:

def should_continue(state: AgentState):
    last_message = state["messages"][-1]
    if last_message.tool_calls:
        return "tools"   # loop back through tool node
    return "writer"      # research complete

And the graph wiring reduces to five lines:

workflow.add_edge(START, "researcher")
workflow.add_conditional_edges(
    "researcher",
    should_continue,
    {"tools": "tools", "writer": "writer"}
)
workflow.add_edge("tools", "researcher")   # the cyclic loop
workflow.add_edge("writer", END)

What you get from this: fine-grained control over every routing decision, complete visibility into state at every step, and trivial output extraction (state["final_report"]). What it costs: you are building a graph. The mental model is powerful but requires internalising nodes, edges, reducers, and the distinction between cyclic and acyclic topologies before you can be productive.

Agentic patterns this enables: ReAct ^[5] (the researcher loop is a ReAct loop), Plan-and-Execute ^[6], ReWOO ^[7], Reflexion ^[8], DAG pipelines, human-in-the-loop via interrupt().

CrewAI: declarative roles​

CrewAI inverts the mental model. Instead of defining a graph, you define agents with a role, goal, and backstory, and tasks with a description and expected output. Hand them to a Crew and it handles the orchestration. ^[2]

researcher = Agent(
    role="Senior Technology Researcher",
    goal=f"Conduct deep research on '{topic}' and compile key insights",
    backstory=(
        "You are a highly analytical research specialist. To stay within strict "
        "API rate limits, use the search_tool exactly ONCE with a broad query. "
        "Produce precise, structured notes."
    ),
    tools=[search_tool],
    llm=llm,
)

writer = Agent(
    role="Expert Technical Writer",
    goal=f"Synthesize research notes into a professional technical report on '{topic}'",
    backstory=(
        "You are a veteran technical publisher who specialises in explaining complex "
        "advancements in clean, structured Markdown."
    ),
    llm=llm,
)

Tasks are similarly declarative — each specifies what it needs and what it should produce:

research_task = Task(
    description=f"Research '{topic}' using web_search. Identify: core concept, benefits, key players, barriers.",
    expected_output="Detailed, structured notes listing research facts.",
    agent=researcher,
)

write_task = Task(
    description=f"Write a professional report on '{topic}' from the research notes.",
    expected_output="A structured technical report in Markdown format.",
    agent=writer,
)

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, write_task],
    process=Process.sequential,
)
result = crew.kickoff()

Context passing between tasks is implicit — CrewAI injects the previous task's output as input to the next one. You never touch state directly.

This produced the richest research notes of the three runs: 9,351 characters versus LangGraph's 5,573. The declarative backstory and role fields give the model stronger framing to work from, and the verbose CrewAI reasoning traces generate more intermediate content. The trade-off is that sequential execution is the easy path; anything more complex — conditional routing, cyclic reasoning — requires switching to Process.hierarchical and adding a manager agent.

Agentic patterns this enables: Hierarchical agent (add a manager with Process.hierarchical), role-based pipelines, multi-agent delegation.

AutoGen: conversation-based​

AutoGen treats agent coordination as a conversation. Each agent is a participant; they exchange messages, call tools, and signal completion via a termination string. There is no graph, no task object — just agents talking. ^[3][9]

The pipeline runs in two separate phases. Phase 1 is the research conversation:

researcher = autogen.AssistantAgent(
    name="Researcher",
    system_message=(
        "You are a Senior Researcher. Use the web_search tool ONCE to gather facts. "
        "Compile structured research notes. When complete, end with TERMINATE."
    ),
    llm_config=llm_config,
)

user_proxy = autogen.UserProxyAgent(
    name="UserProxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=5,
    is_termination_msg=lambda x: "TERMINATE" in x.get("content", ""),
    code_execution_config=False,
)

register_function(web_search, caller=researcher, executor=user_proxy, ...)
user_proxy.initiate_chat(researcher, message=f"Research '{topic}'...")

The UserProxyAgent is not a human — it is the tool executor. When the researcher emits a tool call, the proxy executes it and sends the result back as a message. The loop ends when the researcher appends TERMINATE.

Phase 2 repeats the pattern with a Writer agent and a fresh proxy, passing the extracted research notes as the opening message:

user_proxy2.initiate_chat(
    writer,
    message=f"Write a professional report on '{topic}' based on:\n\n{research_notes}"
)

AutoGen's conversation-native model is best suited to topologies where agents negotiate, debate, or vote — patterns where the back-and-forth is the mechanism, not just the means to an end.

Agentic patterns this enables: Peer-to-peer network, Consensus/Joint, Human-in-the-loop (swap UserProxyAgent for an actual human).

Telemetry: what the runs produced​

All three ran against the same topic on the same hardware with claude-sonnet-4-6. ^[12]

Three things stand out.

LangGraph and CrewAI both produced full reports. The 41-second gap between them reflects CrewAI's more verbose internal reasoning — it generates more intermediate text per agent turn, which explains the longer notes. Both took the same 15-second rate-limit sleep between agent phases.

AutoGen was fastest and produced almost nothing. 76 seconds, 0 notes, a 322-character "report" that turned out to be the writer's input prompt echoed back. This is not an AutoGen model failure — the model ran successfully. It is a message extraction problem in the runner. AutoGen stores conversation history per agent pair (user_proxy.chat_messages[researcher]), and the termination-message stripping removed content that overlapped with the notes before extraction. The model did its job; the output pipeline had a subtle bug.

This is worth pausing on because it reveals a real architectural difference. LangGraph's typed state makes output extraction trivial — the final report is simply state["final_report"]. CrewAI exposes it via task.output.raw. With AutoGen, you parse message history, and subtle bugs in that parsing can silently produce nothing. The conversational flexibility that makes AutoGen powerful for multi-agent debate also makes structured output extraction more fragile.

A note on n8n​

LangGraph is sometimes compared to n8n — a visual, platform-managed workflow tool that also supports AI agents. Both can orchestrate multi-step agentic workflows, but they operate at different layers. ^[4][11]

Use LangGraph when the logic is complex, cyclic, and needs to live inside your existing Python application. Use n8n when you need rapid API integration, non-developer maintainers, or built-in webhook triggers without writing ingestion routes.

For JigsawFlux use cases — clinics, crisis response, volunteer-run operations — either works on-premises. The deciding factor is usually who will maintain it: developers reach for LangGraph, operational staff reach for n8n.

Framework decision guide​

All three have a free tier and run on hardware you already own. None require a cloud subscription to develop or deploy.

What's next​

This comparison is the foundation for Part 2, which implements and benchmarks agentic patterns directly — using whichever framework is the natural fit for each:

Single-agent patterns

• ReAct ^[5] — reason-act loops using LangGraph's cyclic edges
• Plan-and-Execute ^[6] — separate planning phase from execution
• ReWOO ^[7] — plan all tool calls upfront, then execute without intermediate observation
• Reflexion ^[8] — self-critique and iterative self-improvement

Multi-agent topologies

• Hierarchical — orchestrator delegates to specialised sub-agents (CrewAI)
• DAG — directed pipeline with no feedback loops (LangGraph)
• Peer-to-peer network — lateral agent communication without a central manager (AutoGen)
• Consensus/Joint — multiple agents debate and converge on a shared answer (AutoGen)

The project source is on GitHub: github.com/JigsawFlux/comparing-agent-frameworks. ^[10]

References​

Frameworks

[1] LangGraph — langchain-ai/langgraph, GitHub.

[2] CrewAI — crewAIInc/crewAI, GitHub.

[3] AutoGen — microsoft/autogen, GitHub.

[4] n8n — n8n.io, workflow automation platform.

Research papers

[5] Yao, S. et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.

[6] Wang, L. et al. (2023). Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. arXiv:2305.04091.

[7] Xu, B. et al. (2023). ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models. arXiv:2305.18323.

[8] Shinn, N. et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366.

[9] Wu, Q. et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework. arXiv:2308.08155.

Project sources

[10] JigsawFlux. comparing-agent-frameworks — README.md, framework comparison matrix and architecture overview.

[11] JigsawFlux. comparing-agent-frameworks — langgraph_vs_n8n.md, architectural comparison of LangGraph and n8n.

[12] JigsawFlux. comparing-agent-frameworks — telemetry from python run.py --framework all --topic "solid-state batteries" using claude-sonnet-4-6 (2026-06-28). github.com/JigsawFlux/comparing-agent-frameworks.

This is a JigsawFlux project. JigsawFlux builds open-source tools for health tech, humanitarian response, and crisis management — tools designed to work on constrained budgets, unreliable infrastructure, and donated hardware. If you are working on something in this space, or want to contribute, the JigsawFlux GitHub organisation is where the work happens.

The problem this POC targets is specific: small and charity hospitals where doctor time is genuinely scarce and IT budgets are measured in hundreds of dollars, not thousands. A consultation isn't just a diagnosis — it's intake, medical history retrieval, triage sorting, prescription recording, pharmacy stock checking. The typical workflow hands all of that to a doctor anyway, because there's no other option. The result: a clinician spending 40% of their time on work that doesn't require clinical judgment.

The premise here is simple. AI handles everything that doesn't require a clinician. The doctor steps in exactly once — to read the AI-produced intake summary and give a diagnosis. That's it. The prescription agent takes over from there.

This is a JigsawFlux project. JigsawFlux builds open-source tools for health tech, things that matters — tools that have to work in the real world, not the well-funded one. That means two hard constraints shaped every architecture decision here: cost and deployability. Viable on a shoestring budget. Runnable in places where "cloud-native" isn't an option — a clinic with a single server, unreliable internet, and an IT team of one.

Built on AWS Bedrock (Claude Haiku 4.5), LangGraph for orchestration, LangChain @tool wrappers for data access, and Streamlit for the UI. Total cost: < $0.01 per consultation. Deployable on a £25/month VPS or a clinic's own hardware, with the option to go fully on-premises as models improve.

Why Not Lambda + Bedrock?​

The obvious AWS pattern for an LLM-powered app is Lambda + Bedrock: a Lambda function receives a request, calls the model, returns a response. It works well for single-turn, stateless interactions.

A medical consultation is not a single-turn interaction. It looks like this:

Patient submits symptoms        [seconds]
↓ intake agent runs             [~5 seconds]
↓ graph pauses — doctor queued  [minutes to hours]
↓ doctor submits diagnosis
↓ prescription agent runs       [~3 seconds]
↓ patient receives prescription

That middle gap — between intake completing and the doctor responding — can be minutes or hours. Lambda invocations are stateless. Every invocation discards all context. Modelling a pause-and-resume workflow with Lambda means rebuilding state management, custom polling, webhook handlers, and retry logic from scratch. The honest Lambda-native path for a stateful workflow like this is Lambda + Step Functions + API Gateway + DynamoDB — a real AWS stack with real AWS complexity and real AWS consulting costs.

But there's a second problem with Lambda for this use case: it is irrevocably cloud-only. A charity clinic in a low-connectivity region cannot run Lambda on their own hardware. If the internet goes down, consultations stop. Patient data has to leave the building to be processed. There is no on-premises option, no hybrid option, no path to data sovereignty.

LangGraph running on a single Python process is none of that. It runs on a clinic's existing server, a cheap VPS, a donated laptop. The only network call during a consultation is the Bedrock API invocation — a small JSON payload, only made when the model is actually thinking. Patient records stay local. Connectivity interruptions only affect the model call, not the workflow state. And in a future phase, Bedrock can be swapped for a locally-hosted model (Ollama running llama3.2 or similar), taking the cloud dependency to zero.

The Architecture​

Three layers, each with a clear responsibility:

┌─────────────────────────────────────────────────┐
│  LangGraph                                      │  Orchestration
│  StateGraph · nodes · edges · interrupt()       │  (workflow logic, state, routing)
├─────────────────────────────────────────────────┤
│  LangChain AWS  (langchain-aws)                 │  Integration
│  ChatBedrock · @tool wrappers                   │  (model wrappers, tool schemas)
├─────────────────────────────────────────────────┤
│  AWS Bedrock  (Claude Haiku 4.5)                │  Intelligence
│  Foundation model · pay-per-token               │  (reasoning, generation)
└─────────────────────────────────────────────────┘

The LangGraph StateGraph encodes the consultation as nodes (agents) and edges (routing logic). Here's the full workflow:

Every node reads from and writes to a single typed state object that flows through the entire graph:

# graph.py
class ConsultationState(TypedDict):
    session_id: str
    patient_id: str
    symptoms: str
    patient_history: dict

    intake_summary: str
    is_emergency: bool
    triage_score: int    # 1=Routine, 2=Minor, 3=Moderate, 4=Severe, 5=Urgent non-emergency
    triage_reason: str   # one-sentence AI justification for the score

    doctor_clarification_req: str
    patient_clarification_ans: str

    doctor_notes: str
    prescription: str

    status: str          # intake | emergency | awaiting_doctor | clarifying | prescribing | complete
    messages: list       # rendered in patient chat UI

This typed contract is what makes the graph inspectable and debuggable. At any pause point you can call graph.get_state(config) and see exactly what every field holds.

Wiring the graph up is straightforward — build_graph() is the only assembly point:

# graph.py
MODEL_ID = os.getenv("BEDROCK_MODEL_ID", "us.anthropic.claude-haiku-4-5-20251001-v1:0")

def _get_model():
    return ChatBedrock(
        model_id=MODEL_ID,
        model_kwargs={"temperature": 0.3},
        region_name=os.getenv("AWS_DEFAULT_REGION", "us-east-1"),
    )

def build_graph():
    workflow = StateGraph(ConsultationState)

    workflow.add_node("intake_agent",        intake_agent_node)
    workflow.add_node("emergency_protocol",  emergency_protocol_node)
    workflow.add_node("doctor_review",       doctor_review_node)
    workflow.add_node("clarification_agent", clarification_agent_node)
    workflow.add_node("prescription_agent",  prescription_agent_node)

    workflow.set_entry_point("intake_agent")

    workflow.add_conditional_edges(
        "intake_agent",
        should_escalate,
        {"emergency_protocol": "emergency_protocol", "doctor_review": "doctor_review"},
    )
    workflow.add_edge("emergency_protocol", END)

    workflow.add_conditional_edges(
        "doctor_review",
        should_clarify,
        {"clarification_agent": "clarification_agent", "prescription_agent": "prescription_agent"},
    )
    workflow.add_edge("clarification_agent", "doctor_review")
    workflow.add_edge("prescription_agent", END)

    checkpointer = MemorySaver()
    return workflow.compile(checkpointer=checkpointer)

The conditional edge functions are one-liners that read directly from state:

def should_escalate(state) -> Literal["emergency_protocol", "doctor_review"]:
    return "emergency_protocol" if state.get("is_emergency") else "doctor_review"

def should_clarify(state) -> Literal["clarification_agent", "prescription_agent"]:
    return "clarification_agent" if state.get("doctor_clarification_req") else "prescription_agent"

Human-in-the-Loop: How interrupt() Actually Works​

The doctor_review node is not a model call. It's a pause point — here's the real code:

# graph.py
def doctor_review_node(state: ConsultationState) -> dict:
    context = {
        "session_id": state["session_id"],
        "patient_id": state["patient_id"],
        "intake_summary": state["intake_summary"],
        "patient_history": state["patient_history"],
    }
    if state.get("patient_clarification_ans"):
        context["clarification"] = {
            "question": state["doctor_clarification_req"],
            "answer": state["patient_clarification_ans"],
        }

    # Graph pauses here until Streamlit calls graph.invoke(Command(resume=doctor_input), config)
    doctor_input = interrupt(context)

    if doctor_input.get("action") == "clarify":
        clarification_req = doctor_input["question"]
        _update_consultation(
            state["session_id"],
            doctor_clarification_req=clarification_req,
            status="clarifying",
        )
        return {
            "doctor_clarification_req": clarification_req,
            "patient_clarification_ans": "",
            "status": "clarifying",
        }

    doctor_notes = doctor_input.get("notes", "")
    _update_consultation(
        state["session_id"],
        doctor_notes=doctor_notes,
        status="prescribing",
    )
    return {"doctor_notes": doctor_notes, "status": "prescribing"}

When interrupt(context) fires, LangGraph serialises the entire ConsultationState to the checkpoint store (MemorySaver in this POC; DynamoDB in production) and stops. The doctor's Streamlit tab polls the database for pending consultations and shows the intake summary.

When the doctor acts, Streamlit resumes the graph with a single call:

# app.py — doctor submits diagnosis
graph.invoke(
    Command(resume={"action": "diagnose", "notes": final_notes}),
    config,
)

# app.py — doctor asks patient a clarifying question
graph.invoke(
    Command(resume={"action": "clarify", "question": question.strip()}),
    config,
)

# app.py — patient answers the doctor's question
graph.invoke(Command(resume=answer.strip()), config)

Each Command(resume=...) call picks up execution from exactly where interrupt() left it — no rebuilding state, no re-running the intake agent, no webhooks.

The Streamlit UI shares the in-process graph across all browser tabs via @st.cache_resource. This is what makes the patient → doctor state handoff work without any network calls:

# app.py
@st.cache_resource
def get_graph():
    return build_graph()  # single graph instance, shared across all sessions

MemorySaver lives inside that cached object. Patient tab writes state; doctor tab reads it — same process, same memory, zero coordination overhead.

The clarification loop uses the same interrupt() mechanism a second time. clarification_agent_node pauses the graph waiting for the patient's answer; when they reply, the graph resumes and routes back to doctor_review via a fixed edge. A cyclic human workflow that would need custom state machines, polling infrastructure, and webhook handlers in a Lambda-based system becomes three graph edges and two interrupt() calls.

Demo Walkthrough​

Here's a complete consultation using Patient P001 (Alice Thompson, headache and fever).

1. Login — role and PIN selection

The login screen is a placeholder for Amazon Cognito. The hardcoded PIN (1234 for patients, doctor for doctors) illustrates exactly where real authentication would plug in — the rest of the graph doesn't change.

2. Patient submits symptoms — AI intake runs

The patient types their symptoms and clicks Start Consultation. The intake agent fetches the patient's record and medical history from SQLite, searches the knowledge base, assigns a triage score (1–5), and hands off to the doctor queue. The entire intake takes a few seconds.

The graph is now paused at doctor_review. Nothing in the system runs until the doctor acts.

3. Doctor desktop — triage queue and response panel

In a separate browser tab the doctor logs in. The queue shows all pending consultations sorted by triage severity. For Alice's case:

The doctor sees the triage badge (🟡 Level 3 — Moderate), the AI-generated intake summary with clinical context drawn from Alice's history and allergies, and a response panel with two actions: Submit Diagnosis or Ask Patient.

4. Patient receives the prescription

After the doctor submits, the prescription agent checks pharmacy inventory (Amoxicillin is seeded as out of stock — it flags this and suggests an alternative) then records the prescription. The patient's view updates:

Try these scenarios yourself:

The Tool Layer: @tool Now, MCP Later​

The intake and prescription agents access data through LangChain @tool-decorated functions:

@tool
def get_patient_record(patient_id: str) -> dict: ...

@tool
def get_medical_history(patient_id: str) -> list[dict]: ...

@tool
def search_knowledge_base(query: str) -> list[dict]: ...

@tool
def record_prescription(session_id: str, prescription: str) -> str: ...

These are bound to the Haiku model in intake_agent_node and the agent runs a standard tool-calling loop — invoke, check for tool calls, execute them, feed results back, repeat until the model produces a final JSON response:

# graph.py — intake_agent_node (core loop)
def intake_agent_node(state: ConsultationState) -> dict:
    llm = _get_model()
    intake_tools = [get_patient_record, get_medical_history, search_knowledge_base]
    llm_with_tools = llm.bind_tools(intake_tools)
    tool_map = {t.name: t for t in intake_tools}

    messages = [
        SystemMessage(content=INTAKE_SYSTEM),
        HumanMessage(content=f"Patient ID: {state['patient_id']}\nSymptoms: {state['symptoms']}"),
    ]

    for _ in range(6):  # cap tool-calling iterations
        response = llm_with_tools.invoke(messages)
        messages.append(response)

        if not getattr(response, "tool_calls", None):
            break  # model produced final answer — no more tool calls

        for tc in response.tool_calls:
            fn = tool_map.get(tc["name"])
            if fn:
                result = fn.invoke(tc["args"])
                messages.append(ToolMessage(content=json.dumps(result), tool_call_id=tc["id"]))

    # parse the structured JSON response and return updated state fields ...

The model is instructed via INTAKE_SYSTEM to respond only with a JSON object containing intake_summary, is_emergency, triage_score, triage_reason, and a patient_message in the patient's detected language. The prescription agent follows the same loop pattern, using check_pharmacy_inventory and record_prescription as its tools.

For a POC this is the right choice: zero infrastructure, SQLite queries are synchronous, and the entire stack is Python.

In production, three problems make this untenable:

1. Security — the agent has unrestricted database access. There is no way to enforce that the intake agent can only read records for the current patient.
2. Reuse — a second agent type (specialist referral, pharmacy checker) would need to duplicate or tightly share these functions.
3. Auditability — no independent record of which agent accessed which data, a compliance requirement in clinical settings.

Model Context Protocol (MCP) solves all three. Each tool becomes a standalone server process with its own IAM role, access policy, and CloudWatch audit log:

The last row is the key. The LangGraph node functions in graph.py don't change — they still call get_patient_record() and search_knowledge_base(). Only the implementations in tools.py swap from direct SQLite calls to MCP client calls. The tool signatures are the interface contract; the interface is already final.

AWS Verified Permissions (Cedar policies) in Phase 2 adds the access enforcement: "intake_agent may call get_patient_record only for the patient_id present in the current session."

Cost​

For a small or charity hospital, cost isn't an optimisation — it's a constraint. A £2,000/month telemedicine SaaS subscription is simply not available. £20/month might be.

LangGraph on a cheap VM is modestly cheaper than the Lambda-native equivalent — but the bigger difference is operational complexity and the option to run on hardware you already own. A clinic that already has a server pays only for Bedrock tokens.

Hybrid deployment options:

The "Bedrock API only" row is worth emphasising. During a consultation, the only data that leaves the clinic's network is the prompt sent to the model — the patient's symptoms and the anonymised intake context. The patient database, medical history, and prescription records never leave the machine. That matters for GDPR compliance and for clinics operating in jurisdictions with strict patient data rules.

Two model decisions keep the token cost low:

Haiku 4.5, not Sonnet 4. Haiku is ~8× cheaper per token. In this system, the doctor is always the final clinical authority — the triage score is a queue-sorting mechanism, not a clinical decision. The AI's job is to produce a useful intake summary, not to diagnose. Haiku 4.5 does that reliably, including structured JSON output (triage_score, intake_summary) and native multilingual support for non-English-speaking patients at no extra cost.

Cross-region inference profile. One gotcha worth flagging: Claude Haiku 4.5 requires a cross-region inference profile, not a direct model ID. Invoking the bare model ID returns a ValidationException. The default in graph.py is already the correct form:

# graph.py
MODEL_ID = os.getenv("BEDROCK_MODEL_ID", "us.anthropic.claude-haiku-4-5-20251001-v1:0")
#                                                         ^^^
#                                          cross-region prefix — required for on-demand throughput

If you override BEDROCK_MODEL_ID with a bare model ID (e.g. anthropic.claude-haiku-4-5-20251001-v1:0), switch it back to the us. prefixed version.

POC → Production Roadmap​

The architectural bet at the centre of this design: graph.py — the nodes, edges, conditional routing, and interrupt() calls — never changes across any production phase. Infrastructure evolves; the workflow doesn't.

Phase 2 is the architectural pivot. Everything before it is scaffolding. Everything after it scales. The move from @tool wrappers to MCP servers is the only change that touches security, reusability, and auditability simultaneously — and it does so without touching the graph.

Beyond the core phases, the design also supports additive extensions for low-resource environments: a WhatsApp/SMS patient interface via Twilio (the graph's interrupt() is channel-agnostic — state simply sits in DynamoDB until the next SMS arrives), voice note intake via AWS Transcribe or self-hosted Whisper, and native multilingual support (already live in the POC — Haiku 4.5 detects the patient's language and responds in kind while always producing the doctor summary in English).

What's Next​

The full source — graph.py, tools.py, app.py, seed_db.py, and all architecture documentation — is on GitHub: github.com/JigsawFlux/agentic-clinic.

To run it locally:

git clone https://github.com/JigsawFlux/agentic-clinic
cd agentic-clinic
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python seed_db.py
streamlit run app.py

You'll need AWS credentials and Bedrock model access for us.anthropic.claude-haiku-4-5-20251001-v1:0 in us-east-1. The README has the full setup steps including the Bedrock console access request and a troubleshooting section for the two most common ValidationException and ResourceNotFoundException errors.

The four scenarios in the demo section are good starting points — emergency path, clarification loop, out-of-stock pharmacy substitution, and multilingual intake. Each exercises a different branch of the graph.

This is a JigsawFlux project. JigsawFlux builds open-source tools for health tech, humanitarian response, and crisis management — tools designed to work on constrained budgets, unreliable infrastructure, and donated hardware. If you're working on something in this space, or you want to contribute to this project, the JigsawFlux GitHub organisation is where the work happens.

Neither story is about bad engineering. Both are about one broken default: when developers get unrestricted access to frontier models, they use frontier models for everything.

I've been working through a framework to fix this — not by restricting access, but by routing the right task to the right model. The goal is frictionless development that doesn't quietly drain your budget. Here's how it works.

The shift that changed everything​

Until early 2026, enterprise AI tooling largely ran on flat-rate subscriptions. You paid a seat fee, your team used whatever they needed, and costs were predictable. That model is gone. GitHub Copilot retired flat-rate allowances in favour of token-metered "AI Credits" on June 1, 2026. Every major provider has followed the same trajectory.

The problem isn't the pricing model — it's that usage habits haven't caught up. Complex agentic tasks and heavy reasoning models consume tokens exponentially faster than simple completions. A developer running Claude Opus to autocomplete a for-loop is swinging a sledgehammer at a drawing pin. The pin goes in either way; the difference is what the swing cost.

That's the sledgehammer problem, and it's the single largest source of AI budget waste: identical output, ten times the price.

Tier your workload​

The fix is simpler than it sounds: match the model to the complexity of the task. Not every request needs deep reasoning. Not every request needs a lightweight model either. Three tiers cover almost everything.

The goal isn't to ban frontier models — it's to use them where they actually earn their keep. A well-structured RAG pipeline or a cross-cutting refactor across a million-line codebase? That's a Tier 3 problem. A standard API endpoint in Go? It isn't.

Two traffic lanes: IDE vs programmatic​

Tiering the workload is half the answer. The other half is routing enforcement — making sure models are actually selected based on task complexity rather than developer habit. The architecture I recommend splits AI traffic into two distinct lanes, each governed differently.

Lane A: IDE traffic via GitHub Copilot​

Everyday inline coding and IDE chat stays within the GitHub Enterprise account. Governance happens through native Copilot policies: seat-based licensing, Targeted Model Rules, and usage caps. The key move is enforcing "Auto" mode as the default for standard users — it selects an appropriate model rather than defaulting to the most expensive one. Senior architects and principal engineers can be granted Tier 3 access for the work that justifies it.

Lane B: Programmatic traffic via Azure APIM​

Custom scripts, CI/CD pipelines, and internal tools route through a single Azure API Management gateway. Rather than each team managing its own provider keys and burning Tier 3 credits by default, everything flows through a central control point that provides:

• Identity-based access control via Entra ID
• Semantic caching — identical prompts return cached responses at zero token cost
• Intelligent routing across Azure AI Foundry and AWS Bedrock
• Dollar-based budgets enforced per team or pipeline

The caching point deserves emphasis. An automated pipeline making 50 near-identical classification requests doesn't need to pay for 50 model calls. It pays for one, caches the response, and the remaining 49 return instantly — the most straightforward cost reduction in the entire framework.

Architecture​

What this looks like day-to-day​

Three scenarios that show the framework working in practice:

The developer writing .NET boilerplate in VS Code opens a file and starts typing. Copilot's Auto mode kicks in with a Tier 1 model — fast, cheap, accurate for the task. The developer never thinks about model selection, and the team's AI Credits aren't quietly being drained by Opus completions for standard controller logic.

The architect testing a RAG pipeline writes a Python script to benchmark embedding strategies. Instead of managing three different provider API keys, they authenticate once via Entra ID and send all calls through the APIM gateway. The gateway checks their team budget, routes to the appropriate tier, and logs everything. If their budget is close to the limit, a threshold alert fires — not a surprise invoice.

The data pipeline categorising code risk across 50 repositories runs nightly. The first repository's analysis is processed and cached. Repositories two through fifty hit the semantic cache and return at zero token cost. A task that would otherwise consume 50× the tokens costs the same as one.

Rolling this out​

Three phases, and the order is deliberate:

1. Gateway first. Stand up Azure APIM and migrate all programmatic API calls to the unified endpoint. Establish identity-based routing and per-team budgets before anything else. You can't govern what you can't see.
2. IDE governance second. Once programmatic traffic is visible and controlled, implement GitHub Enterprise Targeted Model Rules. Restrict Tier 3 access to roles that genuinely need it; set Auto as the default for everyone else.
3. Team education last. Infrastructure changes enforce behaviour. Education reinforces it. Once the system is in place, rolling out guidelines on prompt scoping, context compression, and model selection across VS Code, Cursor, and Antigravity lands on a foundation that already supports the habits you're trying to build.

The temptation is to start with education — it feels fast and low-risk. But guidelines without enforcement fade. Get the infrastructure right first.

The non-profit dimension​

Everything above assumes an enterprise with a budget to optimise. But the same billing shock hits harder when you're running on grants, donations, or a volunteer-funded open-source project. Non-profits face a specific version of this problem: the same productivity pressure, the same tooling expectations from technical staff, and far less capacity to absorb a surprise $2,000-per-engineer monthly bill.

The tiering framework still applies — but for cost-constrained organisations, there are additional levers worth considering beyond just routing between model tiers.

Local inference removes marginal token cost entirely. Tools like Ollama let you run open models (Llama, Mistral, Phi, Gemma) on local hardware or a small cloud VM — I've written about exactly this setup before, running Ollama on a Kubernetes home lab. The per-query cost drops to near zero; you trade API fees for infrastructure and a ceiling on model capability. For Tier 1 tasks — boilerplate, unit tests, simple completions — a locally hosted 7B or 13B model is often indistinguishable in output quality from a cloud API call, at a fraction of the ongoing cost.

Local hosting earns its keep in two situations beyond pure cost. The first is edge deployment: when inference needs to run on devices in the field — clinics with unreliable connectivity, crisis-response hardware, remote sensors — a cloud API isn't slow, it's unavailable. The second is subtler: for high-frequency, low-complexity workloads, the network round trip itself becomes a cost. Every cloud call pays latency and egress on top of the token price. A local model answering in tens of milliseconds, with no per-call fee, beats a cloud frontier model answering the same mundane question in two seconds — both on responsiveness and on the invoice.

Corporate cloud hosting sits between local inference and direct API access. Running models through AWS Bedrock or Azure AI Foundry — rather than calling Anthropic or Google directly — typically costs 20–40% less per token under enterprise agreements. More importantly for non-profits, it keeps data within a known compliance boundary, avoids egress fees from mixed-cloud setups, and makes budget governance easier through existing cloud billing tools. If your organisation is already on Azure or AWS, routing AI workloads through Foundry or Bedrock is often the fastest path to meaningful cost reduction without changing tooling.

The decision tree for cost-constrained teams:

1. Can a local model (Ollama + Llama/Mistral) handle this task acceptably? Use it.
2. Is your org already on Azure or AWS? Route through Foundry or Bedrock first.
3. Does the task genuinely need a frontier capability? Then pay for the direct API — but only then.

The same tiering logic, extended one level further down.

References​

1. Visual Studio Magazine, "Copilot Billing Shock Hits Developers" (June 4, 2026)
2. InfoWorld, "GitHub shifts Copilot to usage-based billing" (April 28, 2026)
3. Forbes, "Uber Burns Its 2026 AI Budget in Four Months on Claude Code" (May 17, 2026)
4. AI Magazine, "Why Uber Has Already Burned Through Its AI Budget"
5. Inc. Magazine, "1 Company Spent Half a Billion Dollars on Claude in a Single Month" (June 5, 2026)
6. Tom's Hardware, "Mystery company accidentally blew $500 million on Claude AI in a single month" (May 29, 2026)

Appendix: How this post was written​

This post was drafted and iterated entirely using Claude Code — which makes it a small live example of the tiering approach it describes. Three sessions, three different kinds of work, with measurably different token costs for each.

Session 1 — structural work (Sonnet + Haiku subagent)

The first session covered the heavy lifting: converting a raw strategy document into a structured blog post, rewriting from third-person enterprise-doc tone to first-person narrative, and setting up the publishing infrastructure (frontmatter, truncate markers, draft log).

Session 2 — prose and new sections (Sonnet)

The second session expanded the post: narrative restructuring, the non-profit section, and the first version of this appendix. Same model as Session 1, but cheaper — targeted edits on a stable structure produce far fewer output tokens than a full rewrite.

Session 3 — narrative pass (Fable 5)

The final session switched to claude-fable-5, Anthropic's narrative-optimised model, for a light editorial pass: metaphor consistency, sentence rhythm, and voice. The smallest change set of the three sessions — and, unexpectedly, the most expensive.

Look at the numbers side by side. Sonnet generated 31.4k output tokens across two sessions of structural and prose work for $2.01. Fable generated 5.1k tokens — a sixth of the volume — for $2.25. The lightest session cost the most, because premium per-token pricing dominated token count entirely.

That accidental result is a cleaner demonstration of this post's thesis than anything I planned: model choice drives cost more than workload size does. Whether the premium model was worth it for an editorial pass is exactly the kind of question the tiering framework exists to force. Total across all three sessions: $4.28.

Full token breakdown: _DRAFT_LOG.md in this post's directory.

This post moves the whole thing into Kubernetes. The goal isn't enterprise-grade infrastructure — it's a home lab setup that's reliable, easy to extend, and honest about its limitations.

Manifests are in the ollama-mcp-starter repo under backend/k8s-deployment/.

The Hardware​

The server is an Intel NUC Hades Canyon (NUC8i7HVK) — a small-form-factor machine with a skull logo on the lid and a surprisingly capable spec for its size:

It draws modest power for a home server, runs quietly, and fits on a shelf. These are the things that matter when it's on 24/7.

The GPU caveat​

The Vega M GH is a capable GPU for graphics workloads, but Ollama's GPU acceleration uses CUDA — an NVIDIA-only technology. AMD GPU support via ROCm exists in Ollama but requires manual configuration and is not supported through the standard Kubernetes GPU operator.

In practice: Ollama on this box runs CPU-only. The i7-8809G handles inference well enough for personal and home lab use — expect 10–20 tokens/second with llama3.1:8b. More on this in Lessons Learned.

Why Move to Kubernetes?​

Running Ollama on bare metal works, but once you add a second service (Open-WebUI), you're managing two processes manually. A third service and it becomes unwieldy. Kubernetes solves this cleanly:

• Auto-restart — pods restart automatically on crash; no babysitting processes
• Resource limits — cap CPU and memory per service so one greedy process can't starve the others
• Persistent storage — PersistentVolumeClaims keep model data across pod restarts
• Internal DNS — services talk to each other by name (ollama-service.ollama.svc.cluster.local) rather than fragile IP addresses
• Ingress — one load balancer IP, hostname-based routing to multiple services

For a single-node home lab, MicroK8s is the right choice. It installs as a snap package, ships with the add-ons you need, and doesn't require a multi-node cluster to be useful.

Architecture Overview​

Before diving into setup, here's the logical view of the full system:

Two access paths to Ollama:

• MCP path (from Part 1) — Browser → Static Frontend → Node.js Backend → MCP Server → ollama.local
• Direct path — Browser → ai.local → Open-WebUI → Ollama (internal ClusterIP)

Both paths go through MetalLB and Nginx on the NUC. The MCP app and frontend still run on the dev machine; only the inference layer lives in Kubernetes.

MicroK8s Setup​

Install​

sudo snap install microk8s --classic
sudo usermod -aG microk8s $USER
newgrp microk8s

Verify the node is ready:

microk8s kubectl get nodes

Enable Add-ons​

microk8s enable dns
microk8s enable storage
microk8s enable ingress
microk8s enable metallb

When enabling MetalLB, you'll be prompted for an IP range. Use a small slice of your local subnet that won't conflict with DHCP — for example:

192.168.1.50-192.168.1.60

MetalLB will assign IPs from this pool to LoadBalancer services. The ingress controller picks up 192.168.1.54 in this setup.

kubectl alias​

MicroK8s ships its own kubectl. To use the standard kubectl command (useful for remote access from another machine):

microk8s config > ~/.kube/config

Then install kubectl on your client machine and point it at the exported config.

GPU Operator​

microk8s enable gpu

This installs the NVIDIA GPU Operator. It works well if your node has an NVIDIA card — it handles driver installation, container runtime configuration, and makes GPUs schedulable as resources.

On this NUC, the GPU is AMD (Vega M GH), so the operator deploys but has nothing to drive. Ollama falls back to CPU. The gpu-operator-resources namespace will exist but be idle. See Lessons Learned.

Deploying Ollama​

Why a StatefulSet?​

Ollama stores pulled models in /root/.ollama. A Deployment gives pods random names and doesn't guarantee stable storage attachment. A StatefulSet gives a stable pod name (ollama-0), ordered startup and shutdown, and a consistent binding between the pod and its PersistentVolumeClaim.

The Stack Manifest​

ollama-stack.yaml creates four resources in sequence:

1. The ollama namespace
2. A 50 Gi PVC (models are large — llama3.1:8b is ~5 GB; headroom matters)
3. The Ollama StatefulSet
4. A ClusterIP service on port 11434

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ollama
  namespace: ollama
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: ollama
          image: ollama/ollama:latest
          resources:
            limits:
              cpu: "4"
              memory: "16Gi"
            requests:
              cpu: "2"
              memory: "4Gi"
          volumeMounts:
            - name: ollama-volume
              mountPath: /root/.ollama

The memory limit of 16 Gi gives llama3.1:8b enough headroom to load the full model into RAM without competing with Open-WebUI for the remaining 16 Gi.

Model Preload with initContainer​

The first time Ollama starts, no models are pulled. If your app tries to chat before a model exists, it fails with a confusing error. ollama-automated.yaml solves this with an initContainer that pulls the model before the main container starts:

initContainers:
  - name: pull-model
    image: ollama/ollama:latest
    command: ["/bin/sh", "-c"]
    args:
      - |
        ollama serve &
        sleep 5
        ollama pull llama3.1:8b
        kill %1
    volumeMounts:
      - name: ollama-volume
        mountPath: /root/.ollama

The init container starts ollama serve in the background, waits for it to be ready, pulls the model, then stops. The main container finds the model already on disk and starts serving immediately.

Apply it:

kubectl apply -f backend/k8s-deployment/ollama-stack.yaml
# or, to pre-pull the model on first boot:
kubectl apply -f backend/k8s-deployment/ollama-automated.yaml

Deploying Open-WebUI​

Open-WebUI is a polished chat interface that connects directly to Ollama. open-webui-stack.yaml deploys it in the same ollama namespace:

env:
  - name: OLLAMA_BASE_URL
    value: http://ollama-service.ollama.svc.cluster.local:11434

The key line is OLLAMA_BASE_URL. It uses Kubernetes internal DNS (<service>.<namespace>.svc.cluster.local) to reach Ollama — no hard-coded IPs, no reliance on external networking. If the Ollama pod restarts and gets a new IP, the DNS name still resolves correctly.

A 10 Gi PVC stores chat history and Open-WebUI configuration.

kubectl apply -f backend/k8s-deployment/open-webui-stack.yaml

Once running, Open-WebUI is available at http://ai.local (after ingress is configured below).

MetalLB + Nginx Ingress​

Why MetalLB?​

In a cloud cluster, type: LoadBalancer services get a public IP automatically from the cloud provider. On bare metal, nothing assigns that IP — services stay in <pending>. MetalLB fills that gap by assigning IPs from a configured pool to LoadBalancer services on your local network.

ingress-lb.yaml creates a LoadBalancer service for the Nginx ingress controller:

kubectl apply -f backend/k8s-deployment/ingress-lb.yaml

MetalLB assigns 192.168.1.54 from the configured pool. Verify:

kubectl get svc -n ingress
# ingress-loadbalancer   LoadBalancer   ...   192.168.1.54   80:...,443:...

Ingress Rules​

Three ingress objects route hostnames to services:

kubectl apply -f backend/k8s-deployment/ollama-ingress.yaml
kubectl apply -f backend/k8s-deployment/open-webui-ingress.yaml
kubectl apply -f backend/k8s-deployment/dashboard-ingress.yaml

Client Hosts File​

Any machine that wants to reach these hostnames needs a single /etc/hosts entry pointing the names at the MetalLB IP:

192.168.1.54  dashboard.local ollama.local ai.local

On macOS: /etc/hosts. On Windows: C:\Windows\System32\drivers\etc\hosts.

After this, http://ai.local opens Open-WebUI in the browser, and http://ollama.local is the Ollama API endpoint.

Connecting the MCP App from Part 1​

The MCP app from Part 1 pointed at the bare-metal Ollama IP. Switching to the k8s deployment is a one-line .env change in both backend/.env and mcp-server/.env:

# Before (bare metal)
OLLAMA_HOST=http://192.168.1.80:11434

# After (k8s ingress)
OLLAMA_HOST=http://ollama.local
OLLAMA_MODEL=llama3.1:8b
OLLAMA_TIMEOUT=180000

The app now survives an Ollama pod restart without any manual intervention — the pod comes back up, the DNS name resolves, and the next request succeeds. The 3-minute timeout (180000 ms) accounts for the slower CPU-only inference on llama3.1:8b.

Verifying the Deployment​

Check everything is running:

kubectl get pods,svc,ingress -n ollama

Expected output:

NAME                             READY   STATUS    RESTARTS
pod/ollama-0                     1/1     Running   1
pod/open-webui-648d966b5-jsvfl   1/1     Running   2

NAME                         TYPE        CLUSTER-IP       PORT(S)
service/ollama-service       ClusterIP   10.152.183.132   11434/TCP
service/open-webui-service   ClusterIP   10.152.183.48    80/TCP

NAME                                    HOSTS        ADDRESS
ingress/ollama-ingress                  ollama.local 127.0.0.1
ingress/open-webui-ingress             ai.local     127.0.0.1

Test Ollama directly:

curl http://ollama.local/api/tags

Demo​

With everything running, Open-WebUI is available at http://ai.local — a full chat interface served entirely from the home lab, with no cloud API involved.

llama3.1:8b handling a practical question with a detailed, well-structured response. The model name and URL in the browser confirm it's running locally through the k8s ingress.

Lessons Learned​

The GPU assumption cost time. I installed the NVIDIA GPU operator early, assumed Ollama would detect the AMD Vega M and use it. It didn't — CUDA and ROCm are separate stacks. The Ollama container never saw the GPU. Lesson: check GPU vendor before planning acceleration. For AMD, ROCm support in Ollama requires a custom image and a different operator setup — a future post if I get there.

CPU inference is slower than expected on larger models. llama3.2:3b from Part 1 was snappy on CPU. Upgrading to llama3.1:8b dropped throughput noticeably — around 10–15 tokens/second. The 3B model was fine for quick tests; the 8B model is better quality but requires patience. The 3-minute timeout in the MCP app exists because of this.

The initContainer preload is worth the complexity. On first deploy without it, the Ollama pod starts, the Open-WebUI pod starts, a user immediately tries to chat, and gets a confusing "model not found" error. The initContainer adds maybe 5 minutes to first boot (model download) and eliminates that failure mode entirely.

StatefulSet vs Deployment matters less than the PVC. I spent time deliberating over this. The real thing that matters is the PVC binding — models must persist across restarts. StatefulSet makes the PVC binding explicit; a Deployment with a manual PVC binding would also work. StatefulSet is cleaner.

Ingress address shows 127.0.0.1 — that's normal. MicroK8s reports the ingress ADDRESS as 127.0.0.1 rather than the MetalLB IP. This confused me initially. The actual external IP lives on the LoadBalancer service in the ingress namespace, not on the ingress objects themselves.

What's Next​

The infrastructure is stable. The next steps are on the application side:

• Streaming responses — the MCP app waits for the full model completion before showing anything; streaming would feel much more interactive
• Containerise the app — the Node.js backend and MCP server still run locally on the dev machine; packaging them as containers and deploying to the same cluster would make the whole system self-contained
• AMD GPU support — ROCm-based Ollama on the Vega M is worth exploring; a 4 GB HBM2 GPU could significantly improve throughput even at reduced model precision

The full source, including all manifests, is on GitHub. Feedback and contributions welcome.

Practical use cases for Solution, Infrastructure & Enterprise Architects Featuring Claude™, GitHub Copilot™, and Gemini™ + Antigravity™

The rapid explosion of LLMs and AI tools to assist software engineers and architects is mind-boggling. Selecting the right tools for each use case can be a daunting task. I have been experimenting with three superpowers: Claude Code™, GitHub Copilot™ + VS Code, and Antigravity™ + Gemini™. Variations such as using Cursor with Claude™ can easily be derived from this.

This article also shares a GitHub repo that demonstrates some of the architecture use cases.

Table of Contents​

1. Three Superpowers​

AI Tools for Architects — Beyond Code Generation

ARCHITECTURE × ARTIFICIAL INTELLIGENCE

Three tools, differentiated by strength: Claude™ · GitHub Copilot™ · Gemini™ + Antigravity™

2. What Changed: The 2025–2026 Leap​

Agentic Capabilities​

All three tools now operate as agents — not just suggesting code, but autonomously planning, executing multi-step tasks, and iterating on their own output. Copilot has a coding agent that creates PRs from issues. Claude Code runs shell commands and edits files. Gemini Code Assist has full agent mode.

Massive Context Windows​

Claude supports 200K+ tokens. Gemini offers 1M+ tokens. This means entire codebases, not just files, can be analysed in a single session — transforming architecture understanding from guesswork to comprehensive analysis.

Built-in Security & Review​

Copilot now runs code scanning, secret scanning, and dependency checks inside its agent workflow. Gemini does automated PR reviews on GitHub. Claude's multi-agent system can reason about threat models holistically.

These capabilities move AI tools from 'coding assistants' to 'architecture collaborators'.

3. Three Tools, Three Architectural Superpowers​

Claude™ — The Architecture Reasoner​

• Deep reasoning + 200K context
• Claude Code: agentic CLI (shell + files)
• Multi-agent: sub-agents for parallel tasks
• TOGAF artefacts, ADRs, trade-off analysis
• Projects for persistent architecture context

Sample use case: Reverse-engineer codebase → Mermaid diagram + ADR

GitHub Copilot™ — The Agentic DevOps Partner​

• Coding agent: assign issues → auto-PRs
• Agent mode: multi-file edits + self-healing
• Built-in security scanning (code, secrets, deps)
• Multi-model: GPT, Claude, Gemini inside Copilot
• MCP integration for external tools + context

Sample use case: IaC scaffold + security scan + code explanation

Gemini™ + Antigravity™ — The Agent-First Cloud Architect​

• 1M+ token context (entire codebase analysis)
• Antigravity IDE: up to 5 parallel agents
• Native multimodal: diagrams + code + docs
• Built-in Chrome browser for visual verification
• Gemini 3.1 Pro: frontier reasoning + coding

Sample Use case: Upload diagram → component spec + GCP mapping

4. Architectural Use Cases at a Glance (2026)​

Key insight: the tools have converged in many areas. The differentiator is now where each tool lives — your terminal, your IDE, or your browser.

5. Case Study Claude · Architecture Reasoner​

What is done?​

1. Upload a microservices codebase to a Claude Project (persistent context across sessions)
2. Ask: "Reverse-engineer the architecture — identify services, dependencies, data flows, and concerns"
3. Claude produces a structured architecture narrative + Mermaid C4 diagram
4. Follow-up: "Generate an ADR for replacing sync REST with async event-driven messaging"
5. Result: Full MADR-format ADR with trade-offs, risks, and migration path

Prompts​

You are a senior solution architect.

Analyse this codebase and:
1. Identify all services and responsibilities
2. Map the dependency graph
3. Highlight architectural concerns
   (coupling, resilience, scalability)
4. Produce a Mermaid C4 context diagram
5. Suggest 3 improvement priorities

[PASTE CODE or use Project context]

Then generate an ADR (MADR format) for
migrating to async event-driven messaging.

Output: Architecture narrative · C4 Mermaid diagram · MADR ADR · Improvement priorities

6. Case Study 2 · GitHub Copilot · Agentic DevOps Partner​

Scenario A: Agent Mode — IaC + Security​

For Infrastructure / Solution Architects

1. Open VS Code → Agent mode in Copilot Edits
2. Prompt: "Create AKS cluster with private networking, AAD, and RBAC in Terraform"
3. Agent scaffolds multi-file Terraform config with self-healing
4. Ask Chat: "Scan for security vulnerabilities" — built-in code + secret + dependency scan runs automatically

Scenario B: Coding Agent — Issue to PR​

For Enterprise / Domain Architects

1. In GitHub, assign an issue to Copilot: "Add OpenTelemetry tracing to order service"
2. Coding agent researches the repo, plans implementation, creates branch
3. Agent self-reviews with Copilot Code Review before tagging you
4. Review the PR — security scanning already passed, iterate via PR comments

2026 update: Copilot now supports multi-model (GPT, Claude, Gemini), MCP integration, and runs in VS Code, JetBrains, Eclipse, and Xcode.

7. Case Study 3 · Gemini + Antigravity · Agent-First Cloud Architect​

What is done​

1. Upload a whiteboard/Visio architecture diagram to Gemini (browser or Antigravity IDE)
2. Ask: "Identify components, integration patterns, and data flows from this diagram"
3. Follow-up: "What concerns do you see? Generate a component spec."
4. In Antigravity: spawn parallel agents — one maps to GCP services, another writes IaC
5. Bonus: Paste codebase alongside diagram — 1M+ tokens cross-references both

Antigravity + Gemini: Why This Changes Architecture​

• 🚀 Antigravity: agent-first IDE (VS Code fork) with Manager View for up to 5 parallel agents
• 🖼️ Built-in Chrome browser: agents verify UI changes visually — no context-switch
• 📋 1M+ tokens: analyse entire repos + diagrams + docs in a single session
• ☁️ Native GCP integration — deploy, diagnose K8s, review Cloud Console from the IDE

Antigravity: free public preview. VS Code fork — your extensions carry over. Powered by Gemini 3.1 Pro + supports Claude Sonnet/Opus.

8. Elevated Use Cases: Docs, Security & TOGAF​

Architecture Decision Records — Claude (★★★)​

Describe the problem, constraints, and options. Claude outputs a MADR/RFC-format ADR with context, options, pros/cons, and rationale. Use Projects to maintain architectural context across multiple ADRs.

Security: Scanning + Threat Modelling — Copilot + Claude​

Copilot's agent now auto-runs code scanning, secret detection, and dependency checks. Claude generates STRIDE threat models from architecture descriptions. Gemini reviews PRs for policy violations.

TOGAF Business/Data/App/Tech Layers — Claude + Gemini​

Claude reasons about ArchiMate-aligned artefacts across all four TOGAF layers with mapping tables. Gemini adds value by processing existing architecture diagrams as visual input alongside requirements.

POC/SPIKE/MVP Rapid Build — All three (★★★)​

Copilot's coding agent builds features from issues. Claude Code scaffolds and tests full projects in your terminal. Gemini's agent mode builds inside the IDE. Assign the boring parts to AI, focus on design.

9. The AI-Augmented Architecture Workflow (2026)​

1️⃣ Discover​

Upload code + diagrams to Claude (Projects) or Gemini/Antigravity (1M context, parallel agents). Get architecture narrative and concern flags.

2️⃣ Design​

Generate options + ADRs with Claude. Scaffold IaC with Copilot Agent. Cross-reference diagrams in Gemini.

3️⃣ Build​

Assign issues to Copilot Coding Agent. Use Claude Code for terminal-based builds. Antigravity: parallel agents for multi-task builds.

4️⃣ Validate​

Copilot auto-scans for security. Claude generates threat models. Gemini reviews PRs. All before human review.

Guiding Principles​

• AI augments judgment, it doesn't replace it — always validate outputs against your architectural principles
• Combine tools: Gemini for intake, Claude for reasoning, Copilot for implementation and security
• Treat AI output as a first draft. Review like you'd review a junior architect's work — verify, refine, approve.

10. Case study output​

The full source is on GitHub. See the docs folder. Contributions and feedback welcome.

ADRs​

• ADR - Event Bus
• ADR - Schema Registry
• ADR - Kafka connect
• ADR - KSQL
• ADR - Producer
• ADR - Dashboard

Architecture and trade-off analysis​

• Architecture
• Trade-off Analysis

11. Start Today​

You don't need a new tool — you need a new reflex.

This Week​

• → Try Claude on one legacy codebase — ask it to explain the architecture and generate a Mermaid diagram
• → Use Copilot Agent Mode to scaffold a Terraform module with security review
• → Upload a whiteboard diagram to Gemini (or Antigravity) and ask it to generate a component spec

This Month​

• → Generate your first AI-assisted ADR for a real architectural decision using Claude Projects
• → Assign a low-complexity GitHub issue to Copilot's Coding Agent and review the PR it creates
• → Run a security review: Copilot scans code + Claude generates a STRIDE threat model

This Quarter​

• → Establish a team AI-augmented architecture workflow (Discover → Design → Build → Validate)
• → Document 3 trade-off analyses using AI as a first draft, then refine with team review
• → Share learnings — what worked, what hallucinated, what's not ready yet

Let's build something together.

Resources & Links​

Created in May 2026. Content reflects tool capabilities as of this date — these tools evolve rapidly.

Trademark notice: Claude, Claude Code, GitHub Copilot, GitHub, Gemini, Gemini Code Assist, Antigravity, and related product names are trademarks of their respective owners.

Table of Contents​

1. Document Purpose and Scope
2. Architectural Drivers
3. Use Case View (+1)
4. Logical View
5. Process View
6. Development View
7. Physical View
8. Architectural Decisions Summary
9. Risks and Technical Debt

1. Document Purpose and Scope​

1.1 Purpose​

This document describes the software architecture of the Chicago Transit Authority (CTA) Public Transport Optimisation System. It is structured according to the 4+1 Architectural View Model (Kruchten, IEEE Software 1995), which organises the architecture into five complementary views, each addressing the concerns of a different stakeholder group:

Diagrams use Mermaid syntax and follow ArchiMate 3.1 layering conventions:

• Technology Layer — infrastructure elements (brokers, databases, containers)
• Application Layer — software components and their interfaces
• Business Layer — business processes and actors that the system serves

1.2 System Overview​

The system is a real-time streaming pipeline that ingests simulated operational data from the CTA elevated rail network ("L"), processes it through multiple transformation stages, and presents a live transit status dashboard. It demonstrates a full Event-Driven Architecture (EDA) on the Confluent Kafka platform.

1.3 Scope​

• Three train lines: Blue, Red, Green (each with 10 trains, bidirectional)
• Station arrival events, turnstile ridership counts, and weather telemetry
• Static station reference data from PostgreSQL
• A browser-accessible real-time status dashboard

2. Architectural Drivers​

2.1 Quality Attribute Requirements​

2.2 Constraints​

• Python-only application code (no JVM services authored in-house)
• Single-host Docker Compose deployment (development / demonstration environment)
• Confluent Platform 5.2.2 (fixed version)

3. Use Case View (+1)​

The Use Case View captures the key scenarios that motivated and validate the architectural decisions. In the 4+1 model this view acts as the glue — each scenario exercises a slice through every other view.

3.1 Actor Diagram​

3.2 Key Scenarios​

UC-01 — View Live Transit Status​

Trigger: Transit operator opens http://localhost:8888 Flow: Tornado serves status.html populated from in-memory Lines and Weather state that is continuously updated by four Kafka consumers running as async coroutines. Architectural relevance: Drives the Tornado async server choice (ADR-006) and the requirement for in-process Kafka consumer coroutines.

UC-02 — Publish Train Arrival Event​

Trigger: Simulation time step advances; a train moves to the next station. Flow: Station.run() → AvroProducer.produce() → Schema Registry validates Avro → Kafka topic org.chicago.cta.station.arrivals.t001 → KafkaConsumer in server → Lines.process_message() → UI state updated. Architectural relevance: Establishes the end-to-end Kafka + Avro pipeline (ADR-001, ADR-002).

UC-04 — Publish Weather Reading​

Trigger: Simulation hour boundary. Flow: Weather.run() → HTTP POST to Kafka REST Proxy → Kafka topic org.chicago.cta.weather.v1 → KafkaConsumer in server → Weather.process_message(). Architectural relevance: Demonstrates the REST Proxy integration path (ADR-005).

UC-06 — Aggregate Rider Counts​

Trigger: Continuous turnstile events on com.cta.stations.turnstile.entry. Flow: KSQL turnstile table materialises from topic → KSQL TURNSTILE_SUMMARY GROUP BY aggregation → new Kafka topic → KafkaConsumer (is_avro=False) in server → UI ridership count. Architectural relevance: Drives the KSQL aggregation decision (ADR-004).

UC-07 — Transform Station Schema​

Trigger: Kafka Connect pushes a raw station row to com.cta.stations.data.rawt001.stations. Flow: Faust transform_stations agent reads record → resolves red/blue/green booleans to line string → writes TransformedStation to org.chicago.cta.stations.table.v1t001 and updates Faust in-memory table. Architectural relevance: Drives the Faust stream processor choice (ADR-004).

4. Logical View​

The Logical View describes the system's functional decomposition into key abstractions, their responsibilities, and their relationships. This view follows ArchiMate's Application Layer notation.

4.1 Component Overview​

4.2 Key Abstractions​

Producer Hierarchy​

Consumer / Model Hierarchy​

4.3 Kafka Topic Catalogue​

5. Process View​

The Process View describes the system's dynamic behaviour — how processes start, how data flows between them at runtime, and how concurrency is managed.

5.1 System Startup Sequence​

The diagram below shows the mandatory startup order. Components further right depend on components to their left being fully initialised.

5.2 End-to-End Data Flow — Train Arrival​

5.3 End-to-End Data Flow — Turnstile Aggregation​

5.4 Concurrency Model​

The entire consumer application runs in a single OS thread using cooperative multitasking. Kafka polling is non-blocking (0.1 s timeout). The HTTP handler is synchronous but executes between coroutine yield points, keeping UI latency low.

6. Development View​

The Development View describes the organisation of the software in the development environment — module structure, package dependencies, and build artefacts.

6.1 Module Structure​

6.2 Package Dependencies​

6.3 Entry Points and Startup Commands​

Note: Processes 2, 3, and 4 have an implicit startup ordering dependency. The Kafka Connect JDBC connector (configured by the simulation) must produce station data before the Faust app can transform it; the KSQL tables must exist before the dashboard starts. There is no orchestration script enforcing this order.

7. Physical View​

The Physical View maps software components onto physical (or virtualised) infrastructure. This view follows ArchiMate's Technology Layer.

7.1 Container Deployment Diagram​

7.2 Network Port Map​

7.3 Data Persistence Boundary​

All in-process state is rebuilt from Kafka on restart. Durable state exists only in PostgreSQL (station reference data) and the Kafka topic logs.

8. Architectural Decisions Summary​

Cross-reference to the detailed ADR documents in docs/adr/.

9. Risks and Technical Debt​

9.1 Risks​

9.2 Technical Debt​

Document generated by reverse-engineering the source code on 2026-03-12. All diagrams use Mermaid and render natively on GitHub.

Table of Contents​

1. Evaluation Framework
2. Decision Trade-off Analysis
   • 2.1 Event Bus — Kafka vs Alternatives
   • 2.2 Serialisation — Avro vs Alternatives
   • 2.3 DB Ingestion — Kafka Connect vs Custom Producer
   • 2.4 Stream Processing — Faust + KSQL vs Alternatives
   • 2.5 Weather Producer — REST Proxy vs Native Client
   • 2.6 Dashboard Server — Tornado vs Alternatives
3. Cross-Cutting Trade-off Analysis
   • 3.1 Serialisation Consistency
   • 3.2 State Management Strategy
   • 3.3 Concurrency Model
   • 3.4 Operational Complexity
4. Architecture Fitness Function
5. Strategic Recommendations
6. Trade-off Summary Heatmap

1. Evaluation Framework​

Every trade-off is scored against the six quality attributes (QAs) derived from the system's architectural drivers (see architecture.md §2).

1.1 Quality Attribute Weights​

1.2 Scoring Scale​

1.3 Risk Scale​

2. Decision Trade-off Analysis​

2.1 Event Bus — Kafka vs Alternatives​

Decision: ADR-001 — Apache Kafka as the single event bus

Weighted Scoring Matrix​

Positioning​

Trade-off Narrative​

Why Kafka wins here: Kafka's append-only, partitioned log is the single feature that unlocks replayability and fan-out simultaneously — properties that no queue-based broker (RabbitMQ) provides out of the box. The ability to start a new consumer at offset_earliest and rebuild the full station/weather state is architecturally critical for the dashboard's cold-start scenario.

What is sacrificed:

• Operational simplicity. Kafka requires Zookeeper (in CP 5.x), Schema Registry, and REST Proxy as satellites. A RabbitMQ cluster is simpler to operate.
• Latency at p99. Kafka batches records before acknowledgment; for the weather producer that posts once per simulated hour, this is irrelevant, but it rules Kafka out for sub-millisecond latency use cases.

Key risk introduced: 🔴 Single-broker deployment with replication_factor=1. In production, broker failure loses all un-replicated messages.

2.2 Serialisation — Avro vs Alternatives​

Decision: ADR-002 — Apache Avro + Confluent Schema Registry

Weighted Scoring Matrix​

Trade-off Narrative​

Why Avro wins: The Schema Registry's compatibility check acts as a compile-time equivalent at publish-time — a field removal or rename is rejected before a single consumer can be broken. Avro's wire format embeds only the schema ID (4 bytes), making messages far more compact than equivalent JSON.

Protobuf is the credible alternative: Protobuf achieves nearly identical scores. The differentiator is ecosystem fit: confluent-kafka-python's AvroProducer/AvroConsumer were the idiomatic Python Confluent API at CP 5.2.2, whereas Protobuf support required more boilerplate. Today (Confluent Platform 7+), Protobuf is first-class; migrating would be viable.

Key inconsistency introduced: 🟠 TURNSTILE_SUMMARY uses JSON while all other topics use Avro. This forces consumers to branch on is_avro and removes schema-enforcement for rider counts — the metric that most directly feeds the UI.

2.3 DB Ingestion — Kafka Connect vs Custom Producer​

Decision: ADR-003 — Kafka Connect JDBC Source Connector

Weighted Scoring Matrix​

Trade-off Narrative​

Why Kafka Connect wins over a custom producer: Zero-code ingestion eliminates an entire class of bugs: offset tracking, error handling, and retry logic are handled by a battle-tested framework. The connector is idempotent — safe to re-run on simulation restart.

Why Debezium CDC scores higher but was rejected: Debezium captures every INSERT/UPDATE/DELETE via PostgreSQL Write-Ahead Log, which is more correct (would capture station updates, not just inserts). However, enabling WAL replication requires DBA-level PostgreSQL configuration (wal_level=logical), which is overkill when the stations table is quasi-static reference data loaded once from CSV.

Hidden cost of the chosen approach: 🟡 mode=incrementing only detects new rows by monotonically increasing stop_id. A station name correction or line reassignment will silently remain stale in the Kafka topic and in the dashboard until the connector is manually reset and replayed.

When the decision should be revisited​

If station data becomes writable (e.g. an admin UI for updating station names), migrate to Debezium CDC to capture UPDATE and DELETE events.

2.4 Stream Processing — Faust + KSQL vs Alternatives​

Decision: ADR-004 — Faust for record transformation + KSQL for aggregation

Option Space​

Weighted Scoring Matrix​

Trade-off Narrative​

The dual-engine pattern is a deliberate pedagogical trade-off: The use of two tools increases operational complexity (two processes, two different programming models) but each tool is used where it excels:

Key risk: 🟡 Faust's store="memory://" means station state is rebuilt from the full topic on every restart. As the station topic grows this adds startup latency. Replace with store="rocksdb://" for a persistent local state store.

Operational debt: 🟠 No orchestration enforces the startup order:

1. Kafka Connect must publish station data
2. Faust must transform it
3. KSQL must create TURNSTILE_SUMMARY
4. Only then can the dashboard start

A failure anywhere in this chain requires manual intervention.

2.5 Weather Producer — REST Proxy vs Native Client​

Decision: ADR-005 — Kafka REST Proxy for the Weather producer

Weighted Scoring Matrix​

Trade-off Narrative​

This decision scores lowest of all six because there is no functional reason to diverge from the native client — only demonstration value.

Verdict: 🟡 The REST Proxy choice adds cognitive overhead for no functional gain in a Python-only system. If the goal is demonstration, the inconsistency should be documented clearly (it now is in ADR-005). For a production system, weather should use AvroProducer like every other producer, and the REST Proxy demo should be a separate isolated example.

2.6 Dashboard Server — Tornado vs Alternatives​

Decision: ADR-006 — Tornado async web server

Weighted Scoring Matrix​

Positioning​

Trade-off Narrative​

Why Tornado is a reasonable but not optimal choice: Tornado's IOLoop integrates naturally with confluent_kafka's callback-based API and was the idiomatic async web server in the Python ecosystem before asyncio matured. The chosen design — spawn_callback for consumers, synchronous GET handler — achieves the goal with minimal code.

aiohttp / FastAPI score higher today because:

• Both are built natively on asyncio (no legacy compatibility shim)
• FastAPI provides automatic OpenAPI documentation
• The aiokafka library provides a fully async consumer compatible with both

Flask's fatal flaw in this context: A synchronous web server cannot co-locate Kafka consumer polling in the same process without threads. Using threads reintroduces shared-state locking complexity that the async model eliminates.

Key risk: 🟡 All four consumers share a single Kafka group.id. Starting a second dashboard instance would split partition ownership, causing each instance to see only a subset of events — producing an incoherent UI state.

3. Cross-Cutting Trade-off Analysis​

3.1 Serialisation Consistency​

The system uses two serialisation formats across its six topics:

Recommendation: Register an Avro schema for TURNSTILE_SUMMARY and change VALUE_FORMAT='AVRO' in the KSQL CREATE TABLE statement. This removes the is_avro=False branch from the consumer and makes the serialisation model uniform.

3.2 State Management Strategy​

The system employs three distinct state-management patterns with different durability guarantees:

Structural tension: The design makes all in-process state reconstruct-able from Kafka, which is elegant and correct. However, it assumes the Kafka logs are themselves durable — an assumption violated by replication_factor=1.

3.3 Concurrency Model​

Three different concurrency approaches coexist across the system:

Scale-out constraint for the dashboard:

Fix: Each dashboard instance should use a unique group.id (e.g. append a UUID suffix) so every instance receives the full partition set and sees all events.

3.4 Operational Complexity​

The system requires 7 infrastructure containers + 4 Python processes to be started in a specific order:

Total components: 11

Operational risk: 🟠 There is no automated readiness check or restart policy for the Python processes. A crash at any layer requires manual diagnosis and ordered restart.

Mitigation options:

4. Architecture Fitness Function​

A fitness function scores how well the as-built architecture meets each quality attribute.

Scale: ████████░░ = partially met   ██████████ = fully met   ████░░░░░░ = significantly unmet

Overall fitness: 4.0 / 5.0 (80 %) — suitable for demonstration, not production

5. Strategic Recommendations​

Prioritised by risk reduction value vs implementation effort:

Priority 1 — Critical (address before any production use)​

Priority 2 — High (address in first production sprint)​

Priority 3 — Medium (address in backlog)​

6. Trade-off Summary Heatmap​

The heatmap below shows the contribution of each architectural decision (rows) to each quality attribute (columns). Green = positive contribution; Red = negative contribution.

Key:

• 🟢🟢 Strong positive (+2)
• 🟢 Positive (+1)
• 🟡 Neutral (0)
• 🔴 Negative (−1)
• 🔴🔴 Strong negative (−2)

Overall Assessment​

The core architectural spine — Kafka + Avro + Schema Registry + Kafka Connect — scores highly and is well-suited to the problem. The gaps are concentrated in two areas:

1. Operational resilience (single broker, no startup orchestration)
2. Serialisation consistency (KSQL JSON bypass undermines the otherwise strong Avro contract)

Addressing the P1 recommendations above would raise the overall fitness score from 4.0 / 5.0 → ~4.6 / 5.0, making the architecture production-worthy.

Trade-off analysis generated by reverse-engineering source code as of 2026-03-12. Weighted scores are analytical judgements based on code evidence, not empirical benchmarks.

Context​

The CTA (Chicago Transit Authority) public transport optimisation system must ingest and distribute high-frequency, heterogeneous events from multiple sources:

• Train arrivals at every station on three colour lines (Blue, Red, Green), each carrying 10 trains
• Turnstile entry counts produced per time-step at every station
• Hourly weather readings
• Static station reference data held in a relational database

A naive polling or REST-request-per-event approach would not scale to the volume, would couple producers tightly to consumers, and would make it difficult to replay or replay events for new consumers.

Decision​

Apache Kafka (Confluent Platform 5.2.2) is used as the single, central event streaming backbone. All data flows in and out of Kafka topics; no service communicates directly with another.

Evidence from code:

Topics are created with LZ4 compression, a short delete-retention window (2 s) suitable for real-time transit dashboards, and 10 partitions by default for arrival topics (producers/models/producer.py:18-32).

Alternatives Considered​

Consequences​

Positive

• Producers and consumers are fully decoupled; new consumers (e.g. analytics) can subscribe independently without touching producers.
• Log retention enables late-joining consumers to replay from the earliest offset (offset_earliest=True in consumers/server.py:73-91).
• Kafka's partition model provides horizontal scale-out for high-throughput arrival events.

Negative / Risks

• Single-broker setup (kafka0 in docker-compose.yaml) is a single point of failure for local development; production would require at minimum 3 brokers.
• Replication factor is set to 1 throughout — data loss on broker failure.
• Hard-coded localhost:9092 in producer bootstrap config (producers/models/producer.py:66) couples the code to the local Docker environment.

Context​

Multiple independent processes — producers written in Python and stream processors written with Faust and KSQL — exchange messages over Kafka. Without a shared, versioned contract, a schema change in a producer silently breaks downstream consumers. The system needs:

1. A machine-readable schema for every message type.
2. A registry that enforces backward/forward compatibility on publish.
3. Consumers that can deserialise messages without embedding the schema in every message.

Decision​

Apache Avro is the default serialisation format for first-party Kafka topics, with Confluent Schema Registry (port 8081) acting as the central schema store. Python producers that use the shared producer base class publish Avro-encoded messages via AvroProducer, and consumers use Avro deserialisation for those Avro-backed topics via AvroConsumer from confluent-kafka-python.

There are explicit exceptions to that default path. Weather data is produced via the REST Proxy (producers/models/weather.py) rather than through AvroProducer. The dashboard also consumes some JSON topics with is_avro=False in consumers/server.py, including the stations table and the TURNSTILE_SUMMARY topic. Schema files are stored as JSON alongside the producer models:

producers/models/schemas/
  arrival_key.json
  arrival_value.json
  turnstile_key.json
  turnstile_value.json
  weather_key.json
  weather_value.json

Representative schema (arrival_value.json):

{
  "namespace": "com.udacity",
  "type": "record",
  "name": "arrival.value",
  "fields": [
    {"name": "station_id",       "type": "int"},
    {"name": "train_id",         "type": "string"},
    {"name": "direction",        "type": "string"},
    {"name": "line",             "type": ["null","string"]},
    {"name": "train_status",     "type": ["null","string"]},
    {"name": "prev_station_id",  "type": ["null","int"]},
    {"name": "prev_direction",   "type": ["null","string"]}
  ]
}

The producer base class wires schemas at construction time (producers/models/producer.py:75-77):

self.avroProducer = AvroProducer(
    {"bootstrap.servers": "...", "schema.registry.url": "http://localhost:8081"},
    default_key_schema=self.key_schema, default_value_schema=self.value_schema)

The exception is the TURNSTILE_SUMMARY topic produced by KSQL, which uses JSON encoding (VALUE_FORMAT='JSON') and is consumed without Avro deserialisation (consumers/server.py:87-91).

Alternatives Considered​

Consequences​

Positive

• Schema Registry enforces compatibility before messages are published.
• Schema IDs are embedded in the Avro wire format — consumers can always retrieve the exact schema used to write a message.
• Faust's faust.Record dataclasses mirror the Avro schema structure, making the contract explicit in both the registry and the Python type system (consumers/faust_stream.py:14-33).

Negative / Risks

• AvroProducer is marked as a legacy API in newer Confluent SDK versions; migration to SerializingProducer with AvroSerializer will be needed.
• The KSQL TURNSTILE_SUMMARY topic diverges from the Avro convention (uses JSON), creating an inconsistency that consumers must handle explicitly (is_avro=False).
• Schema files live inside producers/ only; the consumer side has no local copy, creating a coupling between producer deployment and consumer startup.

Context​

Station reference data (stop IDs, names, line membership, ordering) is stored in a PostgreSQL table (stations) seeded from a CSV file at container start-up (load_stations.sql). The consumer side needs this data in Kafka so that stream processors (Faust) can enrich and transform it alongside real-time event streams.

Two ingestion options were on the table:

1. Write a bespoke Python producer that reads from the database and publishes to Kafka.
2. Use a managed connector that understands JDBC semantics.

Decision​

Confluent Kafka Connect with the JdbcSourceConnector is used to stream the stations table from PostgreSQL into Kafka automatically.

The connector is configured programmatically at simulation start-up via the Kafka Connect REST API (producers/connector.py:16-57):

The connector is idempotent — if it already exists the setup function returns early (producers/connector.py:19-22).

Faust then reads from the output topic com.cta.stations.data.rawt001.stations (consumers/faust_stream.py:40).

Alternatives Considered​

Consequences​

Positive

• Zero custom ingestion code; the connector handles polling, batching, and offset management.
• Decouples the database schema from producer code — schema changes propagate via the connector.
• New consumers of station data subscribe to the Kafka topic without touching the database.

Negative / Risks

• incrementing mode only detects inserts, not updates or deletes; stale station data will not be corrected unless the connector is reset.
• Hard-coded credentials (cta_admin / chicago) in connector.py:43-44 must be externalised for any non-local environment.
• The connector is registered once at simulation start; a crash before registration completes leaves no station data in Kafka.

Context​

Two distinct stream-processing requirements exist:

1. Station enrichment — raw station rows arriving from the JDBC connector carry boolean red/blue/green columns. A downstream topic is needed that replaces these booleans with a single line string field and retains only the fields required by the UI model.

2. Turnstile aggregation — individual turnstile-entry events must be aggregated into a count per station so the dashboard can display a single rider-count per station rather than a stream of raw entry records.

These two problems have different shapes: the first is a stateless record-by-record transformation; the second is a stateful GROUP BY aggregation.

Decision​

Two separate stream-processing tools are used, each chosen for its natural fit with one problem:

Faust — station transformation (consumers/faust_stream.py)​

Faust is a Python-native stream-processing library. It is used to:

• Subscribe to com.cta.stations.data.rawt001.stations
• Produce TransformedStation records to org.chicago.cta.stations.table.v1t001
• Maintain an in-memory Faust Table as a materialised view keyed by station_id

@app.agent(in_topic)
async def transform_stations(in_stations):
    async for sn in in_stations:
        t = TransformedStation(sn.station_id, sn.station_name, sn.order, "na")
        if sn.red:   t.line = "red"
        elif sn.blue: t.line = "blue"
        elif sn.green: t.line = "green"
        else: continue
        table[sn.station_id] = t

KSQL — turnstile aggregation (consumers/ksql.py)​

KSQL (now ksqlDB) is used to express a SQL aggregation over the turnstile topic:

CREATE TABLE turnstile ( ... )
WITH (KAFKA_TOPIC='com.cta.stations.turnstile.entry', VALUE_FORMAT='AVRO', KEY='station_id');

CREATE TABLE TURNSTILE_SUMMARY WITH (VALUE_FORMAT='JSON') AS
    SELECT station_id, count(station_id) as COUNT
    FROM turnstile GROUP BY station_id;

The KSQL statement is submitted via the KSQL REST API at consumer start-up and is idempotent — it is skipped if TURNSTILE_SUMMARY already exists.

Alternatives Considered​

Consequences​

Positive

• Each tool is used for its core strength: Faust for Python-idiomatic record transformation, KSQL for declarative aggregation.
• The Faust app and KSQL statements are independently deployable and restartable.

Negative / Risks

• Two different stream-processing runtimes increase operational surface area (two separate processes to start, monitor, and upgrade).
• The Faust Table uses store="memory://" — state is lost on restart; the table is rebuilt from Kafka on each startup, which adds startup latency.
• KSQL's output (TURNSTILE_SUMMARY) uses JSON while all other topics use Avro, creating a serialisation inconsistency (see ADR-002).
• The consumers/server.py startup guard (topic_check) blocks the dashboard if either Faust or KSQL has not yet produced its output topic, creating an implicit startup ordering dependency.

Context​

The weather simulation model (producers/models/weather.py) needs to publish Avro-encoded records to Kafka. All other producers in the system use the confluent-kafka Python library's AvroProducer directly against the broker.

During development, a second integration path was explored: the Confluent Kafka REST Proxy (port 8082), which accepts HTTP POST requests with embedded schemas and records.

Decision​

The Weather producer uses the Kafka REST Proxy instead of a native Kafka producer client.

Implementation (producers/models/weather.py:71-86):

resp = requests.post(
    f"{constants.Constants.rest_proxy_url}topics/{constants.Constants.weather_topic_name}",
    headers={"Content-Type": "application/vnd.kafka.avro.v2+json"},
    data=json.dumps({
        "key_schema":   json.dumps(Weather.key_schema),
        "value_schema": json.dumps(Weather.value_schema),
        "records": [{"value": {...}, "key": {"timestamp": self.time_millis()}}]
    }),
)

Schema JSON is inlined in every POST payload rather than being pre-registered with the Schema Registry. The REST Proxy handles registration transparently.

Alternatives Considered​

Consequences​

Positive

• Demonstrates an HTTP-based integration path useful for polyglot producers (languages without a native Kafka client library).
• No Kafka client dependency required in the producing service.

Negative / Risks

• An additional network hop (producer → REST Proxy → broker) adds latency compared to the native client path.
• Inlining the full schema JSON in every request is wasteful; the Schema Registry already holds the schema after the first publish.
• Error handling on HTTP failures is minimal — a failed raise_for_status() logs the error but drops the weather event silently.
• Inconsistency: weather uses REST Proxy while all other producers use the native client, increasing cognitive overhead for maintainers.

Context​

The consumer layer must simultaneously:

1. Poll four Kafka topics continuously and update in-memory state (weather, line status, arrivals, turnstile counts).
2. Serve HTTP GET requests that render the current state as an HTML page.

A blocking web server would stall Kafka consumption while handling HTTP requests. A blocking Kafka consumer would stall HTTP responses while waiting for new messages.

Decision​

The Tornado asynchronous web framework is used as the server runtime (consumers/server.py). Kafka consumers are scheduled as Tornado IO loop callbacks:

for consumer in consumers:
    tornado.ioloop.IOLoop.current().spawn_callback(consumer.consume)
tornado.ioloop.IOLoop.current().start()

Each KafkaConsumer.consume() is an async coroutine that yields control between polls via await gen.sleep(self.sleep_secs) (consumers/consumer.py:70-76). The HTTP handler renders state synchronously on GET without blocking Kafka consumption.

Four consumers are registered on startup:

All consumers share a single group.id (com.chicago.transport.consumer.group.1).

The server listens on port 8888 and serves a single route (/) rendered from consumers/templates/status.html.

Alternatives Considered​

Consequences​

Positive

• Single process handles both Kafka consumption and HTTP serving without threading.
• Tornado's spawn_callback allows an arbitrary number of consumers to coexist on one event loop.
• Simple HTML template rendering — no JavaScript framework needed for the status page.

Negative / Risks

• State is stored in Python objects (Weather, Lines) in process memory; any restart loses accumulated state until topics are re-consumed from the earliest offset.
• All four consumers share one group.id, meaning if a second dashboard instance were started it would steal partitions from the first.
• The dashboard blocks entirely during startup if KSQL or Faust topics are not yet ready (hard exit(1) at consumers/server.py:49-57), requiring a manual restart order.
• No authentication or HTTPS on port 8888.

Index​

System Architecture Overview​

┌─────────────────────────────────────────────────────────────────────┐
│                        PRODUCER LAYER                               │
│                                                                     │
│  simulation.py                                                      │
│    ├─ Line (Blue/Red/Green)                                         │
│    │    ├─ Station ──────────────────► org.chicago.cta.station.     │
│    │    │   (AvroProducer)              arrivals.t001               │
│    │    └─ Turnstile ─────────────────► com.cta.stations.           │
│    │        (AvroProducer)               turnstile.entry            │
│    └─ Weather ──────────────────────►  org.chicago.cta.weather.v1  │
│         (REST Proxy HTTP POST)                                      │
│                                                                     │
│  connector.py ──[JDBC Source]──► com.cta.stations.data.rawt001.    │
│  (Kafka Connect)                  stations  (from PostgreSQL)       │
└────────────────────────────┬────────────────────────────────────────┘
                             │  Apache Kafka  (+ Schema Registry)
┌────────────────────────────▼────────────────────────────────────────┐
│                    STREAM PROCESSING LAYER                          │
│                                                                     │
│  faust_stream.py                                                    │
│    com.cta.stations.data.rawt001.stations                           │
│      ──[transform]──► org.chicago.cta.stations.table.v1t001        │
│                                                                     │
│  ksql.py                                                            │
│    com.cta.stations.turnstile.entry                                 │
│      ──[GROUP BY station_id]──► TURNSTILE_SUMMARY (JSON)            │
└────────────────────────────┬────────────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────────────┐
│                      CONSUMER / DASHBOARD LAYER                     │
│                                                                     │
│  server.py  (Tornado, port 8888)                                    │
│    KafkaConsumer × 4  ──► in-memory Weather + Lines state           │
│    GET /  ──► status.html                                           │
└─────────────────────────────────────────────────────────────────────┘