MCP in the Wild: Real Time Agentic Systems with Live A/B Testing

Arun Gupta’s session walks you through the Multi-Context Protocol (MCP), which breaks agent behavior into bite-sized modules—planning, reasoning, execution—and wires them into a real-time Python/FastAPI/CrewAI multiplayer game. The kicker? You can hot-swap LLMs (GPT-4, Claude, Mixtral) on the fly and see how each one affects your agents, all without touching a line of core code.

On top of that, you’ll learn how to run live A/B tests mid-game, capturing structured MCP metrics like planning time, replanning frequency, and context-switch impact. Armed with these insights and the open-source repo, you’ll be able to benchmark LLM performance beyond just latency or token count—and keep your multi-agent systems neat, modular, and spaghetti-free.

Watch on YouTube