A fundamental pivot is taking place in incident response engineering: the primary obstacle in AI-assisted root cause analysis (RCA) is no longer the reasoning capacity of large language models, but rather the data harness that feeds them. Observability specialists increasingly agree that the real challenge lies within the pipeline responsible for determining which telemetry reaches the model and how it is structured.
For engineering teams integrating LLMs into automated troubleshooting, this shift brings a concrete operational takeaway. Investing effort into refining context preparation now yields far greater dividends than simply upgrading to a larger or more expensive model.
Today, AI-driven RCA systems predominantly split into two distinct architectural camps. Agent-based designs grant the model tools to dynamically query telemetry during an ongoing investigation, whereas deterministic frameworks correlate telemetry upstream and supply the LLM with a single, highly curated context. Platforms like Dynatrace with its Davis AI rely heavily on deterministic, topology-based causal analysis – navigating real-time dependency maps to isolate issues rather than releasing an agent into an unconstrained diagnostic loop. Because both methods fail under different circumstances, determining whether an incorrect diagnosis stems from weak model reasoning or a flawed context harness has historically proven difficult.
To disentangle these variables, engineer Nikolay Sivko from observability vendor Coroot conducted original research dividing AI RCA into two independent functions: data reasoning and context selection. He argued that asking whether “AI can perform RCA” is inherently flawed, as the underlying model and the surrounding harness must be evaluated separately.
Sivko tested this separation by building a deterministic pipeline that correlates system signals into findings before handing them to an LLM in a single, focused context without an agent loop – ensuring any diagnostic error rests solely on the model. In his benchmark, a Chaos Mesh NetworkChaos experiment introduced latency between a catalog service and its Postgres database, inducing slow queries and triggering 502 errors on the front-end. Despite misleading signals mixed into the 9,800-token prompt – such as query times inflated by network round-trip delays – top-tier closed models including Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro successfully pinpointed the root cause and identified the need to delete the experiment along with its schedule. Among open-weight models, Gemma 4 31B stood out as the only self-hostable option to correctly diagnose the issue, while larger models like Qwen3.6 35B and Qwen3 Coder Next failed.
While these results highlight the power of context engineering, they do not entirely eliminate the case for agentic systems. Models that fetch their own telemetry can surface unexpected anomalies that fixed correlation pipelines might overlook, offering an edge during novel or unprecedented outages. However, this adaptability comes at a steep operational price. Experiences shared by teams at ZenML and Incident.io indicate that multi-agent investigations are exceptionally difficult to debug in production because failure runs produce emergent, unpredictable interactions instead of clean stack traces. Driven by these friction points, many developers discussing the trade-off on Reddit report abandoning fully agentic setups in favor of deterministic pipelines with narrow LLM intervention to gain predictability and reduce token expenditure.
Financial costs further reinforce this transition, as Sivko noted that executing a single concise prompt costs only a few cents even on frontier models when heavy correlation happens beforehand. His evaluation concluded that the reasoning phase of AI RCA is essentially solved, redirecting technical focus toward assembling compact, high-signal context prior to calling the model. This perspective aligns with broader industry standards championed by Anthropic, LangChain, and observability provider Mezmo, which all emphasize context curation as a core engineering discipline for reliable AI reasoning.
