Every detection program in production today runs on the same basic loop. A detection engineer studies a threat, translates it into rule logic, and deploys it to a SIEM or detection platform. That rule sits in the pipeline, evaluating events as they flow through, waiting for a match. When the conditions are met, it fires an alert. That alert lands in a queue, and someone picks it up, investigates, and makes a judgment call on whether to close it or escalate it. Then the next alert comes in, and the cycle repeats.

This model has become more sophisticated over the years. Detection-as-code brought version control, testing, and CI/CD to rule management. Behavioral signals replaced some of the noisiest atomic alerts with entity-level risk scoring. AI agents now handle the triage step faster and more consistently than most human analysts can at volume. But the fundamental structure hasn’t yet changed, and detection is still a human-authored, queue-driven process.

Coverage in a rule-based model is bounded by two things: what your detection engineers know to look for, and how much time they have to write and maintain rules for it. Threat hunting exists precisely because teams understand their rules don’t cover everything. Hunters go looking for things the rules missed, patterns that haven’t been codified, behaviors that are too nuanced or too novel for static logic. But hunting is expensive. It requires dedicated time and access to broad datasets and intelligence. Most teams run hunts infrequently, and many don’t sustain a formal hunting program at all.

The industry has spent years optimizing each step in this chain individually: better rules, faster triage, richer enrichment, smarter correlation. What it hasn’t done is question whether the chain itself is the right model when agents are capable of something fundamentally different. In this post, we’ll look at what happens when agents stop waiting for rules to fire and start hunting continuously on their own, how that changes the relationship between detection engineering and threat hunting, and why we’re still early in a transition that will reshape how SOCs generate and act on security findings.

Subscribe now

What Changes When Agents Continuously Hunt

An agent with access to a security data lake, enrichment sources, and threat intelligence doesn’t need someone to anticipate a threat before it can look for one. It can form a hypothesis, query the data, evaluate what it finds, and surface a structured finding, all without a rule telling it what to look for.

These building blocks already exist. Agents can now query log data, pull enrichments from identity providers and asset inventories, and cross-reference threat intelligence feeds to contextualize activities. Data lakes built on formats like Iceberg make months or years of security telemetry queryable in minutes. The agent doesn’t need a detection engineer to write a rule for “unusual cross-account role assumption patterns in AWS”; it can investigate that class of behavior directly by querying CloudTrail, correlating with identity context, and assessing whether the pattern deviates from the environment’s baseline.

What makes this different from existing threat-hunting workflows is its duration and autonomy. A human-led hunt is a project: someone defines a hypothesis, allocates time, runs queries, documents findings, and moves on. It happens once, and then it’s over until the next hunt. An agent-led hunt can run perpetually. The same investigative logic that a senior analyst would apply during a focused hunt can execute on a rolling basis, scanning for patterns across log sources, adjusting based on what it finds, and surfacing results as they emerge. The concept of a “hunt” as a discrete event starts to dissolve when the hunter never stops looking. Continuous agent hunting is the practice of deploying AI agents to independently scan security telemetry on a rolling basis, forming and testing hypotheses without waiting for a pre-authored detection rule to trigger. It collapses the distinction between “detection” and “threat hunting” into a single, ongoing process.

This changes the role of detections in the overall security architecture. Rules don’t go away, but they no longer serve as the sole source of signal in the SOC. The alert queue gets populated by two streams: deterministic alerts from rules that matched known-bad patterns, and probabilistic findings from agents that noticed something worth investigating. The implication is that coverage is no longer gated solely by what your team had time to write rules for. The spaces between your rules, the gaps that today only get examined during periodic hunts, become continuously monitored territory.

From Alert Queues to Continuous Findings

In an agent-led SOC model, the alert queue still exists, but what populates it changes. Alongside the familiar rule-based alerts, agents surface findings from their own continuous analysis. The output looks similar to an alert (entities, evidence, confidence, next steps), but its origin is fundamentally different: it came from reasoning over data rather than a pattern match against a rule. These findings should be routed through the same case management or ticketing workflow as rule-based alerts.

The question is: how do you scope these hunts so agents aren’t just running open-ended queries against your entire log corpus? The answer is to give them an opinionated starting point. This is where the signals pattern becomes foundational. If you’ve already been labeling security-relevant events (authentication changes, privilege escalations, infrastructure modifications) and storing them as a filtered layer on top of your raw logs, you’ve built exactly the dataset that continuous agent hunting should operate against. These signals represent the <1% of your log volume that is security-relevant but not necessarily alert-worthy on its own. A new IAM role creation in AWS, a service account added to a privileged group, and a conditional access policy change in Entra ID: none of these individually justifies waking someone up, but all of them are the kinds of activities an attacker would generate.

The mindset shift is subtle. In the rule-based model, you ask: “What specific pattern is malicious enough to fire an alert?” In the continuous hunting model, you ask: “What category of security-relevant activity should an agent be regularly reviewing for anomalies, patterns, or context that my rules wouldn’t catch?” You’re defining scopes, not rigid signatures.

In practice, this is a set of prompts that tell an agent what to look for. The structure should mirror what you’d write for a triage agent: define the threat model, explain why this activity matters, give the agent clear criteria for distinguishing risky from benign, call out common false positives, and specify the enrichment steps and external context sources that will help it make a confident judgment.

Here’s an example hunt prompt for IAM analysis in AWS:

Scope: Review all IAM signals from the past 24 hours, including role creations, policy attachments, trust policy modifications, and access key generation.

Threat model: An attacker who has gained initial access to an AWS environment will often escalate privileges by creating new IAM roles, attaching permissive policies, or modifying trust relationships to enable cross-account movement. These changes are how an adversary establishes persistence and expands access beyond their initial foothold. IAM modifications are among the highest-value signals in a cloud environment because they directly affect who can access what.

Assessing benign vs. risky: Most IAM changes in a healthy environment are made by infrastructure-as-code pipelines (Terraform, CloudFormation, Pulumi) running through CI/CD systems with known service roles. The key question isn’t whether the change happened, but whether the identity, method, and context are consistent with the organization’s normal change patterns. A role creation by a Terraform execution role during a deployment window, correlated with a recent merge in the infrastructure repository, is almost certainly routine. A role creation by a human identity through the console, especially one that doesn’t typically make IAM changes, is worth investigating.

Common false positives: Platform engineering teams making manual IAM changes during incident remediation or environment bootstrapping. Automated security tooling (like AWS Config remediation or SCPs applied by a governance pipeline) that generates IAM signals as a side effect. Sandbox or development accounts where IAM experimentation is expected and doesn’t carry production risk.

Investigation steps: For each IAM signal, query the identity’s activity history over the past 30 days via the SIEM to establish whether IAM changes are part of their normal pattern. Check the identity provider via MCP to determine the user’s role, team, and whether their account has been flagged for any access reviews. Correlate with CI/CD activity (via GitHub or deployment platform MCP) to determine if the change aligns with a recent code merge or deployment. Look for surrounding signals within a two-hour window: new console logins, AssumeRole calls from unfamiliar source accounts, changes to CloudTrail logging configuration, or S3 bucket policy modifications that could indicate data staging.

Output: Produce a structured finding for any activity where the combination of identity, method, timing, and surrounding context suggests behavior outside the expected operational pattern. Include confidence level, supporting evidence, the enrichment sources consulted, and whether the finding warrants analyst review or can be logged as a tracked observation.

That prompt doesn’t specify a single pattern that’s definitely malicious. It defines a threat model, provides the agent with judgment criteria to distinguish routine operations from attacker behavior, and points it to the enrichment sources that enable a confident assessment. The agent might find nothing notable for three days straight and then surface a finding on day four because a contractor’s identity, one that the identity provider shows hasn’t completed its latest access review, created an IAM role with a cross-account trust policy an hour after authenticating via the console for the first time in weeks. No rule was written for that exact chain of events. The agent connected the signals because it was told what mattered, why it mattered, and how to evaluate what it found.

You can build these hunting scopes across any domain where you have structured security signals: configuration changes, identity-based access anomalies, or data movement in cloud storage. Each scope is a standing brief to an agent, and over time, they become a library of continuous hunts, each running on a cadence that matches the risk profile of the activity it monitors: IAM changes reviewed daily, data movement signals reviewed hourly, authentication policy changes reviewed in near real-time.

What Happens to Detection Engineering

Detection engineering doesn’t disappear in this model, but its center of gravity shifts. Rules become the known-good baseline: high-confidence, high-severity patterns that you want firing deterministically every time. If someone disables CloudTrail logging on a production account, you don’t need an agent to hypothesize about whether that’s interesting.

But the coverage between those high-confidence rules, the territory that today only gets examined during periodic hunts if it gets examined at all, becomes the domain of continuous agent analysis. Detection engineers spend less time trying to write rules for every conceivable variation of a behavior and more time defining the hunting scopes that agents operate against. The work shifts from authoring pattern-match logic to articulating threat models, judgment criteria, and enrichment paths that agents can execute on a rolling basis. In a sense, the prompt we walked through in the previous section is what a detection artifact starts to look like: not a rule, but a standing brief that encodes how your team thinks about a class of threat.

This doesn’t reduce the rigor of detection engineering. If anything, it raises the bar. Writing a rule that matches a specific log pattern is hard. Writing a hunting scope that teaches an agent how to distinguish attacker behavior from routine operations across multiple data sources, with clear guidance on false positives and enrichment steps, requires deeper threat modeling and a more explicit articulation of what “suspicious” actually means in your environment. The discipline gets harder, not easier. But the coverage it produces is dramatically broader.

We’re Still in the Early Innings

Most security teams today are still in the first phase of the agentic transition: agents triaging alerts written by humans. That’s a meaningful step, and teams doing it well are already seeing real gains in analyst productivity and response consistency. But it’s a long way from agents generating their own findings autonomously.

Continuous agent hunting only works if agents have something worth hunting through. You need a queryable data lake with enough retention to establish behavioral baselines, structured security signals that give agents an opinionated starting point rather than a raw log firehose, and MCP tooling that connects agents to enrichment sources outside the SIEM. Without those foundations, an agent hunting continuously is just burning tokens and compute against unstructured data, producing low-confidence findings that create more work than they save. The economics of this architecture are directly tied to how well you’ve curated the data underneath it.

The trust model is also immature. Most organizations aren’t ready to act on a finding that an agent surfaced independently without a human validating the reasoning. That’s a reasonable position today, and teams should build toward autonomy incrementally rather than flipping a switch.

Each step requires the agent to earn trust through transparent reasoning and consistent accuracy. Most teams are somewhere in the first or second stage, and that’s fine. The architecture described in this post is what makes the later stages possible.

But the trajectory is clear. If you’re investing in a security data lake, security-relevant signals, detections designed for agent reasoning, and connecting enrichment sources via MCP, you’re building the architecture that makes continuous agent hunting possible. The flywheel that detection engineering always promised, detect, learn, improve, repeat, finally spins at a pace that matches the threat landscape. The constraint was never the data or the models. It was the human bottleneck in the loop.

Cover photo by Oxana Melis on Unsplash

Continued Reading

Every detection you’ve ever written was, at its core, a message to a human. The alert title, the severity label, the description, and the runbook — all of it was carefully designed to transfer situational awareness to an analyst as fast as possible. The assumption baked into that design was that a person would be on the receiving end: someone who would read the alert, follow a set of deterministic steps (check the asset inventory, look up the user’s recent activity, consult the runbook), and decide what to do next. That workflow, whether executed manually or partially automated through SOAR playbooks, was fundamentally built around human cognition as the reasoning engine. That assumption is now changing.

AI agents are increasingly handling the first layer of alert triage and investigation in modern SOCs. When an agent is the first reader of your detection, almost everything about what makes that detection good changes. The fields that matter, how severity gets used, what a runbook is actually for, and what noise even means all shift. And critically, the detection itself evolves: rather than pure rule logic designed to fire a signal at a human, detections combine rule logic with natural language prompts that communicate the threat model, the criteria for risky versus benign, and the investigative intent behind the rule. You’re not just writing logic anymore; you’re providing guidance for a sophisticated reasoning system.

The human role in this model doesn’t disappear; it becomes more strategic. The questions that require human judgment shift from “is this alert legitimate?” to “how would we judge this particular type of alert as risky?” Coverage decisions, compliance requirements, and threat modeling for your specific environment are areas where human expertise sets the direction that agents then execute against.

In this post, we’ll review the evolution from human-led to AI-led alerting in the SOC: what the old world looked like, what concrete changes the new one brings, and what practical steps teams can take to build detections for an agentic SOC.

Subscribe now

The Old World: Detections Designed for Human Throughput

To understand what changes, let’s review how the current detection lifecycle works.

A detection engineer writes a rule: some combination of log conditions, thresholds, and field matching that fires when a specific pattern appears in the data. The rule generates an alert with a title, severity level, description, and, ideally, a runbook outlining next steps. That alert lands in a queue (a SIEM console, a ticketing system, a Slack channel) and an analyst picks it up.

What happens next is a predominantly human process.

The analyst reads the alert title and forms an initial hypothesis. They read the runbook, which walks through a set of investigation steps: check whether the asset is managed, look up the user’s recent authentication history, cross-reference against known-bad IPs, and verify whether this behavior has been seen before for this user or peer group. Each step requires navigating to a different tab or running a different query. The analyst mentally assembles the context from those data points, reaches a judgment (risky or benign), and either escalates or closes.

SOAR platforms were built to accelerate this by automating the deterministic parts. If the IP is in a threat intel feed, automatically enrich it. If the asset is unmanaged, automatically change the severity. But the reasoning step, the moment where someone had to look at the assembled evidence and decide what it meant, always stayed with the human.

The whole system is optimized around that constraint. Runbooks are written to guide human judgment. Severity labels are calibrated to manage human attention. Alert descriptions are written to help a person quickly orient themselves. Even tuning decisions are fundamentally about protecting analyst time, because every suppressed alert reclaims capacity in a finite human queue. The detection lifecycle, from rule authorship to triage to tuning, is a system designed to move signals through human cognition as efficiently as possible.

That’s not a criticism. Given the constraints, the design is reasonable. But it means almost every convention we have around what a “good detection” looks like was shaped by human cognitive limits rather than by what’s actually optimal for identifying threats.

What Changes When the Reader Is an Agent

When an AI agent handles tier 1 alert triage, the reader of your detection changes entirely, reshaping the requirements and design for a good detection.

The most important shift is that the detection’s metadata, which existed to orient a human investigator, now serves a fundamentally different purpose.

A human analyst reads a description and fills in context from experience. They read a runbook and apply judgment at every step. An agent doesn’t do either of those things gracefully. What it does well is reason over explicit context: a clear statement of what this behavior means, what makes an instance of it risky versus routine, and what investigative goals to pursue. The gap between the two models — implicit knowledge versus explicit context — is where most traditional detections will fall short for agents.

The evolution is that detections stop being purely logical and become a combination of rule-based logic and an investigative prompt. The descriptions become a statement of the threat model: what adversary behavior this rule is designed to surface, why it matters, and what legitimate activity looks like in comparison. The runbook becomes agent instructions: the risk criteria the agent should reason over and the investigative goals it should work toward.

Here’s what that looks like in practice, using a real detection for CloudTrail StopLogging, a common technique for evading detection after gaining access to an AWS environment.

The traditional description and runbook are written for a human reader:

The AI-native version provides a threat model and decision framework rather than a procedure. While this description and runbook are longer, agents can write AI-optimized descriptions and runbooks tailored to your SOC’s operational practices, saving time across the entire lifecycle:

The detection logic is identical. What changed is how the agent is guided to work once the alert fires.

We actually ran this exact experiment. These two versions of this detection were deployed simultaneously on a live environment, both firing on the same event: a StopLogging call on a test trail using Stratus Red Team, performed via an admin from the same IP. Both agents ran the same SQL queries against the same CloudTrail data. The only variables were the runbook and description.

The difference in analysis came down to a single benign indicator that only the AI-native agent surfaced: the detection rule itself had been modified 10 minutes before the StopLogging action, and a detection update occurred at almost the same timestamp, a strong signal that this was active testing rather than an attacker disabling important audit trails.

That CI/CD signal appeared in both agents’ query results. The agent with the step-by-step runbook didn’t recognize it as relevant because its runbook said: “check for a change management ticket or deployment pipeline run.” The agent searched for a formal ticket but didn’t find one and moved on. The AI-native runbook’s explicit criterion — a known IaC role with a matching deployment pipeline run at the same time — primed the agent to scan for CI/CD activity, which is how it discovered the detection upload and weighted it as a meaningful, benign indicator.

The runbook didn’t change the agent’s investigation. It changed what the agent recognized as meaningful and how it ultimately scored the alert’s risk level.

Give Agents Goals, Not Scripts

The traditional runbook’s numbered steps are written for a human who reads “check for a deployment pipeline run” and intuitively knows to look at CI/CD history, recent commits, Slack messages, and deployment logs. An agent reading the same instruction performs a more literal interpretation: it looks for what the instruction most directly implies, finds nothing, and moves on. The AI-native runbook closes that gap by encoding the analyst’s implicit reasoning explicitly, as structured risk criteria rather than procedural steps.

Over-specifying investigation steps creates a different problem: it turns a reasoning system into a script executor. An agent told exactly what to do in a fixed sequence will do exactly that, even when the situation it encounters doesn’t fit the template. If the actor ARN immediately matches a known provisioning role at 2 pm, a scripted agent wastes steps. If unusual IP geolocation activity appears at 2 am, a scripted agent may not place much weight on it because it wasn’t in the procedure. Agents perform better when the runbook specifies what a correct conclusion looks like and leaves room for them to find it.

That said, specificity is right in some cases. Compliance-driven response procedures are one: if your controls require a specific notification within a defined time window when certain events occur, explicitly tell the agent. Response actions with real-world consequences are another, disabling an account, revoking credentials, and isolating a host. Those deserve prescribed guardrails because the cost of the agent improvising is too high. The principle isn’t “never be specific,” it’s that specificity should be reserved for the parts of the workflow where deviation genuinely creates risk. For investigation and reasoning, leave room.

Severity labels are worth addressing briefly here as well. For humans, severity was a prioritization tool (”high” meant “interrupt what you’re doing”). For agents, severity is an input to routing and escalation logic. That distinction changes how it should be assigned: less about managing analyst attention and more about accurately encoding the risk profile of the detected behavior so the agent routes correctly. A medium-severity detection that should almost always resolve as benign is different from a medium-severity detection that’s genuinely ambiguous. Encoding that distinction in severity, or in the description, gives the agent better inputs for its routing decision.

The Tuning Problem Inverted

One of the most counterintuitive aspects of the agentic SOC is what happens to the noise problem.

In the human-led model, alert fatigue was the central operational challenge. Teams spent enormous energy tuning detections to reduce noise rates because every alert was burning analyst time they couldn’t get back. The economics were simple: analyst hours are finite, and every wasted alert is a tax on a resource you can’t scale cheaply.

AI agents change those economics in one important way and leave them unchanged in another. Agents don’t get fatigued by volume. They can process alert queues at a scale no human team could match, so the raw-volume problem that drove so many tuning decisions largely goes away. You no longer need to suppress a class of alerts because your analysts can’t get to them.

But noise still matters.

Bad data (low-quality signals, alerts that fire on activity the agent has no meaningful way to distinguish from benign behavior, detections without sufficient context for the agent to reason about) creates two new problems.

First, it’s expensive. Every alert an agent processes consumes tokens, and every tool call it makes to investigate a low-signal alert, pulling enrichment, querying logs, assembling context, compounds that cost. At scale, a noisy detection program isn’t just a quality problem; it’s a budget problem.

Second, it pollutes reasoning. An agent that processes large volumes of low-signal alerts doesn’t burn out; it builds up a pattern of confident-but-wrong conclusions that is hard to audit and even harder to trust. Alert fatigue in the human model degraded analyst morale and response time. The agentic equivalent degrades decision quality in ways that are less visible and harder to catch.

The tuning imperative shifts from protecting analyst attention to protecting agent reasoning quality. The goal of tuning isn’t really to reduce alert volume in aggregate anymore; it’s to ensure that what fires is something the agent can make a meaningful decision about, either because the signal is clean enough to act on autonomously or because it’s genuinely ambiguous enough to warrant routing to a human.

In practice, this creates a new kind of tuning question: not just “is this alert noise?” but “does this alert give the agent what it needs to make a good call?” A detection that produces a technically accurate signal but lacks the enrichment and context for confident agent reasoning is effectively noisy in the agentic model, even if it would have been workable for a human analyst who could go gather that context manually. The threshold for what constitutes a well-formed detection gets higher.

Where Humans Still Own the Work

None of this means analysts are now obsolete. It means the nature of the work shifts, and understanding where that shift lands is important for teams thinking about how to structure their security operations going forward.

The work that remains deeply human is the strategic layer: deciding what to monitor in the first place. Threat modeling (mapping the specific assets, workflows, and data flows that matter for your organization and identifying the attack paths that could reach them) requires organizational context that agents don’t inherently have. Compliance requirements are similar: understanding which controls need to be verifiable, what your audit obligations are, and how to translate those into detection coverage requires judgment about business risk, not just pattern matching over log data. We’ve explored this in prior posts on threat modeling and detection coverage strategy, and the same principle applies here. Agents are very good at executing against a well-defined coverage map; they’re not yet good at defining what that map should look like in the first place.

Novel threat recognition is another area where humans hold an edge that matters. Agents reason well over patterns they’ve been given context for (known TTPs, behaviors encoded in detection logic, threat models that have been explicitly documented). What they struggle with is recognizing that something new and previously unmodeled is happening. Experienced analysts develop intuition for when a pattern doesn’t match anything they’ve seen before, which is a fundamentally different skill from applying existing knowledge. That intuition still needs to be in the loop.

Response decisions with real-world consequences belong here as well. When an investigation concludes that an account has been compromised, the question of whether to disable it, notify the user, involve legal, or escalate to executive leadership involves organizational, legal, and relationship considerations that agents aren’t equipped to navigate. Agents can surface findings and recommend actions, but the accountability for consequential decisions should stay with people.

And finally, humans need to maintain a critical eye on agent reasoning itself. Agents can be wrong in confident, systematic ways that are harder to catch than the more obvious mistakes a tired analyst might make. Someone needs to review the agents’ decisions, identify patterns in cases they’re getting wrong, and update the detection logic and context when agent reasoning consistently goes off track. Oversight of the agents becomes a core analyst function.

Building Toward This Transition

Teams that want to take advantage of what agentic triage makes possible need to carefully consider what must be true in their environment for it to work. The shift from human-led to agent-led triage isn’t just an architectural change; it requires rethinking the context that agents receive to properly delegate triage to the quality expected of senior team members.

The most important investment is in the data layer. Agents reason over what’s available at alert time. If your detections fire against raw, un-enriched log data with inconsistent field naming and no behavioral context attached, agents will make poor decisions regardless of how capable the underlying model is. Normalized schemas, enriched asset context (is this device managed? is this account a service account?), and behavioral baselines (is this unusual for this user or peer group?) should be present in the data that feeds the agent. This is one of the core reasons the security data lake architecture matters: it provides the queryable, structured foundation that enables agentic reasoning at scale.

The second investment is in detection quality as a prompt engineering problem. Start treating your detection descriptions and runbooks as context for a reasoning system, not instructions for a human reader. What does the agent need to know about why this behavior is suspicious? What context should it gather? What criteria distinguish risky from benign in this specific case? The detection engineering discipline doesn’t go away; it gets more rigorous because the output of that work now needs to be interpretable by an AI system, not just a person.

The third investment is organizational. The shift toward agentic triage creates room to reallocate analyst time toward detection coverage, threat modeling, and oversight, but that reallocation doesn’t happen automatically. Teams that get the most out of this transition will be deliberate about it, giving analysts explicit ownership of the strategic coverage questions while building the feedback loops that let agent performance improve over time.

The goal is a flywheel: better threat modeling yields better detections, which give agents better context, which in turn surfaces better signal, and that signal informs the next round of threat modeling. Getting that flywheel moving is the real work of building an agentic SOC and closing the loop.

The experiment described above was run live on a real Panther deployment, using Panther both as the detection system and as the agentic triage engine evaluating the alerts. If you’re building toward the agentic SOC vision described here, that’s exactly what we’ve built with Panther. Reach out or reply to this email, and I’ll give you a demo.

Recent Posts

Cover photo by Ed Robertson on Unsplash

Our conversation explores Google’s methodical approach to AI adoption, starting with incident summaries and progressing to fine-tuned agents for specific detection workflows. Michael discusses the critical distinction between when to use AI versus traditional automation, Google’s “infer and interrupt” model for faster containment, and why the team’s stretch goal is 70% automated operations. His emphasis on golden datasets for training, human-in-the-loop validation even at scale, and the shift from tool expertise to domain expertise provides concrete guidance for security leaders navigating the march to autonomous SOC operations while maintaining precision.

Share

Topics Covered

• Processing 7 Trillion Log Lines with 99%+ Automation: How Google’s detection and response team handles a million tickets annually with less than 1% requiring human intervention, built on 15 years of detection-as-code fundamentals and automation before AI.

• The AI Adoption Journey from Assisted to Autonomous: Google’s progression from AI-assisted incident summaries (reducing 30 minutes to 90 seconds) to AI-led deduplication agents to autonomous workflows, while maintaining conservative precision requirements given their no-fail mission.

• Fine-Tuned Agents on Gemini with Golden Datasets: Why Google uses fine-tuned models validated by humans for specific agents like exfiltration detection, rather than relying solely on prompting, with golden datasets ensuring high-quality training data.

• The Critical Distinction: AI vs Traditional Automation: How Google’s lead engineer established that not everything needs AI—things requiring judgment, nuance, and data analysis benefit from AI, while deterministic “if A then B” workflows should remain traditional automation.

• Deduplication Agent Achieving 95% Precision: Google’s ticket deduplication agent operates at 95% precision with 38% recall, with humans still in the loop but not on every ticket, demonstrating the precision-recall tradeoff in production AI systems.

• Vulnerability Workflow Automation: How AI collects daily vulnerability data from trusted sources, pulls metadata, evaluates infrastructure impact, writes reports, and recommends actions—reducing hours of work to minutes while asking “is Google infrastructure affected” with high confidence.

• Overseer Agents for Quality Control: Google deploys agents that evaluate other agents’ outputs in aggregate and agents that assess ticket quality based on documentation criteria, kicking incomplete work back to analysts—AI evaluating both AI and human work.

• The Infer and Interrupt Model: Google’s security-wide shift toward detecting suspicious behavior early and automatically containing it (cutting email, locking accounts) rather than spinning up full investigations—necessary because AI attackers don’t sleep and move faster.

• TimeSketch Integration with SecGemini: How Google achieved 50x speed improvements in forensic timeline analysis by integrating Gemini with TimeSketch, inventing new chunking methods, and pulling out events no human would catch without knowing exactly what to look for.

• The Future: Broader Detections with Specific Intel Layers: Michael predicts 70% of detections becoming broader “this looks odd” signals that trigger challenges or containment, layered with shorter-lived specific detections based on current threat intelligence for rapid pivoting.

Subscribe now

The transformation Michael describes aligns with Panther's approach to AI integration, using agents to scale judgment and pattern recognition, while maintaining deterministic logic for routine workflows. Security teams can focus on the critical thinking and domain expertise that Michael emphasized as irreplaceable. Learn more about Panther AI and how our AI SOC platform helps scale human judgment rather than replace it!

Our conversation explores Block’s journey building Goose, a general-purpose AI agent used across the company, and co-designing the Model Context Protocol with Anthropic. James discusses Block’s “democratizing detections” principle, where nearly half of all new detections in 2025 were created with AI, and how the company balances principled risk-taking with security rigor through data safety levels and AI security principles. His emphasis on human accountability for agent actions, the development of Binary Intelligent Triage, which achieves 99.9% efficacy, and Block’s commitment to open source provide concrete guidance for security leaders navigating AI adoption while maintaining high security standards.

Share

Topics Covered

• Building Goose as a General-Purpose Agent: How Block developed Goose as an open source agent for the entire company to perform analysis, deep research, and automate common tasks with recipes, eventually contributing it to the Agentic AI Foundation under the Linux Foundation.

• Co-Designing MCP with Anthropic: Block’s partnership with Anthropic to develop the Model Context Protocol alongside Goose, creating a reference implementation platform that unlocked automation across the company by connecting to numerous systems through MCP servers.

• Prompt Injection Defenses in Goose: Block’s research into hardening Goose against hidden prompt injection attacks, implementing both deterministic detection and adversarial AI concepts where one LLM reviews commands and context provided to another LLM as a judge.

• AI Security Principles and Data Safety Levels: How Block evolved its CDC-inspired data safety levels into AI security tiers, creating an accelerated review path that balances speed with security based on what data agents process and what actions they can take.

• Democratizing Detection Engineering: Block’s principle that anyone at the company can write detections using natural language with Goose and MCP-Panther, leading to 40% of new detections in 2025 being created with AI assistance, including contributions from teams outside security, like Bitcoin product engineers.

• Binary Intelligent Triage Achieving 99.9% Efficacy: How Block’s system stores historical detections, alerts, and investigations in a vector database to perform semantic analysis on new alerts, achieving near-perfect efficacy and enabling confidence in automated analysis actions.

• Human Accountability for Agent Actions: Why Block requires agents to be connected to internal identity so code appears as written by the human operator, maintaining responsibility and avoiding “the agent just wrote that” scenarios, with humans still required to review PRs.

• Headless Goose for Autonomous Workflows: Block’s CLI version of Goose that integrates with frameworks to create PRs, JIRA tickets, and automatically fix vulnerabilities from scanner output, while still requiring human approval before code is pushed.

• The Future SOC Without Tool Expertise: James predicts that security professionals won’t need expertise in specific tools or domain-specific languages, but will work in natural language, while still requiring a deep understanding of complex technical systems and domain expertise to stay ahead of attackers.

• Open Source as Economic Empowerment: How Block’s open-source commitment stems from CEO Jack Dorsey’s belief in economic empowerment and in providing financial tools for everyone, with its secret sauce being the ability to scale and empower people rather than closed-source software.

Subscribe now

The conversation with James mentions Panther's partnership with Block on MCP-Panther, which enables natural language detection, alert triage, and investigation while maintaining code-based rigor and a human-in-the-loop approach. By democratizing detection creation through AI agents while preserving accountability and review processes, security teams can scale detection coverage and empower broader organizations to contribute security expertise. Learn more about Panther AI and MCP integration and how we're building systems that combine accessibility with engineering discipline.

Additional Reading

• Block CISO: We red-teamed our own AI agent to run an infostealer on an employee laptop

• Democratizing Detection Engineering at Block: Taking Flight with Goose and Panther MCP

Our conversation explores Ryan’s philosophy on where LLMs excel in security operations—particularly their strength in semantic understanding and intent classification—versus where traditional deterministic models remain superior. Ryan’s practical experience building custom ML models for phishing automation, combined with his evaluation of commercial AI SOC products, provides a grounded perspective on AI adoption. His emphasis on explainability, the importance of tuning at the detection layer rather than the analysis layer, and the need for human-in-the-loop validation provides concrete guidance for security teams navigating AI agents while building sustainable automation.

Topics Covered

• LLMs for Semantic Understanding Over Decision-Making: Ryan argues the biggest strength of language models is natural language processing for documentation and intent classification, not making binary malicious/benign determinations, where deterministic models prove more reliable and explainable.

• Using LLMs as Feature Generators for Deterministic Models: Rather than having LLMs make security decisions directly, Ryan uses them to generate binary feature flags that analyze email context (tone, product selling, aggression) and feed them into more reliable traditional ML models.

• The 95% On-Call Reduction Through Custom ML: Ryan’s phishing automation model processes 400+ daily reported emails, handling classification (phishing/benign/spam), automated response (quarantine/release), and reducing analyst burden while maintaining high accuracy through company-specific training.

• Agent SOC Limitations and Hallucinations: Ryan’s evaluation of commercial AI SOC products revealed gaps in which agents claim to perform analysis steps they didn’t actually execute, and make false statements such as “user never authenticated from this IP” when logs show otherwise.

• Tuning at the Detection Layer, Not Analysis Layer: Why applying AI-powered allow-listing and tuning at the analysis layer across multiple detections is more dangerous than tuning individual detection rules, as blanket AI rules can inadvertently suppress legitimate alerts.

• SOAR Integration for Contextual Flexibility: How language models can make SOAR workflows less brittle by handling ambiguous cases like determining if a reported email is actually a “forward of a forward” versus a legitimate report, routing appropriately for manual or automated triage.

• The Challenge of Context Management: The difficulty of documenting business partner relationships, third-party integrations, and legitimate, unusual behaviors in ways that both humans and AI systems can reliably access during incident analysis.

• Useful vs. Noise Alert Tagging: Why binary alert classification (useful/noise) with subcategories provides better feedback loops for AI systems than ambiguous “true positive/false positive” labels, enabling pattern matching and detection tuning over time.

• The Importance of Analytical Skills for Detection Engineers: Ryan emphasizes that detection engineers need data science and analytical tool experience beyond security knowledge, recommending that everyone build at least one decision tree model to understand ML effectiveness and limitations.

• Prompt Injection Risks and Documentation Poisoning: How malicious actors can manipulate LLM responses through confluence page ranking or documentation injection, similar to 1990s Google SEO spam, creating attack vectors for autonomous security systems.

Subscribe now

The transformation Ryan describes aligns with Panther's approach to AI—leveraging language models' semantic understanding strengths for alert analysis and investigation, while maintaining deterministic detection logic and human validation. By automating the pattern matching and initial triage that LLMs excel at, security teams can focus on the custom model building, detection tuning, and strategic security decisions that Ryan emphasized as critical for reducing analyst burnout. Learn more about Panther AI and how we're building systems that combine AI efficiency with detection engineering rigor.

This post continues our series on using AI agents and Model Context Protocol (MCP) servers to build and operationalize threat models for security operations:

Threat hunting has historically been a tedious, manual process: you get an indicator or set of suspicious behaviors from threat intelligence, then spend hours (or days) searching through logs to determine if there’s evidence of activity in your environment. Security analysts manually parse reports for IP addresses, domains, or attack patterns, then craft queries against their security data lake, iterating through different log sources until they either find something or exhaust their patience. It’s analytically intensive work that requires deep expertise in both attacker tradecraft and your organization’s specific infrastructure.

This is exactly the kind of work AI agents should excel at. The SOC analyst’s job is fundamentally analytical—synthesizing context from multiple sources, forming hypotheses about adversary behavior, and validating those hypotheses against evidence in your data. The limiting factors for AI-driven security operations aren’t model capabilities anymore; they’re access to the right data and tools (can the agent query your identity provider, logs, and HR systems?), human comprehension speed (can your team review findings fast enough?), and organizational alignment (is everyone clear on what threats matter most?). AI agents like Claude Code provide an intelligent harness that combines data access through MCP servers, tool execution, and iterative workflows to compress what used to take days of manual work into hours of guided analysis.

In our previous post, we demonstrated how to build threat models using AI agents and MCP, resulting in a comprehensive, prioritized set of threats. But a threat model sitting in a document doesn’t improve your security posture until you operationalize it. The right sequence is to prioritize threats with stakeholders, hunt for historical evidence of those threats in your environment, and then formalize successful hunts into detection-as-code rules. Every alert is a claim on your team’s time and attention, so you need to validate that a threat is relevant before committing to long-term monitoring.

This post will dive into using Claude Code and MCP to hunt for the threats you’ve already prioritized, gathering evidence that will directly inform which detections to build and how to scope them. We’ll walk through the stakeholder alignment process that turns threat models into hunt priorities, show concrete examples of AI-assisted hunting workflows, and explain how hunt findings translate into detection logic.

Subscribe now

Stakeholder Alignment on the Threat Models

Before diving into threat-hunting workflows, you need organizational alignment on which threats matter most to your business right now. Many security teams either skip stakeholder buy-in entirely, leading to hunts that don’t align with leadership priorities, or they schedule a three-hour meeting where everyone argues about hypotheticals without clear outcomes. The threat model you’ve built makes this conversation substantially easier because you’re not debating what could theoretically happen; you’re reviewing documented threats with context.

The stakeholder alignment meeting should include security leadership, detection engineers, threat intelligence analysts, and incident response leads. Essentially, anyone who will either execute the hunts or respond to these specific threats. The agenda is to review your identified crown jewels (what systems are existential if compromised versus merely operational), discuss which threat actors are actually relevant to your industry and geography, and align on the top five priority threats from your model that warrant immediate hunting. Prior to this meeting, you should carefully validate the AI’s findings for accuracy/correctness.

The expected outcome is a ranked shortlist of three to five threat scenarios to hunt for with explicit business justification documented for each. The common pitfall is trying to hunt for everything at once, resulting in 20 different threat scenarios across 10 log sources with no clear hypothesis about what success looks like. Focused hunts with clear hypotheses win. For example, if you’ve aligned on contractor privilege abuse, compromised third-party integrations, and SSH lateral movement from developer workstations, you now have a concrete plan for the technical work ahead. The rest of your threat model doesn’t disappear; it becomes your backlog for future hunting sprints as you systematically work through priorities.

Hunting for Signals with MCP and Claude Code

You’ve aligned on three to five priority threats with your stakeholders. Now comes the technical work: hunting through your security data lake to determine if there’s historical evidence of these threats in your environment. This is where AI agents with MCP access compress weeks of manual query iteration into hours of guided analysis.

The workflow is straightforward. Start with a specific threat ID from your model, translate it into testable hypotheses, query your data lake through MCP servers, and iterate based on what you find. Throughout this process, the agent will query real log data, review past alerts, examine existing detection rules, and produce structured findings on where and how to create actionable detections.

Teaching the Agent to Hunt

Claude Code supports a feature called Skills, which are reusable instruction sets that teach the agent how to approach specific types of work.

A Skill is just a folder containing a SKILL.md file followed by instructions. Claude automatically discovers and loads Skills when their description matches the current task, applying specialized knowledge without you having to repeat the same prompting patterns in every conversation.

For threat hunting, this builds a repeatable thought process for the agent to orient on the task and understand effective hunting techniques.

Here’s how to create one:

1. Create the skill directory structure:

mkdir -p ~/.claude/skills/threat-hunter # Or in your local directory

2. Create the SKILL.md file at ~/.claude/skills/threat-hunter/SKILL.md:

---
name: threat-hunter
description: Hunt for cyber threats and investigate security incidents. Use when investigating alerts, hunting for IOCs, analyzing suspicious activity, or performing incident response.
---

# Cyber Threat Hunter

You are assisting a security analyst with threat hunting and incident investigation.

## Threat Hunting Methodology

### 1. Hypothesis-Driven Hunting
Start with a hypothesis about attacker behavior:
- "An attacker with initial access would enumerate cloud resources"
- "Compromised credentials would show unusual login patterns"
- "Data exfiltration would involve large outbound transfers"

### 2. The Pivot Loop
Effective hunting follows a continuous pivot pattern:
1. **Start** with a known indicator (IP, user, hash, domain)
2. **Query** for all activity involving that indicator
3. **Identify** new related indicators from the results
4. **Pivot** to those new indicators
5. **Repeat** until you've mapped the full scope

### 3. Correlation Priorities
When investigating, correlate across these dimensions:
- **Time**: What happened before/after the suspicious event?
- **Identity**: What else did this user/service account do?
- **Network**: What other systems communicated with this IP?
- **Host**: What other processes ran on this machine?

## Investigation Patterns

### Alert Triage
1. Review the alert and triggering events
2. Assess: Is this expected behavior for this user/system?
3. Pivot: What else happened in the same timeframe?
4. Scope: Are there similar alerts across other entities?
5. Conclude: True positive, false positive, or needs escalation?

### User Compromise Assessment
1. Establish baseline: What's normal for this user?
2. Check authentication: Unusual locations, times, or devices?
3. Review access: Did they touch sensitive resources?
4. Look for persistence: New MFA devices, API keys, OAuth grants?
5. Check lateral movement: Access to other accounts or systems?

### IOC Hunting
When given an indicator (IP, domain, hash):
1. Search for any historical activity involving the IOC
2. Identify all affected users and systems
3. Determine first and last seen timestamps
4. Map the attack timeline
5. Look for related IOCs to expand the hunt

## Key Questions to Answer

- **Who**: Which identities are involved?
- **What**: What actions were taken?
- **When**: What's the timeline of events?
- **Where**: Which systems/regions/networks?
- **How**: What techniques were used (map to MITRE ATT&CK)?
- **Why**: What was the likely objective?

Once saved, Claude Code will discover this Skill and apply it whenever you’re working on threat hunting or incident investigation tasks. We recommend refining this guidance based on your own preferred best practices, organizational context, and internal hunting methodologies.

After you create this file, you’ll need to quit and reopen Claude Code. Then, you can invoke the skill explicitly using /threat-hunter <prompt>.

Executing the Hunt for Past Activity

With your threat-hunter Skill in place and MCP servers connected to your SIEM, like mcp-panther, you’re ready to start hunting. The prompt structure is straightforward: simply point the agent at your threat model, specify which threat(s) to hunt for, and set guardrails on scope and output:

Using the threat-model.md file, hunt for evidence of threat T-INSIDER-002 (contractor privilege abuse) across the last 90 days. 

Check for:
1. Related existing detection rules and subsequent alerts
2. Authentication patterns from contractor accounts
3. Access to sensitive resources or data stores
4. Privilege escalation attempts or unusual permission changes

For each finding, assess: Is this expected behavior or anomalous? 
Document evidence with timestamps, actors, and affected resources. 
When identifying gaps, be specific and avoid ambiguity.

Store your findings in markdown format in a local hunts/ directory, 
recommendations in a recommendations/ directory, and track progress 
in a tracker.yml file.

A few notes on effective hunt prompts:

• Start with the CRITICAL or HIGH-severity threats from your model (the ones your stakeholders prioritized).

• You can optionally hunt entire threat categories (such as supply chain or insider threat) or individual threats, depending on how deep you want to go.

• Specify the time range explicitly (30, 60, or 90 days) to balance pulling recent activities with the costs of looking back multiple months at once.

• You should also request external research to ground the hunt in reality: “Include research on common detection methods and past attacks for contractor privilege abuse.”

Interpreting Hunt Results

Threat hunting rarely produces simple yes/no answers. The output from a successful hunt is a structured assessment that synthesizes evidence across multiple dimensions, including what activity occurred, the confidence level assigned to the conclusions, and the context gaps that prevented stronger findings.

The hunt report structure should communicate any uncertainty appropriately while still providing actionable intelligence. A complete hunt produces several outputs:

• Executive summary with findings and confidence: States what you found (or didn’t find) and assigns a confidence level based on data completeness. When you see “No evidence of compromise” paired with MEDIUM confidence, it signals that the hunt found no anomalies, but there are detection gaps that could hide sophisticated activity. The confidence assessment tells you how much weight to give the conclusion.

• Evidence analysis by activity type: Breaks findings into logical categories, like authentication patterns, resource access, privilege changes, or workflow modifications. Each piece of evidence gets assessed as EXPECTED (clearly legitimate business activity), NEEDS REVIEW (requires stakeholder verification to determine if it’s authorized), or ANOMALOUS (suspicious enough to escalate immediately).

• Follow-up items prioritized by severity: Separates findings that need immediate investigation from those requiring routine verification. When the hunt identifies admin role assignments, secret deletions, or workflow modifications, it’s flagging them for validation.

• Detection coverage gaps mapped to attack techniques: Documents where you lack visibility. If the hunt revealed you’re collecting CloudTrail but missing GitHub audit logs, or you have authentication events but no privileged access management telemetry, those gaps represent blind spots in your detection posture.

The hunt findings directly inform what happens next. Genuine anomalies or confirmed policy violations become high-priority detection rules, using the hunt queries you’ve already developed as the starting point for the detection logic. Coverage gaps become your instrumentation backlog, prioritized by the severity of threats you can’t currently see.

Not everything discovered during hunting needs to become an alert. Some detections perform better with scheduled hunting procedures (quarterly supply chain threat hunts, monthly privileged access reviews). The question then becomes “what’s the right operational cadence for monitoring this threat given its likelihood and triage automation?”

Share

The Compounding Value of Agent-Assisted Hunting

At the beginning of this post, we outlined the traditional threat hunting workflow: analysts manually parsing intelligence reports, crafting queries, iterating through log sources, and hoping to find evidence before exhausting their patience. The limiting factors weren’t about whether your team was smart enough to find the threats—it was whether they had enough time to synthesize context from dozens of sources, enough stamina to iterate through query variations, and enough organizational alignment to prioritize the right hunts in the first place. AI agents fundamentally change this calculus by compressing days of manual correlation work into hours of guided analysis, freeing your team to focus on the analytical work that actually requires human judgment: validating hypotheses, assessing business impact, and deciding what deserves sustained monitoring.

What we’ve demonstrated in this post is the full cycle of operationalizing threat intelligence with AI agents and MCP. You start with structured threat models that document business context and detection gaps, align stakeholders on priority threats through focused conversations rather than endless debates, and then deploy agents with direct access to your security data lake to hunt for evidence across multiple log sources simultaneously. The agent synthesizes findings and produces structured assessments with appropriate confidence levels and gaps. This augmentation lets your team operate at a fundamentally different pace, turning quarterly threat-hunting exercises into weekly sprints where you continuously validate your threat model against real evidence in your environment.

The productivity gains compound over time because agents help you maintain detection quality, not just create initial rules. Hunt findings that reveal detection gaps become top priorities for instrumentation. Queries that successfully identify suspicious activity serve as the foundation for new detection-as-code rules. As your threat landscape evolves and new attack techniques emerge, you can re-run hunts against updated intelligence to validate whether your existing detections still provide adequate coverage or whether new blind spots have developed. This is the compounding advantage of AI-first security operations. Your team’s analytical capacity grows with every hunt rather than resetting to zero each time you need to investigate a new threat. In our next post, we’ll show how to formalize these hunt findings into production detection-as-code rules—translating the queries and correlation logic you’ve validated during hunting into sustainable, version-controlled detections that run continuously across your data lake.

Operationalizing AI-Assisted Hunting at Scale

If you’re building this workflow yourself with Claude Code and MCP servers, you now have a blueprint for agent-assisted threat hunting that integrates directly with your existing security data infrastructure. But most security teams don’t have the engineering capacity to maintain custom MCP integrations, develop hunting Skills, and orchestrate these workflows across dozens of analysts. Panther AI brings embedded AI agents directly into your security operations platform, making these capabilities available to your entire team without requiring everyone to become prompt engineers or MCP developers.

Panther’s AI agents have native access to your normalized security data lake and existing detection-as-code rules, enabling them to run ad hoc hunts like the ones we’ve demonstrated here, suggest alert tuning based on historical false-positive patterns, and recommend detection priorities based on threat model alignment. Instead of every analyst learning how to write the perfect hunting prompt, your team gets a shared intelligence layer that operationalizes these workflows consistently across your SOC.

If you’re ready to move from manual threat hunting to AI-assisted security operations, visit panther.com to see how Panther AI accelerates detection engineering and threat hunting workflows for modern security teams.

Continued Reading

Recent Podcast Episode

Cover Photo by Luke Jones on Unsplash

Our conversation explores Mike’s research into cyber leadership qualities and his framework for prioritizing security efforts, which he calls the C3 Matrix. Mike’s perspective on the psychological evolution of cyber threats, from clandestine network attacks to AI-powered assaults on human judgment, challenges conventional approaches. His emphasis on emotional intelligence as a critical leadership trait, backed by Harvard Business Review research showing a 20% impact on revenue goals, provides a data-driven framework for building effective security teams. Mike’s practical experience implementing AI-powered tools in his SOC, combined with his analytical approach to threat operations and deception capabilities, offers concrete guidance for practitioners navigating the transition to AI-enhanced security programs.

Topics Covered

• The Essential Qualities of Cyber Leaders: Mike’s research across 100 sources revealed that 60% prioritize effective communication and 59% value emotional intelligence in security leaders, with Harvard Business Review data showing these traits correlate to a 20% difference in meeting annual revenue goals.

• The C3 Matrix for Security Prioritization: A three-tier framework categorizing assets into Centers of Gravity (compromise means cigars time), Crown Jewels (requires SEC 8K filing but recoverable), and Capability Enablers (supports mission but transparent to customers), helping teams focus security controls where they matter most.

• The Seven Ds of Security: Beyond the passive “discover and detect,” Mike outlines deny, disrupt, degrade, destroy, and deceive as active counter-adversary measures, with deception operations providing the most accurate and actionable threat intelligence.

• Threat Intelligence vs. Threat Operations: Why every SOC needs a dedicated threat operations team that goes beyond consuming external reports to operationalizing intelligence, conducting deception operations, and providing strategic guidance to leadership—a fundamentally different skillset from blue team operations.

• The Psychological Evolution of Threats: How attacks have progressed from technical viruses to phishing to ransomware, and now to AI-powered attacks targeting human judgment and decision-making, with adversaries openly challenging defenders to distinguish reality from fabrication.

• AI as Both Force Multiplier and New Attack Vector: Mike’s practical experience shows AI dramatically reduces investigation time by aggregating data across tools, but also introduces new risks through prompt injection, AI poisoning attacks like the Minja attack, and the potential for AI-based malware that learns network behavior before striking.

• The Bloom’s Taxonomy Limitation: Why AI currently stops at step four of Bloom’s educational model—knowledge, comprehension, application, and analysis—but struggles with evaluation and creation, meaning human analysts remain essential for validation and critical thinking.

• Defense in Personnel: Beyond defense in depth for technology, organizations need multiple people trained on each capability to prevent single points of failure, with cross-functional training programs enabling teams to handle unexpected scenarios.

• Preventing Analyst Burnout: How AI tools help reduce the manual effort of pivoting between multiple security tools during investigations, enabling faster incident resolution and more sustainable work practices for security teams.

Subscribe now

The transformation Mike describes mirrors Panther's AI-powered capabilities—automating the time-consuming work of correlating data across multiple sources while maintaining human expertise for strategic decisions and validation. By reducing the manual burden of log analysis and tool-hopping, security teams can focus on the threat modeling, leadership, and cultural aspects that Mike emphasized as critical for long-term success. Learn more about Panther AI and how we're building tools that amplify human expertise rather than replace it.

We hope you had a great start to 2026!

Threat modeling is the process of analyzing systems from an attacker’s perspective to prioritize defenses, monitoring, and countermeasures. It ensures that detection efforts are high-leverage and focused on preventing the next potential incident rather than reacting to whatever threat is trending this week.

In 2025, security leaders sent a resounding message: focus on the fundamentals, and that starts with understanding your highest-priority threats to the business. But what does modern threat modeling look like with AI at your disposal?

Security teams have historically been siloed and relied on peer expertise to understand how the business’s core systems work and where potential security flaws exist. With AI agents, security teams can gain direct access to context previously locked within engineering, product, and infrastructure teams. This is especially powerful in understanding what you’re protecting.

This post kicks off a series on using AI agents and Model Context Protocol (MCP) servers to build and operationalize threat models for security operations. We’ll use Claude Code and MCP servers to research, analyze, and gather data from your SIEM and organizational context to answer three critical questions: Where should we focus our detection efforts? What are our current blind spots? What should we do about them?

In this post, we’ll walk through generating a comprehensive threat model using AI agents. The subsequent post will show you how to hunt for active threats using that model and immediately formalize findings into detection-as-code rules. These workflows are generally repeatable across different AI agent tools, so you can adapt them to whatever’s in your stack.

Subscribe now

Why Threat Modeling Matters for Detection Programs

Threat modeling defines your priorities as a security team. It answers the question: What’s most important to be monitoring for right now? Without this foundation, detection programs drift toward either alert fatigue (monitoring everything, prioritizing nothing) or reactive mode (chasing whatever threat made headlines this week). Both approaches waste the scarcest resources security teams have: time and attention.

Strong threat modeling creates a through line from detection to response. Each alert should be treated as a line item in your team’s time-and-attention budget. If you’re not confident that an alert maps to a real threat against a critical asset, you’re wasting that budget on noise. Slava Klimov from CoreWeave described this problem on the Detection at Scale podcast #71:

Security teams often build detection coverage based on what’s easy to instrument rather than what’s actually important to protect. The result is high-fidelity alerts on low-value assets and blind spots on crown jewels.

The traditional barrier to good threat modeling has been coordination.

Building a useful model requires input from engineering (what are we running?), product (what’s business-critical?), infrastructure (where are trust boundaries?), and security (what are the active threats?). AI agents with access to organizational context via MCP servers can now synthesize this information directly, removing the weeks of meetings and documentation review that made threat modeling a once-a-year exercise rather than a continuous practice.

The Five Contexts AI Agents Need for Threat Modeling

AI agents can only be as good as the context they’re provided, and threat modeling requires five distinct layers of intelligence that most organizations keep in separate systems. In a previous post on context engineering, we outlined four foundational layers for security operations. For threat modeling specifically, we need to expand this model to distinguish between detection posture (what we can see) and operational history (what we have seen).

Each of the following layers answers a critical question that informs where detection efforts should focus:

1. Identities and Assets tell you what exists and who has access to it. This layer includes user directories, service accounts, cloud resources, SaaS applications, databases, and APIs. For threat modeling, this context answers “What are we protecting?” and “What are the attack paths between assets?” An AI agent querying your identity provider and cloud environment can map trust relationships, identify privileged accounts, and surface shadow IT that traditional asset inventories miss.

2. Threat Intelligence provides the adversary perspective tailored to your industry and technology stack. This includes threat actor profiles actively targeting your sector (financial services faces different adversaries than healthcare) or MITRE ATT&CK techniques commonly used in your vertical. This context answers “Who attacks organizations like ours?” and “What techniques are they using against our sector right now?”

3. Logs and Detection Coverage reveal what you can currently see in your environment. This layer includes which log sources are instrumented (Windows Event Logs, CloudTrail, Kubernetes audit logs), data sources collected but not actively monitored, detection rules currently deployed and their MITRE ATT&CK mappings, and coverage gaps where you have no visibility. This context answers “What can we detect?” and “Where are our blind spots?” that map your theoretical detection capabilities against threat scenarios.

4. Alerts and Case History reveal prior security events in your environment. This layer includes SIEM alerts from the past N days, closed incident tickets and their root causes, false-positive patterns, and mean time to detect across different attack types. For threat modeling, this context answers “What have we seen before?” and “What’s generating noise versus signal?” This historical perspective prevents threat models from treating all scenarios equally, as your operational reality shows that specific attack paths recur while others remain more hypothetical.

5. Organizational Context connects technical assets to business criticality. This includes architecture documentation, data classification policies, revenue-impacting systems, compliance requirements, and deployment patterns (on-prem, cloud, hybrid). This context answers “What matters most to the business?” and “What’s the impact if this system is compromised?” Understanding which applications handle customer PII or which infrastructure supports your core product can prioritize threats based on business impact rather than technical severity scores.

Model Context Protocol makes these five layers accessible to AI agents through standardized interfaces. Instead of manually gathering context from a dozen different browser tabs or documents and synthesizing it, you can connect MCP servers to your agent and let it query across all five layers simultaneously. Security teams can choose from official integrations that either run locally or remotely.

Building the Threat Model with AI

Connecting your MCPs

Before you can generate a useful threat model, your AI agent needs access to the five context layers. At minimum, you’ll want to connect:

• SIEM data (e.g., Panther) for log sources, detection rules, alert history, and query capabilities across your security data lake. This MCP server provides both read access to existing detections and the ability to run ad-hoc queries for threat hunting.

• Ticketing and case management systems (Jira, Linear) to surface historical incident patterns and understand what your team has responded to before.

• Documentation (Notion, Confluence) where application architecture, runbooks, and data classification policies live. This organizational context prevents the AI from treating all assets equally when some are clearly more critical than others.

• Operational monitoring (Sentry, New Relic, CloudWatch) to understand which systems are customer-facing, which handle sensitive data, and where trust boundaries exist in your infrastructure.

The specific MCPs you’ll need depend on your stack, but the point is to give the agent read access to systems that hold context about your environment, threats, and detection posture. Claude Code supports MCP natively, and other AI agent frameworks are rapidly adding support.

Analyzing Your Internal Software

Most companies maintain internal software and services that serve as their revenue source. This context is crucial for threat modeling as it helps map specific components to attack techniques, identify necessary telemetry, and focus detection logic where it matters most.

The sample prompt below dissects these applications and pinpoints the most critical context for AI/SOC consumption, focusing on detection and response rather than application security.

Generate a security threat model for this codebase as a YAML file optimized for AI/SOC consumption. Focus on detection and response, not application security.

Structure it with:

• metadata

• architecture_components (with IDs, data sources, log verbosity)

• data_flows (what’s visible to defenders, detection points)

• trust_boundaries

• authentication (what gets logged, session indicators)

• sensitive_assets (crown jewels, business impact)

• external_integrations (third-party attack surface, credential exposure)

• log_sources (what telemetry exists, gaps, retention)

• attack_paths (MITRE ATT&CK mapped: initial_access → persistence → lateral_movement → exfiltration, with TTPs and detection opportunities)

• threat_scenarios (attacker objectives, kill chain stages, IOCs, forensic artifacts)

• detection_coverage (which ATT&CK techniques are detectable, blind spots)

• response_playbooks (containment actions, evidence preservation, escalation triggers)

• monitoring_recommendations (alerts to build, threat hunting queries, dashboards).

Use explicit IDs, map to MITRE ATT&CK technique IDs where applicable, and identify visibility gaps.

Point the agent at your application repository and let it analyze the code, infrastructure definitions, and documentation. What you’ll get back is a structured system architecture that identifies potential areas of interest for an attacker, which will serve as a reference point for the next prompt to produce a prioritized threat model that considers broader organizational data sources.

Generating Organizational Threat Models

The organizational-level threat model takes a broader view, synthesizing business context, historical incidents, and threat intelligence to produce prioritized recommendations. This is where connecting to your SIEM becomes necessary. The agent needs to query your alert history, detection rules, and log sources to ground the model in operational reality.

This prompt positions the agent as a senior security architect conducting a comprehensive threat modeling exercise:

You are a senior security architect conducting a threat modeling exercise for [COMPANY_NAME]. Your goal is to produce a prioritized threat model covering current risks, controls, monitoring gaps, and historical activity.

Discovery

1. Business Context: Research the company’s industry, geography, technology stack, and what makes it a valuable target.

2. Assets & Identities: Query log sources, cloud environments, identity providers, and critical data stores.

3. Detection Coverage: Review active detection rules, MITRE ATT&CK coverage, alert volumes, and log sources without rules.

4. Historical Activity: Analyze the last 90 days of alerts for patterns, recurring issues, and unresolved items.

Interview

Interview me for additional context. I’m the [TITLE].

If available, request any internal application threat models or architecture documentation to incorporate org-level security findings (authentication flows, sensitive data inventory, third-party integrations, CI/CD pipelines).

Output

Produce a markdown threat model document that includes:

• Executive summary with overall risk level

• Critical assets ranked by business impact

• Relevant threat actors based on industry/geography

• Threat register with unique IDs (e.g., T-001, T-002), likelihood, impact, and mitigation status

• Current security controls and monitoring coverage

• Detection gaps mapped to ATT&CK where applicable

• Prioritized recommendations (immediate, short-term, long-term)

Assign each identified threat a unique ID for tracking and reference.

The agent will conduct a back-and-forth interview to clarify organizational specifics. The resulting threat model included sections like this:

• Business Context that identifies the company profile, industry positioning, and what makes it a high-value target.

• Crown Jewels ranked by existential risk versus operational impact.

• Threat Actor Analysis prioritizes relevant threat actors based on your industry and customer base.

• Historical Incident Analysis based on the past 90 days of alerts from your SIEM.

• Threat Register with unique tracking IDs for each identified threat. For example:

T-INSIDER-002: External contractor abuse of admin privileges
Likelihood: Medium | Impact: High | Risk: HIGH
Mapped to: T1078 (Valid Accounts)
Mitigation: [context-specific recommendations]

This structured output becomes a living document you can reference when prioritizing detection engineering work, conducting quarterly security reviews, or explaining risk to executives. The unique threat IDs enable tracking over time—you can note when T-INSIDER-002 gets addressed with new monitoring controls and update its status accordingly.

What to Look For in Your Threat Model

A useful threat model should tell you three things immediately:

1. Which assets matter most (crown jewels)

2. Which adversaries are relevant to your organization

3. Where detection gaps are mapped to actual attack techniques

If your threat model lists generic threats without tying them to your specific environment, the agent didn’t have enough context. Go back and ensure you’ve connected MCP servers or provide business context manually by typing it out in a stream of consciousness.

The most valuable output is the detection gap analysis—where the model explicitly identifies attack paths you can’t currently detect because you lack the necessary log sources or detection rules. These gaps become your prioritized backlog for threat hunting and detection engineering.

From Threat Model to Detection Priorities

The threat model you’ve just generated is the foundation for your detection engineering roadmap, monthly security reporting, and immediate threat hunting activities. With access to your SIEM, web search, ticketing systems, and internal documentation through MCP servers, AI agents can synthesize context that would typically require weeks of meetings across engineering, infrastructure, and security teams. The result is a living threat model with unique threat IDs and explicit detection gaps, aligning with what matters most rather than a static document that becomes stale the moment it’s published.

The natural next question is: are there signs of historical activity we missed? In the next post, we’ll use Claude Code and mcp-panther to hunt for evidence of these prioritized threats across your security data lake and immediately formalize successful hunts into detection-as-code rules. This workflow, from threat model to hunting queries to production detections, shows how AI agents compress what used to be month-long cycles into actionable security improvements you can deploy the same day.

Cover Photo by A Chosen Soul on Unsplash

Gary offers a refreshing take on building security programs under constraints at one of the world’s most recognized trust platforms. His team’s approach to AI, from automated alert triaging to brand protection, demonstrates how smaller security teams can punch above their weight class. The conversation delves into the cultural challenges of introducing AI, the importance of guardrails, and how to free up security professionals from repetitive work so they can focus on prevention and strategic initiatives.

Topics Covered

• Bootstrapping Security at Trustpilot: Gary’s journey building Trustpilot’s security operations from two people to a team of ten, starting with understanding business pain points and working backwards from POCs to fill security gaps.

• The Alert Capacity Math: Why understanding your team’s capacity—8 hours per day, 15 minutes per alert equals only 32 alerts maximum—forces strategic decisions about automation and horizontal scaling.

• AI for Alert Triage and Enrichment: How Trustpilot uses AI within SOAR workflows to automatically triage alerts, parse JSON, apply logic, and route decisions, including transforming complex security alerts into language end users can understand.

• Competitive Prompt Testing for AI Adoption: Gary’s approach of A/B/C testing three different prompts with the same input during development, measuring outputs, and promoting the winner to production, democratizing AI learning across the team.

• The Intern Framework for AI Safety: Treating AI agents like interns by asking “What would you train them to do before giving them tools to lock users, wipe machines, or take down websites?” Codifying playbooks and implementing infrastructure-as-code for governance.

• Multimodal AI for Brand Protection: Using AI to analyze screenshots and HTML of potential brand infringement sites, scoring violations 0-100, and automating responses while maintaining safety checks and keyword filters.

• Data Governance and Residency Challenges: The balance between giving AI all the data for training versus careful sanitization, especially under GDPR requirements in the UK/Europe, where data categories in breaches must be explicitly reported.

• Enterprise Knowledge Management: Why pointing AI at entire documentation corpora produces confused answers, and the need for curated, well-structured, concise documentation—learning that less is more for both context and processes.

• Creating Space for Shift-Left Work: How automating 20% of alert triage effectively adds a team member’s capacity back, reducing cognitive load and allowing focus on prevention over response, moving from security theater to impactful work.

• Building Weatherproof, T-Shaped Teams: Gary’s philosophy of creating generalists who aren’t tied to specific technologies, encouraging experimentation with tools that don’t scale costs, and maintaining backlogs without creating team burnout.

Subscribe now

The approach Gary describes aligns perfectly with Panther’s AI-powered capabilities—automatically handling the repetitive alert triage and enrichment work that Gary emphasized as essential for scaling lean security teams, while preserving human oversight for critical decisions. By automating the pattern matching, data correlation, and initial investigation that LLMs excel at, security teams can focus on the preventative work and strategic initiatives that truly reduce risk. Learn more about Panther AI and how we’re building the AI-first SIEM that gives security teams their time back.

Recent Posts

More Episodes

Subscribe now

2025 was the year AI in security operations moved from ambitious predictions to production reality. The conversation shifted from “will AI work in the SOC?” to “how do we architect these systems?” and, more sobering, “what happens when we get it wrong?” We saw AI agents handling alert triage, watched the first documented AI-orchestrated espionage campaign unfold, and learned hard lessons about context engineering, guardrails, and operational complexity.

This year, we were drawn to writing from analysts and practitioners wrestling with implementations, researchers documenting risks, and experts explaining market shifts. As we head into the holiday break, here’s our essential reading from 2025 on AI-first security operations—posts that shaped the conversation, challenged the industry, and influenced how we build moving forward.

The Industry Perspective

SACR AI SOC Market Landscape For 2025 by Francis Odum

This 2025 report provides a rigorous evaluation of 13 leading AI SOC vendors to help security leaders distinguish marketing claims from meaningful technical capabilities. Building on the foundation of Odum’s 2024 exploratory research, the report provides a practical decision framework and defined architectural models to guide organizations through the phased adoption of agentic security automation.

Decoupled SIEM: Where I Think We Are Now? by Anton Chuvakin

This 2025 analysis by Anton Chuvakin examines the ongoing tension between “decoupled SIEM” architectures and the industry’s shift toward tightly integrated, AI-powered platforms. Building on his earlier debates regarding security data lakes, the post argues that while federated search and modular components offer a “romantic ideal,” the practical simplicity of unified platforms will likely see them “reign supreme” in the coming years.

Why Agentic AI Startups Will Struggle Against Cybersecurity Incumbents by Nick Heudecker

This 2025 analysis by Nick Heudecker explores the significant “brick wall” facing agentic AI startups as they attempt to disrupt the SOC. The post argues that while startups focus on custom-tuned models, real differentiation comes from the massive volumes of telemetry data already owned by industry incumbents such as CrowdStrike and Palo Alto Networks. Ultimately, Heudecker suggests that startups without a proprietary data advantage risk becoming mere “features” of the giants they intended to replace.

Technical Architecture & Implementation

Model Context Protocol has prompt injection security problems by Simon Willison

Willison sounded the alarm on MCP security risks as early adoption accelerated, introducing the concept of the “Lethal Trifecta”: access to private data, exposure to malicious instructions, and the ability to exfiltrate information. This became required reading for anyone implementing MCP in security contexts, and the vulnerabilities he warned about materialized throughout the year, with 1,000-2,000 exposed MCP servers found without authentication.

Taking Flight with Goose and Panther MCP by Tomasz Tchorz and Glenn Edwards

This blog explores how Block is democratizing detection engineering by integrating its open-source AI agent, Goose, with the Panther MCP server to automate complex security workflows. By enabling natural language-to-rule generation and automated testing, the integration allows non-specialist engineers to contribute high-quality, production-ready security detections that were previously reserved for niche experts.

Security Takeaways from 2025 AI Engineer World’s Fair by Matt Maisel

Maisel bridged the AI engineering and security communities better than anyone, capturing insights from the AI conference that defined the year’s technical direction. His key observation was that “the industry’s focus is moving beyond the model itself and toward the broader systems in which agents operate”, which explained the shift we saw from foundation model discussions to context engineering and agent orchestration.

Risks & Reality Checks

The Dark Side of LLM-Powered Security Automation by Aryan D

This post delivered a balanced treatment of AI security automation risks, covering indirect prompt injection, insecure output handling, and automation bias with technical specificity. Aryan’s warning that “security automation magnifies whatever you plug into it” resonated as incidents accumulated, and the post served as a practical checklist for teams deploying AI agents in production.

Disrupting the first reported AI-orchestrated cyber espionage campaign by Anthropic

Not a blog post but an incident disclosure that changed the conversation. Anthropic documented a Chinese state-sponsored group using Claude for 80-90% of their attack operations against approximately 30 global entities. The AI executed “thousands of requests, often multiple per second,” with sophisticated operational security measures. This validated what many suspected, but few had documented: attackers are adopting agentic AI faster than defenders.

From Detection at Scale

We published several posts in 2025 that tried to push the conversation forward on AI-first security operations.

• The Agentic SIEM introduced the vision of AI agents as “analysts with impressive memories who never need coffee.”

• MCP: Building Your SecOps AI Ecosystem broke down the paradigms and tradeoffs of implementing MCP servers in the SOC.

• The Cursor Moment for Security Operations reinforced AI as a powerful assistant rather than a carte blanche replacement for human intuition.

• The Data Your AI-Powered SOC Needs introduced the four-layer context engineering framework for powering SecOps AI agents.

• The State of AI in Security Operations: 5 Patterns That Defined 2025 synthesized what we learned through the various podcasts and posts.

We’re taking a break for the holidays and will be back in January with fresh perspectives on where AI-first security operations are heading in 2026. In the meantime, if you haven’t caught up on this year’s essential reading, there’s no better time than a quiet week between Christmas and New Year’s!

Happy holidays from the team at Panther!

Our conversation explores his perspective that 40-50% of security work isn’t tied to concrete threat models, why detection observability should precede prevention controls in fast-moving environments, and how AI agents will make previously tolerable security gaps catastrophically exploitable.

Slava’s zero-to-one journey at CoreWeave reveals how security leaders must prioritize when resources are constrained, and the business is moving at breakneck speed. His framework for threat-model-driven work, his mandate that the new detection platform be “AI-first from the get-go,” and his work on host integrity from firmware through userspace offer concrete examples for practitioners building similar programs. This conversation cuts through abstract security principles to focus on implementation: what to build first, how to justify security investments through threat models, and why the age of AI agents fundamentally changes the calculus on security debt.

Topics Covered

• Building Security from Zero to One: Slava’s experience joining CoreWeave and the process of bootstrapping a security program at a hyper-growth AI infrastructure company.

• Observability vs. Prevention: Why establishing deep security observability and forensic capabilities is often less intrusive and more critical than rolling out heavy-handed prevention controls early on in a fast-moving environment.

• The “Threat Model” Problem: Slava’s hot take that 40-50% of security work is not done in relation to a concrete threat model, often driven by a culture of chasing “flashy” projects over solving complex, unglamorous problems.

• Host Integrity at Scale: How CoreWeave verifies software provenance and integrity from the firmware level up to the user space, treating the boot process as a single verifiable model.

• AI Agents & Technical Debt: How the introduction of AI agents into the enterprise will make historical technical debt (like over-provisioned access or exportable bearer tokens) unforgivable and immediately risky.

• LLMs for Engineering Rigor: Using LLMs to strip the “fluff” from engineering design docs to force engineers to expose their true human intuition and local context, rather than just generating boilerplate content.

• The AIUC-1 Standard: An overview of Slava’s contribution to the AIUC-1 standard for AI agent insurance, focusing on determining if an agent’s software provenance and environment make it “insurable”.

• The Evolution of the SOC: The shift toward “AI-first” detection platforms and why the role of the traditional analyst is evolving into end-to-end detection engineering, where manual log analysis is replaced by engineering reliable detection code.

Subscribe now

The transformation Slava describes aligns with Panther’s AI-powered capabilities–automatically handling the initial analysis that Slava emphasized as critical for reducing investigation time, while maintaining the human-in-the-loop validation. By automating the pattern matching and correlation that LLMs excel at, security teams can focus on the threat modeling and strategic security decisions that require human expertise. Learn more about Panther AI and how we’re building the AI-first SIEM for the modern SOC.

Recent Posts

More Episodes

Subscribe now

2025 was a pivotal year for AI in security operations, where agent deployments transitioned from cautiously optimistic experiments to operational realities driven by excitement and mandates to adopt AI. This rise was characterized by autonomous alert triage, threat hunting across vast security data, and intelligent detection tuning. While teams are still striking the right balance between human and agent decision-making, nearly everyone has realized the early benefits of increased efficiency, team capacity, deeper knowledge, and productivity gains enabled by AI.

2025 delivered what enterprises needed: AI models that are simultaneously cheaper and smarter, providing continuous upgrades and economies of scale throughout the year. Frontier models like Claude 4.5 and GPT-5 pushed the boundaries for reasoning, tool calling, and intelligence, while expanding context windows to maintain state across complex investigations without losing critical details. Protocols like MCP connected thousands of important applications to these models, powering one of the most important recent trends in AI: context engineering. Teams can now easily reach across the IT, ops, and security stack to easily orchestrate response and manage incidents, powered by AI.

There was also a rise in technical security teams building their own SOC agents to serve bespoke internal use cases and augment vendor capabilities. As security vendors introduced their MCP servers, and tools like Claude Code became standard in the enterprise, the barrier to entry dropped significantly. Security teams can now scale at the pace of AI innovation. Sophisticated AI capabilities are accessible to every security team that’s ready to adopt them.

As we navigate this exciting technological shift, a few patterns have become clear:

1. Context engineering is the key to effective agents

2. Agent accuracy is preferred over speed, but both are critical

3. Data privacy is table stakes for commercial AI solutions

4. Autonomy with human-in-the-loop balances productivity with safety

5. Focused agents are preferred for specialized security workflows

Context Engineering Powers Effective Agents

AI in security operations is fundamentally a context problem, requiring broad, deep knowledge of an organization’s people, technology, history, and threat models. When agents can access diverse telemetry (e.g., to understand the criticality of an asset), analysis accuracy dramatically increases. Without this foundation, even leading models deliver shallow results that create more work than they save, which is every security team’s worst-case scenario. Agents need the same access to data and tools as their human counterparts so they can continue complementing one another’s strengths.

Consider a data exfiltration scenario that security teams often encounter: an alert fires when an employee downloads a large volume of files from a sensitive internal repository. The alert tells us what happened, but critical questions immediately arise to find the why: Is this employee in a role that normally accesses this data? Was this a personal machine? Have they exhibited other suspicious data access patterns recently?

Traditional SIEM workflows require analysts to manually gather surrounding information through separate queries, forcing them to context-switch and rebuild their mental model of the investigation with each query. Effective agents encode the organizational context needed to answer these questions—pulling employee profiles from identity providers, applying user behavior analytics to detect anomalies, referencing location and device profiles, and following detection-specific runbooks that guide scenario analysis. Agents provide efficient tool calling and context management, making only the necessary queries to test hypotheses and presenting judgment for final approval.

Teams that invested early in data lakes and structured security data pipelines built exactly this foundation. They can now feed agents enrichments, historical patterns, and organizational context, transforming a 30-minute manual investigation into a 2-3-minute agent-assisted analysis.

While agents have transformed how teams automate context gathering, teams have strong opinions about expectations for analytical and reasoning capabilities, especially when precision is required. The challenge is ensuring they reason correctly about what that context means.

Accuracy > Speed

A clear expectation emerged in 2025: security teams prefer accuracy over speed. “I’d rather it take longer and be right” became a common refrain, with current expectations settling around five minutes or less for alert triage and investigation tasks. This reflects the reality that proper context gathering takes time, and agents need to query multiple systems, correlate signals, and apply organizational context before reaching conclusions. Security teams cannot afford to draw the wrong conclusions due to an incorrect tool call or incomplete context. The advantage of AI agents isn’t purely about instant responses; it’s about compressing what used to take 30 minutes of manual work into a few minutes of agent-orchestrated analysis.

Transparency in how agents reach conclusions has become equally critical for production deployments. Agents that don’t show their reasoning or present conclusions without attribution to specific evidence or tools waste analyst time rather than amplify it. When an agent says “this alert is a false positive” without explaining why, the analyst must either blindly trust the conclusion or repeat the entire investigation to verify it. Both outcomes erode trust and create friction. Agents gaining traction in production environments expose their reasoning, show which tools were called and what data was retrieved, and make it easy for analysts to verify, challenge, or follow up on any conclusions. Accuracy and transparency aren’t separate requirements—they’re two aspects of the same fundamental need: agents that security teams can trust to make increasingly autonomous decisions.

Data Privacy Is Non-Negotiable

Security operations teams have made their data privacy expectations unambiguous in 2025. Both the security telemetry fed into agents and the analytical conclusions coming out must remain under strict organizational control. Security leaders closely scrutinize solutions that require fine-tuning or model training to be effective, refusing to send proprietary data to third parties or create new exfiltration risks in the name of security automation. The synthesized insights that agents produce—correlations between alerts, user behavior patterns, threat actor attribution—often carry more strategic sensitivity than any individual log event.

This requirement shapes the entire approach to agent implementation. Teams want to leverage AI capabilities without sending sensitive data for model training, without building dependencies on models trained on their proprietary information, and without creating new compliance headaches. The good news is that zero-shot and few-shot capabilities in frontier models have reached the threshold where fine-tuning is genuinely unnecessary for most agentic workflows in the SOC. Agents can be effective through prompt engineering, tool access, and retrieval-augmented generation rather than requiring custom model training. The data stays in your data lake, the context is assembled at inference time, and the agent reasons over it without any information leaving your control.

Autonomy with Human-in-the-Loop

The autonomy conversation in security operations has matured significantly in 2025. Early reactions swung from banning AI tools entirely to unrealistic expectations that agents would replace Tier 1 analysts. Teams have settled into a more pragmatic model: AI-assisted humans with increasing levels of autonomy based on confidence and risk. The goal with agents is to automate the repetitive grunt work of context gathering that consumes valuable analyst time, but is difficult to automate with deterministic automation. Agents can now handle the initial alert assessment, dynamically adjust priorities based on context, and enrich alerts with threat intelligence before an analyst ever sees them. This progression happens in phases: the “crawl” phase involves simple enrichment/summarization, the “walk” phase involves agents applying reasoning models to make judgments about alerts, while the “run” phase extends that reasoning into automated containment and remediation actions for high-confidence scenarios.

The human role is fundamentally shifting from assessment to oversight. Traditionally, analysts spent 15 to 30 minutes per alert gathering context and deciding next steps—a time-consuming process that agents now handle far more efficiently. The analyst’s interaction moves upstream: instead of investigating every alert from scratch, they validate the agent’s work, provide additional context when the agent escalates uncertainty, and focus on the complex cases that genuinely require nuanced human judgment. At the highest level of autonomy, analysts transition from reviewing individual alerts to managing a team of agents, auditing their output weekly or monthly rather than operating in a constant triage cycle. The human remains in the loop, but the loop itself has changed—less time triaging alerts means more time improving detection logic, refining agent workflows, and addressing the novel threats that agents correctly escalate.

This balanced approach of autonomy with guardrails has emerged as the most successful path forward among CISOs and security practitioners. Agents handle the repetitive, time-intensive work of context gathering and initial assessment, freeing analysts to apply their expertise where it matters most. The productivity gains are substantial, but the model only works when agents earn trust through transparency, accuracy, and knowing when to escalate rather than conclude.

Focused Agents

The most successful agent deployments in 2025 followed a pattern that mirrors how human security teams actually operate: specialized agents working together, rather than a single generalist agent attempting to replace an entire analyst. The “Uber agent” that can handle every aspect of security operations doesn’t exist yet, and teams that have tried to build one have found that generalization comes at the cost of effectiveness. Instead, organizations are deploying focused agents with narrow, well-defined responsibilities—such as a CloudTrail analysis agent that specializes in AWS activity patterns or a detection-tuning agent that optimizes rule performance based on noisy alerts. Each agent becomes an expert in its domain, and collectively they make the SOC more effective.

This architecture enables transfer learning and feedback loops across the entire detection and response lifecycle. When the triage agent repeatedly escalates a specific alert pattern as benign, the detection tuning agent can adjust thresholds or add filters. These specialized agents working in concert create a system that learns and improves continuously, with each agent contributing expertise that compounds across the team. We can now deploy specialized agents, with coordination overhead handled programmatically rather than through meetings and handoffs.

Looking Ahead

The contrast between where we started in 2025 and where we ended is remarkable. Security teams entered the year cautiously experimenting with AI capabilities, unsure of what was hype and what was real. We’re closing the year with agents deployed in production, handling thousands of alerts, running investigations 80% faster, and proving their value in measurable ways. The foundational investments in data lakes, structured telemetry, and detection-as-code are paying dividends, making agents effective rather than just impressive demos.

The future of security operations is a blend of human expertise with agent capabilities. The teams succeeding are amplifying analyst knowledge by automating the repetitive context gathering, initial assessment, and routine response actions that consumed so much time. They’re building focused agents that work together like a well-coordinated team, with humans managing the agent team rather than drowning in alert queues. They’re prioritizing accuracy and transparency over speed, maintaining strict data privacy, and implementing autonomy with appropriate guardrails. The shift from traditional SIEM to agent-driven security operations isn’t complete, but the path forward is clear!

Thanks for reading Detection at Scale. If you found this valuable, please share it with your colleagues who are exploring AI-powered automation in security operations!

Share

Cover photo by Christopher Burns on Unsplash

Related AI Posts in 2025

Ken’s journey from healthcare security at Tempest to securing credit card data at GreenSky provides a unique lens on how security operations have evolved from basic alerting to AI-enhanced investigation. His emphasis on protecting the crown jewels first, embracing automation pragmatically, and maintaining healthy skepticism about AI’s limitations offers a refreshing counterpoint to the “AI will solve everything” narrative that dominates vendor pitches. This is a conversation about real-world implementation challenges, the changing role of security analysts, and why the fundamentals still matter even as the tools become more sophisticated.

Subscribe now

Key Takeaways

1. Prioritize Crown Jewels and Work Outward: Ken emphasizes starting with what matters most—identifying your organization’s most critical assets (such as credit card data at GreenSky) — and building security controls outward from there. This focused approach prevents teams from drowning in generic alerts and ensures resources are allocated where they’ll have the most significant impact on actual risk reduction.

2. AI Enables the “Single Pane of Glass” Through Context, Not Dashboards: Rather than forcing analysts to context-switch between multiple tools, AI acts as intelligent middleware, pulling data from EDR, SIEM, email security, and identity platforms into a cohesive alert context. This reduces investigation time dramatically by having AI assemble the complete picture before an analyst even starts looking, transforming what used to take 30+ minutes into near-instant contextual awareness.

3. Detection Strategy Needs Nuance Beyond MITRE Framework Coverage: While MITRE ATT&CK provides valuable guidelines, Ken cautions against the audit-driven mentality of “we need an alert for everything.” Not every technique applies to every organization, and trying to alert on everything leads to analyst burnout. The more intelligent approach involves understanding your threat model, implementing compensating controls where possible, and focusing detections on what actually matters in your specific environment.

4. Human Judgment Remains Essential Despite AI Advances: Ken draws a critical distinction—AI excels at pattern matching and data analysis but cannot determine intent, which is fundamental to security analysis. While AI can flag that someone accessed sensitive data from an unusual location, only humans can decide whether it’s a legitimate business trip or a credential compromise. This understanding should shape how teams deploy AI: as a force multiplier for analysts, not a replacement.

5. Audit Your Controls Because Tech Debt Compounds Security Risk: Ken shares a hard-earned lesson: security controls established years ago often drift as ownership changes hands, configurations evolve, and cloud environments grow more complex. Regular auditing of access control lists, security group rules, and other foundational controls is essential because that “tiny little crack” in your defenses often emerges from accumulated changes no single person fully understands anymore.

The practical AI implementation Ken describes reflects Panther’s approach to enhancing security operations. Panther AI handles the context gathering that Ken emphasized, providing that “single pane of glass” through intelligent enrichment rather than dashboard sprawl. This allows your analysts to focus on validating AI judgment rather than spending time manually pivoting across multiple tools. Learn more about Panther AI and how we are keeping human expertise at the center of security operations.

Tyler’s path from accidentally enumerating a healthcare database as a Java developer to leading one of the more innovative SecOps teams around provides good context for this conversation. His team has built multiple specialized agents, including “SAGE” (Security Analysis and Guided Escalation) for phishing, account takeover, and an incident response automation that runs entire IR workflows via Slack. The results are notable: a 90% reduction in incomplete post-incident review action items and engineers spending their time building rather than working on tickets.

This conversation gets past the AI hype to discuss the practical realities of building production-grade security agents. Tyler talks about everything from the “bronze-silver-gold” approach to automation maturity, to managing context rot in LLMs, to why the industry needs to start measuring different things. The most helpful part is understanding why starting with specific, high-volume use cases and gradually expanding works better than trying to automate everything at once.

Subscribe now

Key Takeaways

1. Moving to All-Engineering Teams Enables Better AI Outcomes: By dropping the traditional tier 1-2-3 analyst model and staffing entirely with security engineers, FanDuel created a team that can build and maintain its own agentic AI systems. This change unlocked the ability to continuously improve automation rather than operate it, shifting from “working tickets” to “building systems that work tickets.”

2. Context Rot is the New Challenge in Agent Design: Just as analysts can be overwhelmed with too much information, AI agents suffer from “context rot” when given excessive data. The key is finding the right amount of information—enough signal for accurate decisions without overwhelming the model’s attention. This requires careful thought about what data enters the context window and in what order, similar to how you’d organize information for a human analyst.

3. The Bronze-Silver-Gold Maturity Model for AI Automation: Start with bronze (human-in-the-loop validation, no automated closures), move to silver (automated closures with some manual intervention), and eventually reach gold (fully autonomous triage). This phased approach lets teams build confidence, identify missing context, and add necessary integrations step by step rather than attempting full automation right away, which usually fails.

4. Runbooks are Now AI Agent Instructions, Not Human Documentation: The traditional detection runbook has evolved from documentation for human analysts to specific instructions for AI agents. While basic investigation steps should be in the agent’s system prompt, runbooks should focus on the context and investigation patterns unique to each detection. This is where prompt engineering becomes a critical security skill.

5. Incident Response Automation Through Slack Changes Things: FanDuel’s IR automation handles entire incident response workflows with simple Slack commands—automatically creating channels, inviting stakeholders, spinning up Zoom bridges with recording enabled, generating real-time documentation from transcripts, and assigning action items to specific team members in Confluence. This solved the problem of incomplete PIR action items and significantly reduced post-incident administrative work.

The transformation Tyler describes aligns with where Panther is going with our AI-powered triage capabilities. Our agent automatically handles the assessment layer Tyler discussed, gathering context and presenting risk-based summaries so your team can focus on investigation and response rather than manual enrichment. Learn more about Panther AI and how we’re helping teams make this transition.

MCP continues to play an essential role for technical teams that want to orchestrate multiple capabilities rather than use them in isolation. Consider a typical alert triage scenario: your agent needs to check your internal runbook database, query your SIEM for related activity, pull employee context from your HR system, and correlate findings from your threat intelligence platform, all while maintaining conversation context in your team’s Slack channel. This is where MCP’s standardized approach to tool integration creates leverage, allowing teams to combine pre-built agents from various vendors with their own internal tooling and custom MCP servers without writing extensive integration code for each connection.

The pattern emerging from organizations implementing custom security agents breaks down into three distinct approaches: conversational interfaces embedded in communication platforms like Slack, orchestrated multi-agent workflows coordinated through tools like n8n, and enhanced developer environments that bring security context directly into coding assistants like Claude Code or Cursor.

In this post, we’ll explore how these custom agents work, what makes them effective, and the practical challenges teams are encountering as they build this connective tissue between vendor capabilities and internal operations.

Subscribe now

Hear from Practitioners About Real-World Implementations!

This Thursday, I’m hosting a webinar featuring security practitioners from companies like OpenTable and Block who run MCP in production settings. They’ll walk through their implementations, share what’s working, and lessons from workflows they’ve automated. If you’re building custom agents, this conversation will save you weeks of trial-and-error.

Register for “How MCP Helps Security Teams Move Faster”, Thursday, Nov 20th, 2025!

The Custom Agent Opportunity

Vendor-built AI capabilities excel at addressing common security workflows that span many organizations, like analyzing alerts and synthesizing investigation findings. These capabilities become even more powerful when connected to organization-specific context and internal systems that vendors can’t access or don’t know exist. Your internal runbook database, your custom asset inventory system, your team’s historical incident data, or your specific escalation logic represents the connective tissue that transforms generic AI capabilities into precisely tuned automation for your environment.

Consider a typical alert triage scenario. A vendor-provided triage agent can analyze the technical indicators of an alert and assess its severity using general threat intelligence. But it can’t (always) check whether this user recently submitted an IT ticket about unusual activity, whether this asset is scheduled for decommissioning next week, whether similar patterns triggered false positives in your environment last month, or whether this matches the specific attack scenarios your threat modeling identified as high-risk for your industry. Custom agents fill these gaps by orchestrating multiple capabilities together and combining vendor-built analysis with internal context gathering and organization-specific decision logic.

MCP’s standardized approach to tool integration is what makes this orchestration practical for technical security teams. Rather than writing custom integration code for each vendor API and maintaining separate authentication mechanisms for every tool, teams can expose capabilities through MCP servers and let agents discover and use them through a common protocol. This standardization reduces the integration burden from an N×M problem (where every agent needs custom code for every tool) to a single build and reuse across different agents and workflows. The result is connective tissue that’s maintainable by security engineers rather than requiring dedicated integration development teams.

Common Patterns of Custom Agents

Custom agents manifest in multiple distinct forms, each optimized for different security workflows and team interaction patterns. Understanding which pattern fits your use case determines both implementation complexity and operational effectiveness.

Pattern 1: Conversational Interfaces (Slackbots and Chatbots)

Conversational agents embedded in communication platforms like Slack or Microsoft Teams bring AI capabilities directly into the flow of security operations. These agents respond to natural language queries in channels or threads, maintaining conversation context while orchestrating multiple MCP servers to gather information and execute actions. The interface is familiar. Just @ mention the bot and ask a question.

The power of conversational agents lies in their ability to meet analysts where they already work. During alert triage, an analyst can ask “what’s the context around this AWS console login from Romania?” and the agent orchestrates multiple lookups: checking if the user has traveled recently (HR system MCP server), reviewing their normal login patterns (SIEM MCP server), examining recent access modifications (IAM MCP server), and correlating with threat intelligence (vendor MCP server). All of this happens in a Slack thread where the entire team can see the investigation, ask follow-up questions, and collaborate on the response. The agent becomes a force multiplier that eliminates context switching while maintaining natural team communication patterns.

Implementation typically involves connecting a Slackbot application server (for sending and receiving messages), your SIEM MCP server (for querying security data), and any custom MCP servers you build to expose internal systems. The agent uses an LLM to understand queries and determine which tools to invoke, then formats results back into conversational responses. Thread context provides automatic conversation history, and the human-in-the-loop design ensures analysts maintain control over decisions while the agent handles the mechanical work of gathering context and synthesizing findings.

Pattern 2: Orchestrated Workflows (n8n and Low-Code Platforms)

Multi-agent workflows coordinated through platforms like n8n represent a different approach: rather than conversational interaction, these implementations encode complex security processes as visual workflows that orchestrate multiple specialized agents together. Each node in the workflow might represent a different agent capability: one for enrichment, another for severity scoring, and a third for determining escalation paths. The workflow tool provides the orchestration logic, error handling, and state management between them.

This pattern excels at automating repetitive, multi-step processes that must occur consistently and reliably. When a detection fires, an orchestrated workflow can immediately trigger an enrichment agent that gathers context from multiple sources, pass those findings to a specialized analysis agent that assesses severity and identifies similar historical incidents, then route to either automated remediation (for known-safe scenarios) or analyst review (for ambiguous cases) based on explicit business logic. The visual workflow makes this automation auditable and maintainable, and security engineers can see exactly what happens at each step, modify the logic without touching code, and add new agents or tools as capabilities evolve.

The n8n ecosystem has embraced MCP through community-built nodes that allow workflows to connect to any MCP server as a tool. This means a single workflow can orchestrate agents powered by different LLM providers, call multiple vendor MCP servers for different data sources, and invoke custom MCP servers for internal systems, all through a standardized interface. The workflow platform handles the complexity of chaining these calls together, managing failures and retries, and providing observability into how the automation executes. Teams can start with simple linear workflows (gather context → analyze → notify) and gradually add sophistication (parallel enrichment, conditional branching, human approval steps) as they learn what works for their environment.

Pattern 3: Enhanced Developer Tools (Claude Code, Cursor)

The third pattern integrates MCP directly into AI-powered coding assistants, bringing security context and capabilities into the detection development workflow. Tools like Claude Code or Cursor can connect to MCP servers that expose your SIEM APIs, allowing the AI assistant to query your actual security data, understand your log schemas, and test detection logic without leaving the development environment. This tight integration between AI assistance and security infrastructure accelerates the entire detection engineering lifecycle.

Consider the typical detection development process: an engineer focuses on a specific threat model, researches how it manifests in logs, writes detection logic in the SIEM’s query language, deploys it to a test environment, runs it against historical data, tunes for false positives, and iterates through multiple rounds of refinement. With MCP-enhanced coding assistants, this process compresses significantly. The engineer can describe the threat scenario in natural language, and the assistant queries the SIEM MCP server to retrieve sample logs showing how this activity appears in the environment, generates detection logic tailored to the actual log schema, and validates the syntax before deploying anything to production.

This pattern particularly benefits teams practicing detection-as-code, where rules are developed in version-controlled repositories rather than directly in SIEM interfaces. The coding assistant understands both the security context (what you’re trying to detect) and the technical implementation details (your data structure, query syntax, deployment process) by combining its training with real-time access to your security infrastructure through MCP. The result is faster iteration, fewer bugs, and detection rules that account for your environment’s specific characteristics rather than being adapted from generic examples.

During the day, I’m founder & CTO at Panther, where we’re building an AI SOC platform with an open-source MCP server that provides all the benefits described above! Whether you’re orchestrating agents in n8n, building detections in Claude Code, or using our AI copilot, the same MCP interface gives you programmatic access to your security operations. We open-sourced the server because this connective tissue layer should be transparent for teams building custom automation. Check it out on GitHub!

What’s Working (and What’s Still Rough)

Early adopters report significant time savings on routine tasks. Context gathering that previously required 15 minutes of jumping between dashboards now completes in under two minutes through conversational agents. Alert enrichment that analysts manually applied to every suspicious event now runs automatically via orchestrated workflows. Detection engineers who spent hours researching log schemas and testing queries now iterate faster with AI assistants that understand their specific data structure. But the real value might be institutional knowledge capture. Senior analyst investigation techniques encoded into agent prompts and tool design become reproducible workflows that junior analysts can leverage immediately, and runbook knowledge transforms from documentation into executable logic that gets tested every time an agent uses it.

The most critical insight from successful implementations is that agent effectiveness depends less on model sophistication and more on thoughtful tool design. Narrow, composable tools consistently outperform kitchen-sink capabilities. Rather than building a single “query_siem” tool that accepts arbitrary queries, practical implementations create focused tools like “get_user_login_history” and “check_asset_vulnerabilities” that each do one thing well and compose naturally together. Tools that return analysis-ready context rather than raw log lines dramatically improve agent performance, and production implementations start with read-only operations before carefully expanding to write capabilities with explicit approval steps and comprehensive audit logging.

The rough edges are real and require honest acknowledgment. Agents with access to 20+ tools sometimes struggle with tool selection; context window limits can become binding constraints during lengthy investigations; and error handling remains challenging when agents misinterpret data or make incorrect assumptions. Human review stays critical for high-stakes decisions. Security considerations around credential management, audit logging, and supply chain concerns for community-built MCP servers require the same disciplined approach you’d apply to any privileged access. These aren’t showstoppers, but they require thoughtful implementation rather than naive deployment.

Moving Forward

Custom AI agents represent a fundamental shift in how technical security teams approach automation, moving from consuming standalone AI features to building connective tissue that orchestrates multiple capabilities together. MCP’s standardized approach to tool integration makes this accessible to security engineers rather than requiring dedicated development teams, and the three patterns we’ve explored (conversational, orchestrated, developer-focused) provide clear templates for different security workflows.

The teams building custom agents now are learning which tools pair well, how to design agents that scale human judgment, and which operational patterns work at scale. This experimentation is valuable precisely because security operations are inherently organization-specific. Your tools, your infrastructure, and your team structure are all unique, and generic AI features can only take you so far.

If you’re considering building custom agents for your security operations, learn from teams who are already running these implementations in production. Our webinar this week features practitioners who will walk through their architectures, share what surprised them, and demonstrate actual workflows they’ve automated. Moving forward, the security teams operating at scale won’t be the ones with the best individual tools, but the ones who orchestrated them most effectively.

Thanks for reading Detection at Scale. If you found this valuable, please share it with your colleagues who are exploring AI-powered automation in security operations!

Share

Related Reading

Cover Photo by Shubham Dhage on Unsplash

Subscribe now

Today, you write SIEM queries. Tomorrow, you’ll supervise an agent who writes them for you. AI agents represent the ultimate evolution of security automation, building dynamic, nondeterministic investigation paths that traditional playbooks can’t anticipate.

Consider triaging a phishing email alert. You identify the delivery method, determine who received it, validate if anyone clicked the link, hunt for signs of credential theft, and track lateral movement if credentials were compromised. The typical response flow involves multiple manual queries across various systems, but with agents, this transitions to typing a series of coherent, natural-language questions. But how does the agent know where to look? How does it understand the query syntax for accessing the data to answer the question? How does it access your threat intelligence feeds or ticketing systems for context? The answer is tool calling.

Tools are structured interfaces that serve as the middleware layer between an agent’s reasoning and the execution of tasks in your various security platforms. A tool might be as simple as search_alerts with a clear function that accepts time ranges and filter conditions, or as sophisticated as indicator_pivot that chains multiple queries across systems before returning an answer. The agent uses these tool definitions to properly format queries, make correct API calls, and access the appropriate data sources.

The result is a powerful virtual assistant that can perform everyday security analysis tasks that require interaction with data and tools, quickly triaging alerts, confirming hypotheses, and synthesizing answers across large, complex data sets.

In this post, we’ll examine how tools define agent capabilities, how they decide which tools are most appropriate to call based on the situation, and the constraints you should know while using agents. These concepts will help you become more effective and productive by delegating work to AI agents.

Quick note: I’m the Founder/CTO at Panther — a SOC platform helping security teams scale with AI agents and as-code workflows. In this post, I’ll use examples of how Panther powers these agent workflows in leading SOCs. If you want to see the product behind the ideas, you can check it out here.

How Tools Define What Your Agent Can Actually Do

We all have experience using AI to summarize text, spot patterns, or answer questions, thereby expediting analysis. But for agents to carry out the work delegated to them, they need tools to access real-world context or to perform tasks dynamically. Remember, LLMs (Large Language Models) are bound by their training data, and tools serve as a bridge to break free from that isolated knowledge. By providing agents the same access to security tooling as human security analysts, they can execute similar work in the SOC and become a force multiplier on our teams. But what do tools look like in practice? How are they defined? Let’s explain using an everyday use case: Alert Tuning.

Every day, security teams triage and resolve alerts that flag high-risk behaviors. As teams process these alerts, they interpret what happened, gather surrounding context, and make a judgment call. When alerts are noise, the analyst determines which part of the rule failed and documents the findings in a ticketing system like Jira to prevent future alerts.

What would it look like to automate that process with an AI agent? It would need access to read the detection logic, review the created alerts, query log data, and then file a ticket. Let’s see this in action with Panther’s AI copilot:

Let’s dissect what’s happening here. The user asks a question about tuning CloudTrail alerts from the last three weeks. The agent calls a tool to list alerts for that timeframe, discovers 18 alerts, then fetches each detection rule to understand why those alerts originated. Once it has this external context, it reasons about which rules need tuning and which are working correctly, and produces an analysis for how fewer or more accurate alerts could have been generated.

This agent revealed that most rules appear to be legitimate administrative actions, which is a great observation and creates solid candidates for tuning. One particular piece of feedback was identifying a redundant rule that caused duplicate alerts and increased work for the team:

Tool Definitions

Tools accomplish a specific task and contain a description that explains when they should be used and the expected output. This combination of tools, each with clear boundaries and purposes, combines into an agent that performs a specific role on your security team. Tools are defined using a name, description, and a set of parameters:

These tools are then loaded into the agent’s system prompt, and once an agent decides to use a tool, it passes parameters to obtain the correct output. For example, when an analyst asks, “Show me all high alerts from yesterday,” the agent parses that request, recognizes that list_alerts is the appropriate tool, and constructs a function call with severities=[”HIGH”] and appropriate date parameters.

Understanding how agents decide which tools to use can help ensure they carry out delegated tasks correctly. Let’s learn how to influence their decisions in the right ways.

Share Detection at Scale

How Agents Decide Which Tools to Use (And How to Guide Them)

There are two ways agents choose which tools to use, and understanding the distinction changes how you approach your role in the process.

The first is direct invocation, where you use the agent as an execution layer. You say, “list alerts from failed logins in the last hour,” and the agent dutifully calls the list_alerts tool with those exact parameters. You’re the director; the agent is just translating your natural language into the proper tool call.

The second is agentic invocation, where you give the agent a higher-level goal and it decides which tools to use and in what sequence, based on its reasoning about the information it needs. You say, “We need to investigate the privilege escalation alert and find any related signs of compromise.” The agent uses its available tools to check the alert details, look up the user’s recent activity, compare that activity pattern against historical baselines, and check threat intelligence for the source IP. You’ve shifted from investigator to supervisor.

The way agents select tools depends on the clarity of tool declarations, the task you’ve given them, and any constraints in your instructions. When an agent receives a task, it examines all available tools, reads their descriptions to understand what each does, and builds a mental model of how it might accomplish the goal. It follows similar reasoning you would use as an analyst. The difference is that you’ve learned investigation patterns through experience and training, while the agent learns them through the quality of your prompts and examples.

The most effective way to teach agents good tool orchestration is to show them what good investigations look like. This is called few-shot prompting, and in the context of cybersecurity agents, it’s documenting and relaying to the agent how you’d handle a case. Take a phishing investigation: “When investigating phishing, first check the sender and recipients, then find logs for anyone that clicked embedded links, and create an alert to detect any follow-on activity.” By giving the agent these example workflows, you’re teaching it the investigative rhythm and logic that should carry over to similar cases.

Now that we have covered how tool selection occurs, let’s dive into the limitations and constraints of tool use and the optimal methods for keeping agents focused.

Tool Constraints: Keeping Agents Focused and Effective

When you’re investigating an alert, you don’t start by pulling every log from every system for the past month. You scope deliberately. You ask targeted questions, you narrow your searches to relevant timeframes and systems, and you gather just enough context to reach a conclusion. When you’re dealing with security logs that can scale into terabytes across dozens of sources, this discipline is essential. The same principle applies to agents, but unlike human analysts who develop intuition about reasonable scope through experience, agents need explicit guidance about how to investigate efficiently.

Here’s what happens when agent investigations lack proper constraints. An analyst asks, “Was this authentication successful?” and the agent, seeing that it has access to a broad set of tools, decides to pull all authentication events for that user across all systems for the past week, then all network activity for those same systems, then all process execution logs from any endpoint that user touched. The agent isn’t being malicious or careless; it’s trying to be thorough. But now you’re waiting five minutes for results that should have taken 30 seconds, you’re spending tokens processing megabytes of irrelevant log data, and the agent’s working memory is cluttered with extraneous information that obscures the actual answer.

The underlying issue is that each tool call adds conversational turns to the agent’s reasoning process, and more turns create more opportunities for attention drift and degraded reasoning quality. An agent that makes 15 tool calls to answer a simple question isn’t just slow and expensive; it’s more likely to lose track of the original question or weigh irrelevant information too heavily in its final answer. The goal isn’t to prevent agents from using multiple tools when necessary; it’s to ensure each tool call is purposeful and moves the investigation forward rather than sideways.

The fix is understanding that agent focus comes from two sources: well-designed tool definitions and clear guidance in your prompts. If you’re building tools yourself, make them specific to investigation patterns rather than generic data access. Instead of a single broad search_logs tool, create focused tools like check_authentication_outcome that accepts a user, timestamp, and source system, and returns only the success or failure status. Tools with clear, narrow purposes naturally guide agents toward efficient investigations because agents don’t have to make complex decisions about how to scope their queries.

The other lever you control is the instructions and examples you provide. Be explicit about investigation scope: “Check only the past hour unless you find a suspicious pattern,” or “Start with authentication logs, only expand to network logs if you see evidence of lateral movement.” This is where AI playbooks become essential, representing the evolution from manual step-by-step procedures to encoded instructions that agents follow autonomously. Traditional playbooks told humans which buttons to click and which queries to run. AI playbooks encode the reasoning behind those decisions so agents can adapt the investigation to what they find.

For impossible travel alerts, an AI playbook might specify: “Compare authentication locations and timing. If locations are more than ~500 miles apart and there is less than 1 hour between them, check for concurrent sessions. Only query network logs if you find evidence of account takeover.” You’re teaching the agent the same investigation discipline you’d teach a junior analyst, but you have to be more explicit because the agent can’t read between the lines or apply common sense about what “reasonable scope” means. When agents have clear guidance about how to use tools efficiently, you get faster investigations, lower costs, and more reliable conclusions because the agent isn’t drowning in its own data collection.

From Writing Queries to Harnessing AI Capabilities

The shift from performing investigations manually to supervising agents is about the analyst’s job evolving to a higher level of leverage.

Understanding tools changes what it means to be effective in an AI-first SOC, because you’re no longer executing investigations end-to-end; you’re designing the guidance agents use. When you write an AI playbook that encodes how to investigate impossible travel alerts, you’re creating reusable investigative logic that handles hundreds of similar cases without your direct involvement. This is the fundamental skill that separates analysts who thrive with AI agents from those who struggle with them.

Start by picking one investigation workflow you handle repeatedly, document it as if you’re training a junior analyst, and encode it as guidance for your agent. Be explicit about which tools to use, when to expand scope, and what findings should trigger deeper investigation. Test it, refine it based on how the agent actually performs, and you’ll quickly develop intuition for what makes agents effective versus what leads them astray. The analysts who master this are learning to teach agents how to investigate effectively rather than just investigating themselves. That’s the skill that matters going forward, and it starts with understanding that tools are the interface between what agents can reason about and what they can actually do in your environment.

Thank you for reading! To get more blogs like this, subscribe below.

Subscribe now

If you are implementing the AI SOC workflows mentioned in this post but don’t want to build it from scratch, check out Panther! We provide a highly scalable SIEM with AI agents for autonomous triage, threat hunting, and natural language search. Security teams using Panther triage 80% of their alerts autonomously while keeping analysts focused on complex cases that require human judgment. Reach out to discuss bringing AI agents into your security operations. You can also send me a DM below. Thanks!

George Warbacher, Head of Security Operations at Live Oak Bank, has spent the past year bridging that gap. His journey from tinkering with Cursor and Claude Code to building production agents for his SecOps team reveals something crucial about where AI is genuinely transforming security work versus where it remains speculative. After months of late nights building agents from scratch, George developed a refined perspective on what’s hype and what’s real when it comes to AI in the SOC.

The conversation touches on everything from the technical challenges of managing agent context and memory to the broader implications of natural language interfaces replacing query language expertise to why George believes SOAR platforms face disruption. Most importantly, it illuminates a pragmatic path forward for security teams looking to adopt AI without falling prey to vendor hype or unrealistic expectations about automation replacing human judgment.

Subscribe now

Key Takeaways

1. The Real Power of AI in SecOps is Natural Language Investigation: The most immediate operational value is enabling analysts to investigate alerts using natural language instead of mastering specific query languages, platform APIs, and tool nuances. This fundamentally lowers the barrier to entry and accelerates investigations without requiring deep platform expertise.

2. Building Production Agents Requires Engineering Rigor Beyond Demos: Creating agents locally is straightforward, but making them durable enough for production use demands solving challenges like retry logic, failbacks, context management, and multi-user execution that most tutorials skip entirely. This gap explains why many organizations struggle to move beyond proof-of-concepts.

3. SOAR Platforms Face Disruption from Natural Language Automation: Just as cursor and Claude Code made building software accessible through conversation rather than coding mastery, AI agents will make security automation more accessible by replacing static playbooks with dynamic, conversational workflows that junior analysts can build and modify without extensive programming knowledge.

4. The Analyst Role is Transforming, Not Disappearing: AI will shift the role from alert analysis toward investigation and threat hunting rather than eliminating security analyst positions. Agents will increasingly handle tier 1 work, while human analysts focus on the complex investigative work that emerges from agent output rather than starting from raw alerts.

5. MCP Creates the Integration Layer Security Teams Need: The Model Context Protocol represents a significant breakthrough for security operations by enabling agents to interact with security tools through standardized interfaces. This solves the longstanding challenge of creating a true single pane of glass by letting agents orchestrate actions across disparate security platforms through natural conversation.

The transformation George describes is already happening in production at Panther, where our AI agent automatically triages alerts by gathering comprehensive context from your data lake, analyzing historical patterns, and presenting intelligent summaries in minutes instead of the 30+ manual minutes traditional workflows require. Learn more about Panther AI and our MCP server implementation.

Continued Reading

Share Detection at Scale

The SOC has a fundamental scaling problem. Not only are there too many alerts to monitor, but performing the job effectively requires high technical nuance across understanding operating systems, networks, cloud environments, attacker tactics, and the latest intelligence. Working in the SOC is also stressful, error-prone, and requires high attention to detail. Querying an incorrect timeframe, missing one event, or failing to check a related log source could wildly change the course of an investigation or introduce the organization to significant levels of risk. Precision and speed matter in security operations.

Throughout the years, security teams have tried various solutions to these problems. We adopted detection as code, built deterministic response automation to contain incidents and deeply understand alerts, and adopted data lakes to handle new scale needs.

AI agents introduce a new category of automation that can finally address the fundamental scaling constraints facing security operations. This capability shift demands changes in SIEM architecture, team expectations, and the infrastructure foundation enabling effective AI-powered security operations.

Pattern Matching

During the last two decades in cybersecurity, we witnessed several fundamental shifts—from the emergence of modern SIEMs in the early 2000s, through the introduction of EDRs in the mid-2010s and XDRs in the late 2010s, to the adoption of data lake architectures and SOAR platforms, and now the integration of AI in a new context. Yet one constant remains: humans must ultimately always determine “good” versus “bad” in alerting and security operations. Why is this? Because attempting to automate every possible attack permutation would result in inaccurate, impossibly complex, and unmaintainable security code.

Security operations teams are very sophisticated pattern matchers. When alerts come in, like when our infrastructure team adds an IAM role that can be assumed from an AWS account outside of our organization, we typically know exactly why (because we just spoke with them about it) or we have a gut feeling that “oh, this is bad.” There are too many possible conditions to predict ahead of time to avoid that alert, and hence, we are “flooded with alerts,” which has been the canonical problem to solve in cyber. But what else is a very sophisticated pattern matcher? A large language model (LLM).

Generative AI has high potential to automate most routine security operations tasks because LLMs can process vast amounts of context, instructions, tools, and data, then produce a complete analysis. When we prompt a model, every additional word (token) guides its attention to the right place. This means we can break from rigid, traditional automation and begin delegating novel tasks to AI agents, which are LLMs with carefully crafted prompts that specify personas and goals. As long as the model is given the appropriate depth and variety of context, it can perform nearly as well as a human analyst. But if it’s missing any key business or security contexts, it will naturally perform worse and be perceived as a hallucinated inferior solution.

Understanding that LLMs excel at sophisticated pattern matching opens the door to how we structure security operations workflows, moving from single-point-of-failure bottlenecks to AI agents that can operate across the entire security lifecycle.

The Multi-Agent SOC

The security team’s free time is a very fleeting resource. Between on-call rotations, incident response fire drills, and the constant pressure to stay current with new types of threats, analysts are burning out at alarming rates. The real challenge is the cognitive load of making high-stakes decisions under pressure, often with incomplete information. When you factor in the need for continuous learning, it becomes clear that throwing more people at the problem isn’t sustainable.

We need to fundamentally change how security operations work gets distributed between humans and machines, allowing analysts to focus on the strategic, creative problem-solving that humans excel at while delegating the repetitive, context-heavy tasks to AI agents.

There are several opportunities to apply agents across the lifecycle of security operations:

1. Threat Hunting and Modeling: What’s important for our organization to protect? Do we have the data to back that up? Can we find the indicators of an attack?

2. Detection Creation: What behaviors do we need to track? Which ones deserve an on-call page? What are our security significant events?

3. Incident Response: What do we do once we get paged? How do we assess/react/recover?

Let’s start with threat modeling agents, which provide a massive speed improvement in querying and understanding our vast security data that we spend large sums of time and money collecting. These agents can analyze your data using natural language and perform research on particular tactics/techniques, search for evidence of indicators, or look around to discover high-priority assets, baseline behavior, and map potential attack paths. Traditional threat modeling exercises happen quarterly at best and quickly become stale, but an AI agent can help maintain a living threat model that updates as your infrastructure changes, new vulnerabilities are disclosed, and landscapes shift.

For Detection Creation, AI agents can bridge the knowledge gap between our security team’s monitoring needs and how these get implemented as actionable rule logic. Rather than spending months or years learning specialized syntax to translate business logic into detection rules (a process that often takes weeks), agents can quickly assess your available data and unique environment characteristics and generate tailored detections that just work. This process quickly becomes a flywheel, where the more high-quality rules created and optimized, the easier net-new creation becomes. Additionally, incident response and triage agents can feed learnings into detection creation agents.

Finally, Incident Response, which often takes the most time and creates the most stress. For most security teams that haven’t yet applied AI in this area, they tend to create a playbook for a given type of alert, document a series of steps and if/else logic for handling the alert, then apply the playbook to one or many rules. The problem is that every incident has a unique context that doesn’t always fit neatly into predetermined logic trees.

AI agents can fundamentally change this by acting as triage assistants that intelligently combine the technical details of an alert with the broader business context and the identified leads during triage. Imagine an agent that automatically correlates a suspicious login attempt with recent employee departures and historical attack patterns against your industry. Instead of 30 minutes of manual research, the agent delivers a rich briefing in only 2-3 minutes: “This login attempt from Romania targeting Sarah’s account is concerning because she left the company last week, her access should have been disabled, and we’ve seen similar patterns in recent attacks against financial services companies.” The agent doesn’t always make the final decision, but it arms the human analyst with the context needed to make an informed judgment quickly.

The Platform Shift

Traditional SIEM platforms weren’t designed for the kind of rich, contextual analysis that AI agents require. Legacy systems store data in proprietary formats, limit access through rigid query interfaces, and charge prohibitive costs for the data volumes that effective AI agents need to consume. The shift toward data lake architectures creates the foundation that AI agents need to be truly effective. When security data lives in open formats in data lakes, agents can access vast amounts of historical data without the performance bottlenecks or cost penalties of traditional systems, enabling them to analyze years of data to understand normal patterns, seasonal variations, and subtle attack progressions that would be impossible with limited data retention.

The “connective tissue” enabling AI SOC evolution extends far beyond data architecture, and SIEM platforms will likely evolve to fulfill this critical role. This requires robust APIs for agent interactions with security tools, comprehensive data catalog management for handling diverse log formats, and sophisticated identity and access controls that enable agents to operate securely throughout your environment. Security data pipelines have become essential—not merely for cost optimization, but for ensuring AI agents can access clean, enriched, and properly formatted data.

Most importantly, the infrastructure needs to support “context engineering,” the practice of systematically providing AI agents with the business context, threat intelligence, and environmental knowledge they need to make informed analysis. This means maintaining knowledge bases about your assets, business processes, risk appetite, and operational procedures in formats AI agents can consume and reason about.

Evolution Determines AI Success

While humans have remained the final arbiters of security alerting for decades, AI agents can now shoulder the exhausting work of context gathering, pattern analysis, and routine decision-making that overwhelms security teams. The organizations successfully deploying AI agents are embracing architectures purposefully designed for this new paradigm. Data lakes, open formats, flexible APIs, and a robust data fabric are requirements for survival in the modern security landscape.

For security teams ready to embrace this evolution, the potential for step-function improvement is tangible and immediate. The fundamental scaling problem that has plagued SOCs—too many alerts, too much context to gather, too few analysts with too little time—finally has a viable solution to build upon. The question isn’t whether this transformation will happen, but whether your current SIEM platform can power it or will become an obstacle to progress.

AI agents can transform every aspect of the security operations lifecycle. The agent-driven SOC is being deployed today by forward-thinking teams that understand the power of combining human expertise with AI capabilities. The journey from traditional SIEM to AI-powered security operations starts with choosing infrastructure that can truly support this vision—and the time to begin is now.

Subscribe now

Thanks for reading! I’m the Founder and CTO at Panther, building intelligent AI agents and security data infrastructure to automate and accelerate core security operations workflows. If you want to learn more about how Panther incorporates security pipelines, open data lakes, signals/detection layer, and AI agents into its platform, check out our demo or book a meeting with me below! Panther is trusted by leading security teams like Coinbase, Asana, Discord, and more.

Continued Reading

The conversation centers around Steven's "crawl, walk, run" methodology for implementing AI in security operations, moving from simple data enhancement to autonomous decision-making with appropriate human oversight. He discusses the evolution of human-in-the-loop strategies, explaining how teams can build trust in AI agents over time while maintaining proper audit trails and governance. The discussion explores practical implementation details around enrichment agents, triage automation, and containment workflows, highlighting the importance of scoped permissions and security considerations when deploying AI agents with real operational impact.

Steven also addresses the organizational side of AI adoption, emphasizing the critical need for top-down buy-in, comprehensive training programs, and messaging that focuses on individual productivity benefits rather than cost-cutting narratives. Throughout the discussion, Steven reinforces that while AI won't replace security professionals, those who learn to use AI effectively will significantly out-compete those who don't.

Key Takeaways

• The Crawl, Walk, Run Framework Works: Steven's three-phase approach provides a practical roadmap—start with data enrichment agents, progress to reasoning models (triage), then move to action-taking agents (containment). This graduated approach builds organizational trust while delivering immediate productivity gains.

• Human-in-the-Loop: Rather than reviewing every agent decision forever, successful implementations start with intensive human oversight, gradually shifting to audit-based review as confidence builds.

• Detection Engineering Becomes Mission-Critical: As AI enables more granular, environment-specific detection logic, detection engineering skills become more valuable, not less. Organizations will shift from generic rule sets to highly customized detection logic tailored to their threat landscape and infrastructure.

• Organizational Change Requires Individual Value Proposition: Top-down AI mandates fail without proper training and clear individual benefits. Successful adoption focuses on how AI eliminates tedious work and enables analysts to focus on high-value activities that advance their careers.

• Security Considerations Are Engineering Problems: Concerns about AI agent security, MCP server trust, and permission scoping are solvable through proper engineering practices, vendor management processes, and incremental deployment strategies rather than barriers to adoption.

• The Productivity Multiplier Reality: Steven's prediction that AI-proficient security professionals will out-compete their peers isn't hyperbole—it's already happening. Entry-level positions are evolving, but professionals who master AI augmentation will have significant competitive advantages in the job market.

Relating Reading

Every SOC analyst knows the frustration of data gathering during their rotation: a suspicious login alert fires, but the investigation becomes a scavenger hunt across multiple log sources. Is this user typically remote? Has this IP been flagged before in an incident? Are there related alerts from the same timeframe? Every minute that goes by could mean potential escalation or another false positive, burning valuable analyst time.

AI SOC analysts promise to solve this problem by automating the tedious work of alert triage and investigation, but AI agents can only be as good as the context they are provided. Feed them isolated alerts without the proper background context, and you'll get shallow analysis. Give them the best data at the right time, and they can reason through complex security scenarios with the depth of your best analysts. The question is: What data does it take for AI agents to thrive in the SOC?

The answer lies in "context engineering"—the art and science of providing AI systems with the right source and depth of information needed to solve complex problems. In security operations, this means going beyond simple alert forwarding to building rich, contextual intelligence that helps AI agents understand what happened, why it matters, who was involved, and how it fits into your organization's unique threat model. Effective AI-driven security operations require four critical layers of contextual data across alerts, identity, asset, and enrichments, helping AI agents understand individual security events and how these events fit into your organization's broader risk landscape.

This blog post will explore the data layers that bring AI from a basic alert processor into an intelligent security analyst. We'll examine how historical alert patterns provide crucial learning opportunities, why identity and asset context separate real threats from false positives, and how enrichment data helps AI agents make the nuanced decisions that effective security operations demand.

Most security teams are already collecting this data, so let's integrate it for AI-driven security operations that work to our advantage.

Share

The Context Challenge

As teams introduce AI SOC analysts to automate triage and investigation workflows, the question becomes: how do we ensure these agents have access to the rich contextual intelligence that makes them as informed as their human counterparts?

Consider how your analysts approach a suspicious login alert. They don't just look at the raw events; they start building context by checking the user's recent activity patterns, cross-referencing the source IP against threat intelligence feeds, examining similar alerts from the same time window, and factoring in insider knowledge about ongoing projects. This contextual reasoning separates a 30-second false positive dismissal from a 30-minute investigation that goes nowhere.

Agent failures are context failures, not model failures, though they can also result from poor tooling integration, limited data access, or insufficient breadth in external connections. When AI-powered security tools produce shallow analysis, miss obvious patterns, or generate recommendations that feel disconnected from your environment, the underlying AI model isn't always the limiting factor. Modern large language models excel at complex reasoning when provided with comprehensive context and enough chain of thought. The challenge lies in gathering, structuring, and delivering that context effectively.

Alert and Signals History

The fastest way to triage an alert is to check if we have a record of it in historical patterns. As much as we strive for novelty in detection development, what typically ends up happening is continuous alerts from "the usual suspects" (looking at you, Jim from Marketing – kidding). But when an alert is genuinely unique, answering "have we seen this one before?" becomes crucial for determining whether it's malicious and needs established response procedures. This environmental learning—understanding whether patterns represent first-time occurrences versus recurring themes specific to your infrastructure—helps AI agents distinguish between suspicious geographic access and routine activity from your distributed remote workforce, or between genuine anomalies and normal business evolution.

Signals analysis takes this context check a layer deeper, where all alert events are proactively analyzed rather than just checking for identical alert matches. This can be particularly useful for examining indicator attributes across all alerts, such as checking if an IP address has appeared in other events, whether specific user attributes correlate with multiple alert types, or if attack techniques are used consistently across different timeframes. AI agents can easily take this history into context by checking for the same alert through 30-60-90 days, alerts from the same actor across detections, or all alerts around the same time period. It's not a perfect science (e.g., how far do we go back? how much data do we need to add into context?). However, these will typically yield more indicator clues to aid in additional data gathering and increase confidence.

🤖 "This suspicious login pattern has triggered 12 similar alerts over the past 6 months. Ten were false positives related to our mobile development team's remote testing environment, but two led to confirmed account compromises during our Q3 security incident."

Outcome and quality tracking becomes particularly valuable when analysts mark alerts as false positives, confirm genuine issues, or escalate to incident response, which creates crucial learning signals for future triage decisions. AI agents can begin to recognize the common indicators that separate benign anomalies from genuine security concerns, but only when they have access to this historical resolution data.

Share Detection at Scale

Identity Intelligence

While alert history provides the foundation for pattern recognition, identities provide organizational context about the human or non-human (service accounts) causing the alert. Understanding who is involved in an alert—their role, typical behaviors, access patterns, and organizational context—is often the difference between a real attack and an admin who ran an overly privileged command in production or someone from HR downloading many sensitive files from Google Drive.

User profiling enables AI agents to contextualize behaviors to understand whether unusual activity is genuinely suspicious or consistent with someone's job function. A DevOps engineer making widespread production changes during scheduled maintenance represents regular operational activity, while a marketing coordinator performing the same actions should trigger immediate investigation. Without this organizational context, AI agents default to taking all behaviors at face value, either overgeneralizing or missing legitimate issues.

🤖 "This user is a Senior Site Reliability Engineer based in our Seattle office. While the 2:47 AM login time is outside normal business hours, it correlates with a P1 incident escalation in our ticketing system and matches their historical pattern during infrastructure emergencies."

Team and organizational dynamics add another layer of contextual understanding that helps AI agents reason about lateral movement, privilege escalation, and insider threat scenarios. When multiple users from the same team exhibit similar behavioral changes simultaneously, this might indicate a targeted campaign or reflect organizational changes like new project assignments or initiatives.

AI agents need access to current organizational data that reflects these changes in real-time, rather than static user profiles that become outdated and lead to incorrect assessments.

Asset Intelligence: The Technical Context

Just as identity context helps AI agents understand who is involved in security events, asset intelligence provides crucial insight into what systems are accessed and their relative importance to business operations. This technical context transforms generic security alerts into risk-prioritized investigations, aligning with business impact.

Asset classification and business criticality enable AI agents to understand the difference between a suspicious login to a development sandbox and identical activity targeting user data in production databases. A brute-force attack against a decommissioned test server might represent low-priority cleanup work, while the same attack pattern against customer-facing payment systems demands immediate escalation. AI agents with proper asset context can automatically adjust investigation priority and escalation procedures based on the criticality of affected systems.

Infrastructure and deployment context help AI agents distinguish between legitimate cloud-native behaviors and potential security concerns. Auto-scaling events, serverless function executions, and container orchestration activities generate numerous security events that appear suspicious without proper infrastructure context. An AI agent that understands your Kubernetes deployment patterns can differentiate between regular pod lifecycle events and genuine lateral movement attempts while recognizing when cloud resource creation deviates from established automation patterns.

🤖 "This EC2 instance shows unusual outbound network connections to external IP addresses. However, the instance is tagged as 'ml-training-prod' and the connections align with our standard machine learning data pipeline that pulls from public datasets. The concerning factor is the timing—these connections typically occur during scheduled batch processing windows, but this activity is happening outside the defined maintenance schedule."

Vulnerability and patch context provide AI agents with essential risk assessment capabilities that help prioritize security events based on exploitability. A network scan targeting systems with known unpatched vulnerabilities represents a more urgent threat than identical activity against fully updated infrastructure. AI agents with access to vulnerability management data can correlate attack patterns with specific CVEs, helping security teams understand whether observed activity represents opportunistic scanning or targeted exploitation of known weaknesses.

Enrichment Intelligence: External Context That Matters

The final layer of contextual intelligence comes from external data sources that provide AI agents with broader threat landscape awareness. This enrichment context helps transform isolated security events into comprehensive threat assessments incorporating global intelligence and external indicators.

IP and domain reputation give AI agents the external perspective to assess whether network connections represent legitimate business activity or potential threats. Geographic location data, hosting provider information, and reputation scores help AI agents understand the difference between routine CDN connections and suspicious command-and-control communications. However, adequate enrichment goes beyond simple reputation scores, including contextual factors like recent registration dates, certificate anomalies, and infrastructure patterns that indicate potential threat actor infrastructure.

File and hash intelligence enables AI agents to quickly assess the risk level of unknown binaries, documents, and other artifacts discovered during investigations. Rather than treating every unknown file as equally suspicious, AI agents with access to comprehensive threat intelligence can prioritize investigation efforts based on known malware families, campaign attribution, and behavioral analysis from sandbox environments. This context is particularly valuable for prioritizing incident response efforts when multiple potential threats require simultaneous attention.

Campaign and technique correlation helps AI agents understand how individual security events fit into broader attack patterns and threat actor behaviors. When suspicious PowerShell execution correlates with techniques commonly used by specific threat groups, AI agents can provide analysts with relevant context about likely attack progression, typical dwell time, and effective containment strategies. This strategic context transforms reactive alert response into proactive threat hunting based on anticipated attacker behaviors.

Subscribe now

Making Context Work in Practice

The four layers of contextual intelligence, historical alert patterns, identity awareness, asset classification, and external enrichment, enhance AI agents from basic alert processors into more sophisticated analysts. However, the real value emerges from the interconnections between these data layers and how they inform each other during investigations.

Consider a practical example: an AI agent receives an alert about unusual database queries from a service account. Historical context shows this is the first time this particular query pattern has been observed. Identity intelligence reveals the service account is associated with a financial reporting application that typically runs predictable batch processes. Asset context indicates the target database contains customer payment information, a high-value target requiring immediate attention. Enrichment data shows the queries originated from an IP address recently flagged in threat intelligence feeds associated with a financially motivated threat actor group.

Each context layer provides valuable information, but the combination creates a comprehensive assessment that enables rapid and informed decision-making. The AI agent can immediately escalate this incident as a likely attack against high-value financial data, providing analysts with the context needed for effective response rather than generic "unusual database activity" alerts.

The key to successful AI-driven security operations is ensuring your AI agents have structured, current, and immediately accessible data when security events unfold. This requires thoughtful integration and onboarding of the logs and integrations that can provide these angles of intelligence.

Modern security operations teams that get this right find their AI agents becoming genuine force multipliers, handling routine triage with human-level contextual awareness while freeing analysts to focus on complex investigations and strategic security initiatives. Investing in proper context engineering pays dividends through faster incident response, more accurate threat prioritization, and security operations that scale with business growth rather than becoming bottlenecks.

During the day, I’m the Founder @ Panther Labs, building intelligent AI agents and infrastructure to automate and accelerate triage and investigation times for security teams while improving accuracy and quality. If you want to learn more about how Panther incorporates these layers of intelligence into its AI SOC analyst agent, check out our homepage demo or request a demo! Panther is trusted by leading security teams like Coinbase, Asana, Discord, and more.

Related Reading