No Going Back: AI Redefines SecOps for Security Service Providers (MSSP)

Most of the security industry's response to AI is to do the same thing it has always done, only faster. Buy a tool, bolt it onto the existing pipeline, point it at the alert queue. The panel that LimaCharlie's Christopher Luft hosted spent its time arguing, from several directions, that this is exactly the wrong instinct. The participants do not agree on much. Joshua Neil, co-founder and CTO of Alpha Level, has spent twenty-five years on machine learning for threat detection and is openly skeptical that large language models deserve the spotlight they get. Eric Capuano of the Digital Defense Institute is building agentic workflows every day. Daniel Lees works on cloud security at Google's scale. Kris Merritt founded CrowdStrike's Overwatch. What unites them is a refusal to treat AI as a faster horse. As Merritt put it most bluntly, the danger is that we just scale what we have today, and that is not the answer.

The expensive mistake is using one tool for every job

The sharpest disagreement in the room turned out to be the most useful. Neil refuses to let "AI" collapse into "LLM." He has no definition of AI under which a language model qualifies but supervised machine learning does not, and he pointed out that agentic systems have been taking full-lifecycle remediation actions since roughly 2017, citing Defender's deep learning malware detection, which scores its own confidence, decides what to do, and remediates without a human. The interesting thing is that nobody on the panel really fought him. Merritt agreed that the industry called machine learning "AI" for years before LLMs made it fashionable. Lamothe-Brassard, LimaCharlie's founder, offered the analogy that everyone kept returning to: building a factory line to produce a hundred identical bicycles a day is ML, a deterministic pipeline worth six months of tuning. The LLM is the guy across the street with a toolbox who fixes your one broken bike. He will occasionally strip a nut. That is fine, because the job is not high-volume and repeatable.

This matters for service providers because the wrong tool is not just less effective, it is more expensive. Neil kept circling cost, noting that he repeatedly sees vendors arrive with LLM solutions that are far pricier than something simpler would achieve, and that graph methods for correlation are cheap, understandable, and controllable. Lees reinforced it from the other end: at Google's data volumes, the gamechanger has not been autonomous reasoning but the mundane work of cross-telemetry onboarding and normalization, the toil of writing per-source integration patterns and regular expressions that LLMs now absorb. The lesson for an MSSP running thin margins across many tenants is unglamorous. Use deterministic ML for alert triage and high-flow detection where precision still needs to improve by orders of magnitude before it can act alone. Use LLMs for the fuzzy, language-shaped work: configuration, onboarding, research, summarization, pulling indicators out of an intelligence PDF.

The shift from assistant to operator is real, but it is a tool problem

Lamothe-Brassard traced why "operator" is more than a marketing word. Early LLMs were text in, text out, so the natural pattern was assistant-shaped: call a simple API, pull an alert, summarize it. Two races then ran in parallel, one in tool-calling plumbing and one in model reasoning, and tool calls arrived first. For a while you could insert yourself into the loop and tell the model which tools to call in which order. His argument is that in roughly the last six months the models caught up, and now scripting that sequence yourself throws away half the value, because the model can reason about which tools it needs and hold the thread across a long investigation. Capuano described the same effect from experience: coupling the assistant with agentic operation and then removing himself from the loop often produced better results, because he stopped injecting his own bias into a conversation he did not always know how to have.

Lees grounded this in what it buys an analyst. Instead of a skilled person manually stitching a login here to a permission five systems away, the LLM reasons across the graph of the infrastructure and the relevant path bubbles to the top of the haystack as an explainable story. Merritt pushed the same idea toward multi-tenant economics. The difference, he argued, is between a hundred and fifty analysts each judging a sliver of a fraction of what actually matters and a smaller team making decisions across half of everything an environment sees.

What separates platforms built for this from platforms retrofitting it

Pressed on what breaks when AI is bolted onto a platform designed for a graphical interface, Lamothe-Brassard laid out three patterns. One-off integrations expose a tiny surface to AI, which he believes describes the vast majority of security tooling by volume. MCP servers, which boomed this year, tend to push vendors to wrap many actions into a few coarse tools, which strips agency from the model and burns context. LimaCharlie's own first attempt had 170 tools consuming roughly 70,000 tokens, which simply does not work. What holds up is genuine API coverage, the way the hyperscalers expose nearly everything through their command lines and therefore give AI close to full coverage. He noted that CLI tools turned out to be, in his phrase, the best MCP, a cheat code dropped in the middle of all this. The dividing line for buyers is concrete: a product built interface-first faces a long road to AI operation, while one built API-first, with the interface rolled up on top, gives an operator-grade agent real reach.

Do not automate the work you should not be doing

The most quietly radical thread came from Merritt, and it reframes everything else. The industry, he argued, spends close to zero time on analytical tradecraft, then tries to force analysis into deterministic workflows. His view is that the moment you can structure analysis into a standard operating procedure, you should simply automate it, which means humans following an SOP are doing the wrong work. LLMs are valuable precisely because they scale nondeterministic problem solving, the part that does not reduce to a rule. He warned that an LLM is only as good as its user plus one, since a tier-one analyst will accept output a tier-two analyst would correctly call wrong, so dropping these tools onto junior staff can amplify problems rather than solve them. Capuano agreed there is a real risk, then made the counterpoint that codifying a team's best practitioners into agentic workflows raises the floor: when the big incident lands and the junior analyst is on call, the investigation runs on the team's combined best thinking rather than one person's training level.

Neil, the longtime skeptic, supplied the reason none of this is optional. He has spent twenty years fighting CISOs who rejected anything that was not a readable rule, and what finally opened their appetite for inferential methods was the rise of LLMs. That shift is arriving just as it has to, because agent-speed attacks can rotate every credential in a cloud environment in seconds, and Lees confirmed seeing exactly that. If humans are the ones interdicting, Neil said, we have already lost. The consensus prediction was not a smarter chatbot. It was orders of magnitude fewer false positives through just-in-time, context-aware detection, intent-based security where you state a goal instead of writing policy as code, and a natural language interface into entire security operations with real data science underneath. Every panelist kept the human in the loop for the decisions that count. The argument was never about replacing judgment. It was about refusing to scale the wrong work and finally building the operation around how analysis actually happens.