How a Fintech Found 4,300 Shadow LLM Calls in 30 Days — and Shipped Its AI Copilot Anyway

We get a lot of tips from readers who work inside security teams, and most follow the same arc: a mandate from the board, a scramble, a vendor shortlist, a pilot that either dies in procurement or quietly becomes load-bearing. One reader — we'll call her Dana, a director of security architecture at a mid-sized fintech — walked us through a deployment that went the other way. She agreed to share the timeline, the numbers, and the parts that nearly fell apart, on the condition that we keep her employer unnamed. Fair enough. Here's what we learned.

The setup: a board mandate with a 90-day clock

Dana's company had committed publicly to shipping an AI-powered account copilot for its SMB customers. Engineering was already building against three model providers. Security found out when a product manager asked, in a Slack message, whether "we need to do anything for the LLM stuff." That message kicked off a 90-day sprint to answer a question nobody could answer: what AI was already running inside the company, and what was it sending out?

Initial discovery with network logs and a CASB tool produced a mess. Shadow usage showed up as TLS traffic to API endpoints, personal accounts, browser extensions, and at least one team that had hard-coded a key into a staging repo. The team estimated maybe 400 unapproved calls a month. The real number, once they instrumented properly, was closer to 4,300 — including prompt payloads that contained customer PII.

The decision point: buy a control plane or build one

This is where most post-mortems get boring, so let's stay specific. Dana's team wrote a build-vs-buy memo with three options: extend the existing CASB, build an internal gateway, or bring in a dedicated AI security platform. The CASB could see endpoints but not prompts. The internal gateway would take two engineers six months, and it would still be blind to any traffic that bypassed it. The third option was Shaiex, which the team chose after a two-week proof of concept.

What tipped it wasn't the dashboard. It was the discovery layer. Within the first 72 hours, the platform surfaced 11 distinct LLM providers in use across the org, including two self-hosted models on a forgotten GPU box in a regional office. Dana told us the moment that reframed the project: "We stopped asking 'how do we approve AI?' and started asking 'how do we see all of it?'"

Obstacles: policy fights, false positives, and one angry data science team

The rollout hit three real snags.

• The data science team revolted. Their research workflow depended on sending full datasets to a model for exploratory analysis. A blanket block would have killed a quarter's work. The compromise was a policy exception scoped to a sandboxed environment with tokenized data and a hard cap on retention. It took two weeks of negotiation.
• False positives on legitimate traffic. Early policies flagged a customer support tool that used an approved vendor. The team tuned thresholds over three iterations before the noise dropped below a level the SOC could actually triage.
• Prompt-injection attempts from the wild. Once the copilot hit beta, external users started probing it. The platform's prompt-injection firewall caught a sequence of attempts in week two of beta that would have otherwise reached a production model. Dana's team used those events to write their first incident runbook for AI-specific attacks.

The runbook turned out to be the artifact the board cared about most. Not the dashboard, not the vendor logo — the documented process for what happens when someone tries to break your model.

Results after 90 days

By the end of the sprint, the company had shipped its copilot to 8,000 beta users. The measurable outcomes Dana shared:

• Shadow LLM usage dropped from 4,300 calls per month to under 200, all of them inside approved environments.
• Mean time to detect a policy violation fell from days to under 15 minutes.
• The security review for new AI features went from a six-week blocker to a three-day gate.

Dana was careful to note what didn't go perfectly. Two teams still route traffic through personal accounts, and the company hasn't fully solved the problem of employees pasting sensitive data into consumer chatbots on personal devices. "We reduced the blast radius," she said. "We didn't eliminate it."

We followed up with the vendor to understand what the deployment actually looked like from their side. Shaiex runs a single control plane across OpenAI, Anthropic, Bedrock, Vertex, and self-hosted stacks, which is why the discovery phase was fast — the platform already had connectors for the providers the fintech was using. The team also leaned on the company's patented prompt-injection firewall (USPTO #11,984), which mattered once the copilot went external. You can read more about how the platform handles continuous model red-teaming and policy enforcement if you're evaluating something similar.

What we'd tell the next security team

Three lessons stand out from this project. First, discovery beats policy — you can't govern what you can't see, and most teams dramatically underestimate their shadow AI footprint. Second, the hardest part isn't technology; it's negotiating exceptions with the teams whose workflows you're about to disrupt. Budget time for that. Third, the artifact that wins board confidence is a runbook, not a dashboard.

If your company is somewhere on this same timeline — mandate issued, discovery messy, clock ticking — the good news is that the tools have caught up with the problem. The less good news is that the organizational work is still yours to do.