The AI SOC Truth Test · Draft 0.1
AI SOC claims,
measured.
Every SOC vendor now says “AI-powered,” and many say “agentic.” The AI SOC Truth Test gives MSPs and MSSPs five questions to check any of those claims. CYREBRO AI created the test, so we’ve published our own answers first.
Disclosure: CYREBRO AI created the AI SOC Truth Test and is one of the platforms it measures. This is an open draft (version 0.1), not independent or neutral third-party research. It publishes measurements and gives no grades, rankings, badges or endorsements. How it’s governed
Why this exists
“AI-powered” is on every SOC. What it means isn’t.
If you’re building your business on someone else’s SOC, you need a simple way to tell real automation from a new label on the same work. Until now there hasn’t been one.
“the rebranding of existing products, such as AI assistants, robotic process automation (RPA) and chatbots, without substantial agentic capabilities.”
Five questions, same for everyone
Zero-touch rate, alert time, human role, build burden and multi-tenant readiness. Any platform should be able to answer them.
We answer first
A company that calls out agent washing has to show its own work. CYREBRO AI’s initial answers are published below.
Open to challenge
This is draft 0.1. Comment on it until November 11, apply to the independent review panel, or submit your platform for version 1.0.
The five measures
Five questions any AI SOC should be able to answer
Here’s the one-minute version. Open any measure below to see how draft 0.1 defines it, what a vendor should publish, and the open question we want your view on.
Zero-touch rate
Of all the alerts the platform handles, what share does it take from first signal to closed verdict with no human touching them?
Why it matters
It’s the plainest test of “agentic.” If people still review most alerts, the AI is assisting, not operating.
How draft 0.1 measures it
The share of alerts, over a stated period, that reach a final verdict and are closed without any human action. Any human review, approval, edit or reopening counts as a touch.
What a vendor publishes
The rate, the period it covers, the number of alerts in that period, and what the vendor counts as a touch.
Source
The vendor’s own reporting or public claims, with the source stated.
Open question for comment: Should alerts that the platform closes on its own, and a person later reopens, count against the rate?
Alert time
How long does it take from the moment a log arrives to the moment the platform reaches a verdict?
Why it matters
Speed claims are easy to make and hard to compare. Fixing where the clock starts and stops makes them comparable.
How draft 0.1 measures it
Elapsed time from log ingestion to verdict, as the platform records it. The clock starts when the log is ingested, not when an alert is raised, and stops at the verdict, not at a later notification.
What a vendor publishes
The typical figure, the period it covers, and how the two timestamps are recorded.
Source
The vendor’s own reporting or public claims, with the source stated.
Open question for comment: Should version 1.0 ask for a median and a slow-case figure (such as the 90th percentile) instead of a typical figure?
Human role
When a person is involved, who is it, when do they step in, and what do they do?
Why it matters
“AI-powered” can still mean an analyst behind every alert. A partner needs to know whose people do the work, and when.
How draft 0.1 measures it
A plain description, not a number. Who: the vendor’s analysts, the partner’s analysts, or the client. When: every alert, a sample, or escalations only. What: investigation, approval, or remediation.
What a vendor publishes
The description, and whether it changes by service tier.
Source
The vendor’s own reporting or public claims, with the source stated.
Open question for comment: Should the test separate people who approve an action from people who carry it out?
Build burden
Who builds and maintains the detection logic, and what else does a partner have to buy, deploy or run?
Why it matters
If the partner has to write the rules or bring a separate SIEM, the work and the cost move to the partner.
How draft 0.1 measures it
A plain description of who writes and tunes the detections, and which components, such as a SIEM, are included and which a partner must supply.
What a vendor publishes
The description, and a list of anything a partner must supply.
Source
The vendor’s own reporting or public claims, with the source stated.
Open question for comment: Should the test also ask how much work it takes to bring a new client on board?
Multi-tenant readiness
Was the platform built to serve many clients from one place, with each client’s data kept separate?
Why it matters
MSPs and MSSPs run many clients at once. A single-tenant platform stretched across clients adds work and risk.
How draft 0.1 measures it
A plain description of how clients are kept separate, and how a partner manages them together and one at a time.
What a vendor publishes
The description, and whether multi-tenancy was built in or added later.
Source
The vendor’s own reporting or public claims, with the source stated.
Open question for comment: What evidence of client separation should version 1.0 ask vendors to point to?
CYREBRO AI’s initial answers
We go first
Here are CYREBRO AI’s answers under draft 0.1, from our own reporting. They apply the same rules we ask of every vendor. Our complete answers, including the zero-touch rate, come with version 1.0.
Self-reported · draft 0.1from log ingestion to verdict.
Measured the way the final methodology defines it, once the comment period has shaped that definition.
CYREBRO AI’s escalation desk (or the partner’s analyst) on escalation only, with remediation by the partner or client, guided by step-by-step recommendations.
PublishedDetection logic built and maintained by CYREBRO AI, with CYREBRO Hyper-SIEM built in.
PublishedMulti-tenant by design.
PublishedWhat the test is, and isn’t
Measurements in the open. No grades, no badges.
It is
- An open draft (version 0.1), clearly labeled as a draft
- Measurements, sourced from each vendor’s own reporting or public claims
- Created by CYREBRO AI, and says so plainly
- Applied the same way to every vendor, including CYREBRO AI
- Open to public comment and to an independent review panel
It is not
- A finished standard, or something a platform can pass
- Grades, scores, rankings, badges, or endorsements
- Presented as independent or neutral third-party research
- A comparison chart naming competitors
- A gated lead form. You can read the whole method without signing up.
Draft log · live
Every comment, update and change, in the open
The comment period runs in public. This log shows published comments and our replies, progress updates, and every change to the draft, newest first. It refreshes on its own while you read.
Take part
Shape version 1.0
Three ways in. Each one goes to the same small team, and you’ll hear back by email. Prefer email? Write to comments@soctruthtest.com.
Comment on the draft
Tell us what’s missing, unclear or unfair: in a measure, in the method, or in CYREBRO AI’s own answers. Public comment closes Wednesday, November 11, 2026.
- You choose whether your comment appears in the draft log, and under what name.
- Every comment is read and considered for version 1.0, published or not.
- The review panel may read private comments when it audits the log.
Public comment has closed
The comment period for draft 0.1 ended on November 11, 2026. Version 1.0, with a summary of what changed and why, follows the week of November 30. You can still apply to the review panel or submit a platform.
Apply to join the independent review panel
The panel reviews the methodology for version 1.0 and audits the draft log. Members are named publicly with version 1.0, the week of November 30.
- Open to security practitioners, MSP and MSSP leaders, and researchers.
- Not eligible: Paul Warnagiris (board member and investor), CYREBRO AI staff, and the partners quoted at launch.
- Every applicant discloses conflicts of interest. Panel members’ disclosures are published.
Submit a platform for version 1.0
Vendors can submit their own platform to be measured in version 1.0. Your answers are published as you submit them, with your sources, alongside CYREBRO AI’s complete answers. They aren’t graded or ranked.
- Answer now, or register and we’ll send you the final version 1.0 questions.
- Same rules as CYREBRO AI: your own reporting or public claims, with the source stated.
- You’ll see exactly what will be published before it goes live.
Disclosure and governance
Who made this, and who keeps it honest
A test written by a vendor only works if everyone can see who wrote it and how it’s checked. Here’s ours.
Who made it
CYREBRO AI created the AI SOC Truth Test and is one of the platforms it measures. We published our own answers before inviting anyone else’s.
The test grades, ranks and endorses no one, CYREBRO AI included.
The review panel
An independent panel reviews the method for version 1.0 and audits the draft log. Members and their conflict disclosures are published with version 1.0.
Not on the panel:
- Paul Warnagiris (board member and investor)
- CYREBRO AI staff
- The partners quoted at launch
Moderation
Until the panel is seated, CYREBRO AI moderates the draft log under a published rule. We publish every on-topic comment whose author agrees to it, edit only to remove other vendors’ names and personal details, and mark every edit.
The panel then audits the full log.
What happens when
From draft to version 1.0
Draft 0.1 published
The five measures and CYREBRO AI’s initial answers go public. Comments open.
We are herePublic comment closes
Every comment is read and considered for version 1.0.
We are hereVersion 1.0
The named review panel, CYREBRO AI’s complete answers including the zero-touch rate, and vendor self-reports.
We are hereQuestions
Fair questions about a test like this
Is the AI SOC Truth Test independent?
No. CYREBRO AI created it, and we say so wherever it appears. What makes it fair is that the questions, the method and our own answers are all public; the same rules apply to every platform, ours included; and an independent review panel oversees version 1.0. CYREBRO AI staff, Paul Warnagiris (a board member and investor) and the partners quoted at launch can’t sit on that panel.
Can a platform pass or fail?
No. The test publishes measurements, not grades. There are no scores, rankings or badges, and no platform, including CYREBRO AI’s, can claim to have passed it.
Why hasn’t CYREBRO AI published its zero-touch rate?
Because the comment period may change how the zero-touch rate is defined. Publishing a number before the definition is settled would defeat the purpose. CYREBRO AI’s zero-touch rate will be published with version 1.0, measured the way version 1.0 defines it.
Where do the answers come from?
From each vendor’s own reporting or public claims, with the source stated. Draft 0.1 doesn’t independently check them. How version 1.0 should handle evidence is one of the questions open for comment.
Can I use the five questions with my current vendor?
Yes, that’s the point. Copy them into an RFP or a vendor review and ask anyone. You don’t need to sign up or tell us.
Will you compare vendors?
Draft 0.1 names no other vendor. In version 1.0, vendors that submit a platform will have their own answers published as they submitted them, with sources. Nothing is ranked or graded.
What happens to my comment?
We read every comment and consider it for version 1.0. If you agreed, we publish it in the draft log, usually within two business days, sometimes with a reply. Version 1.0 will say what changed because of public comment.
How are review panel members chosen?
Applications are open now. How members are chosen will be published alongside the named panel in version 1.0, together with each member’s conflict disclosure.
For reporters and analysts
Citing the test accurately
Suggested citation
The AI SOC Truth Test, draft 0.1. Created by CYREBRO AI. Published October 12, 2026. soctruthtest.com
Please include
CYREBRO AI created the AI SOC Truth Test and is one of the platforms it measures. Draft 0.1 is open for public comment until November 11, 2026.
Media questions: comments@soctruthtest.com
Hold every AI SOC to the same five questions
Starting with us.
Privacy notice
This notice covers what you send through this site, by form or by email.
What we collect
The details you enter: your name, work email, organization and role, plus your comment, application or submission. We also note how you reached the page (for example, a campaign link or referring site). This site sets no cookies and loads nothing from third parties until you send a form.
How we use it
To read and respond to what you send; to consider it for version 1.0; to publish your comment in the draft log, only if you agree and only under the name you choose; to contact panel applicants and submitting vendors about version 1.0; and, only if you ask, to tell you when version 1.0 is published. The review panel may read private comments when it audits the draft log.
Where it’s kept
In a customer relationship management (CRM) system run for CYREBRO AI by Elano, its launch partner for this test. We don’t sell it or share it with other vendors.
Your choices
To see, correct, unpublish or delete what you sent, email comments@soctruthtest.com. For everything else, see the CYREBRO AI privacy policy.