What that looks like in practice is a specific question asked before any agent goes into production: when this agent acts outside its intended scope, how will we know, and what stops it? Most teams cannot answer that question for the agents already running in their environment, not because they are careless but because the deployment happened under pressure and the governance conversation happened AFTER, if it happened at all.

The Gravitee State of AI Agent Security report, published in February 2026 and drawing on 919 executives and practitioners, found that only 21.9% of organizations treat AI agents as independent, identity-bearing entities within their security model.1 The agents in the other 78.1% are authenticated through shared API keys and generic service accounts — persistent, unmonitored access pathways running continuously across production systems. The same research found that 88% of organizations had confirmed or suspected an AI agent security incident in the prior year.

When something goes wrong in that environment, the question the board asks is not whether the model was safe. The question is who authorized the agent, what did it have access to, what did it actually do, and where is the audit trail — and those are very difficult questions to answer when the governance architecture was never built.

The agents are in production. Whether your organization can answer those four questions right now is worth knowing before you need to.

UPDATE 1: What I am now thinking as I write this though, is more questions I haven’t seen most formally work through yet.

• If you were to actually threat model this problem — not conceptually, but rigorously, the way a serious security architecture exercise gets done — what portions of the enterprise are genuinely at risk?

• Which business processes are now touching agent access pathways they didn’t have six months ago?

• What does the revenue-at-risk calculation look like when you map agent permissions against your most critical systems?

• And what does the tail look like — not the median incident, but the scenario where an unmonitored agent with shared credentials and no audit trail gets manipulated in exactly the wrong way at exactly the wrong time?

I honestly don’t think most organizations have run that exercise. I’m not sure most understand they have the instrumentation to run it today — the rapid knowledge graph module built by your’s truly. But the gap between “we have agents in production” and “we understand our exposure” is where the next significant incidents are going to come from — and the organizations that close it proactively are going to look very different from the ones that close it reactively.

TJ: If you are confronting epistemic governance problems in your organization where different people trust different versions of truth based on whose political interests the data supports, I would be genuinely interested in hearing what patterns you are seeing.

I am six weeks into Carnegie Mellon’s Chief Data & AI Officer executive program, and my team just received our first deliverable assignment—a fifteen-page project plan documenting company research, industry analysis, stakeholder mapping, and our approach to developing an enterprise data and AI strategy roadmap for the organization we selected.

The program is explicitly demanding rigor, which is appropriate given what these foundational assessments are meant to support. What I have learned across twenty years as a security leader, though, is that rigor requires more than thorough research execution—it requires constitutional governance over how truth gets established, how that truth gets tested adversarially, and how uncertainty gets preserved through synthesis instead of being systematically laundered into false confidence.

I have watched the same pattern destroy value mechanically in every organization I have worked with over the years. Let’s consider a few instances:

• Your analysts discover through careful financial mapping that the largest customer segment generates genuinely negative unit economics. The CFO explains it fluently as strategic investment in market share that will deliver profitability at scale, and your analysis documents this as growth opportunity with margin expansion potential.

• The same team identifies that sixty percent of revenue sits behind customer agreements containing change-of-control provisions. Business development assures everyone these clauses rarely trigger in practice, and your assessment codes this as customer concentration risk, manageable.

• Technical diligence reveals the core platform requires fifty million dollars and eighteen months to migrate off the current hyperscaler. The CTO positions this as cloud-native architectural sophistication, and your deliverable describes it as strong technical foundation.

Structural fragility somehow became strategic positioning. Nobody fabricated data, but the analytical framework did exactly what it was designed to do: systematically remove uncertainty until observable constraints transformed into narrative elements supporting whatever direction leadership was already inclined to pursue.

The ever-present pattern where findings get shaped to support whatever narrative leadership already committed to, instead of letting the evidence determine which strategic options actually survive contact with reality.

This happens mechanically rather than through individual analytical failure. The problem is architectural.

When Feedback Loops Are Brutal and Compressed

My background and career track has forced me to develop different instincts because cybersecurity creates feedback loops that are both brutal and compressed. (TJ: My cardiologist and wife can both attest to this to no end.) When your threat model fundamentally mischaracterizes risk or your controls prove ineffective, breaches happen on timelines you do not control, production stops, and regulators arrive with enforcement authority that does not care about your explanations. You develop discipline around uncomfortable truths because the alternative is professional extinction.

I am expanding breadth and influence now through data and AI strategy frameworks where feedback loops stretch across quarters or years instead of weeks. Bad security architecture surfaces in breach reports that create immediate forcing functions for accountability. Bad epistemic governance in foundational strategic assessments hides inside multi-year transformation programs where it compounds through every subsequent decision until the accumulated analytical errors become visible as destroyed enterprise value that nobody can trace back to the initial assessment work that set everything on the wrong trajectory.

Think about what happens when you build data strategy recommendations on stakeholder interviews where different executives trust different data sources selectively based on which reports support their existing positions, and your framework provides no mechanism for resolving those conflicts beyond documenting that perspectives vary. The technical work proceeds exactly as planned, but your entire strategic foundation rests on politically contested epistemological ground that will collapse the moment implementation requires someone to definitively lose an argument about whose version of reality governs investment decisions.

Six Controls That Make the Work Harder On Purpose

Before we touched company research or begins competitive analysis, i proposed to build a Stage 0—a constitutional layer that locks in before our substantive work began. Six controls that make our work measurably harder, because difficulty is how you know constitutional governance is actually functioning. These six controls are laid out directly below:

0.1 Strategic intent lock declares what our analysis is permitted to conclude and what it is forbidden to do. Company assessment only—no strategy recommendations embedded in findings, no transformation roadmaps implied through selective emphasis. If readers could interpret our findings as containing implicit recommendations, the lock failed.

ROLE:
You are acting as the Diligence Director (authoritative).

MISSION (NON-NEGOTIABLE):
Define what this diligence is allowed to decide — and what it is explicitly forbidden to do.
This exists to prevent strategy creep, solution bias, and retrofitted conclusions.

ANALYTICAL BURDEN:
You MUST:
• Declare the exact decision classes this diligence supports
• Explicitly prohibit strategy design, roadmaps, and recommendations
• Lock the lens to discovery, not prescription
• Define the consumer(s) of truth (e.g., IC, board, CDAIO, acquirer)

MANDATORY ADVERSARIAL TESTS:
You MUST explicitly answer:
• What would constitute misuse of these outputs?
• What decisions should NOT be made from this work?
• What incentives exist to overstep these boundaries?
• Who benefits if these boundaries blur?

0.2 Evidence hierarchy defines what constitutes legitimate evidence and how we resolve conflicts. SEC filings outrank earnings calls because filings carry legal liability. Audited financials outrank investor presentations because one has independent verification. Every material assertion traces to source documents through transparent lineage.

ROLE:
You are acting as the Evidence & Methodology Authority.

MISSION:
Define what counts as evidence, how it is weighted, and how conflicts are resolved.
This is the line between diligence and opinion.

ANALYTICAL BURDEN:
You MUST:
• Define primary vs secondary vs inferential evidence
• Establish a trust hierarchy:
  REGULATORY > AUDITED > CONTRACTUAL > ISSUER > THIRD-PARTY > MEDIA
• Define how stale, contradictory, or partial evidence is handled
• Define minimum citation requirements per claim type

MANDATORY ADVERSARIAL TESTS:
You MUST explicitly answer:
• What evidence would NOT be sufficient?
• When is inference allowed — and when is it forbidden?
• How are analyst echo chambers avoided?
• How is availability bias prevented?

0.3 Mandatory falsification requires documented searches for disconfirming evidence before we claim any hypothesis has support. The most valuable finding is often we expected X but evidence indicates not-X, except institutional patterns reward confidence so contradictory evidence gets buried as areas requiring additional investigation.

ROLE:
You are acting as the Red Team Lead.

MISSION:
Ensure every major line of inquiry attempts to prove itself wrong.
No hypothesis survives unchallenged.

ANALYTICAL BURDEN:
You MUST:
• Require explicit hypotheses for each stage
• Mandate disconfirming evidence searches
• Track hypothesis survival or collapse
• Prevent hypothesis drift into assumption

MANDATORY ADVERSARIAL TESTS:
You MUST explicitly answer:
• What would disprove each hypothesis?
• What evidence was sought but not found?
• Where did confidence decrease?
• Which hypotheses collapsed — and why?

0.4 Uncertainty preservation distinguishes between things not yet verified and things that cannot be verified with available information. We attach confidence levels to every significant claim. When stakeholder interviews contradict financial filings, we document that contradiction rather than averaging it into comfortable middle positions that obscure the fact we cannot determine ground truth.

ROLE:
You are acting as the Epistemic Risk Analyst.

MISSION:
Prevent uncertainty from being smoothed, hidden, or laundered.
Uncertainty must be classified, bounded, and preserved.

ANALYTICAL BURDEN:
You MUST:
• Distinguish UNKNOWN from UNKNOWABLE
• Require confidence levels on all claims
• Prevent aggregation from erasing variance
• Force explicit statements of confidence decay

MANDATORY ADVERSARIAL TESTS:
You MUST explicitly answer:
• What do we not know?
• What cannot be known from public sources?
• Where does uncertainty materially affect decisions?
• Where would false certainty be dangerous?

0.5 Independent enforcement establishes red-team review where at least one team member challenges every major finding with authority to force complete rework regardless of schedule pressure. Without this, all other controls become performance theater.

ROLE:
You are acting as the Independent QA & Red Team Authority.

MISSION:
Ensure no stage self-certifies rigor.
All stages must be challengeable, reversible, and auditable.

ANALYTICAL BURDEN:
You MUST:
• Define cross-stage QA checks
• Require red-team review for synthesis stages
• Establish fail / redo authority
• Prevent schedule or convenience pressure from overriding rigor

MANDATORY ADVERSARIAL TESTS:
You MUST explicitly answer:
• Who can force a redo?
• What triggers escalation?
• How are weak stages detected?
• How is dissent preserved?

0.6 Output integrity structures our deliverable so uncertainty bounds cannot be stripped out by readers who only engage with executive summaries. Confidence qualifiers embed directly with claims rather than living in footnotes that disappear during excerpting.

ROLE:
You are acting as the Output Integrity & Risk Analyst.

MISSION:
Ensure outputs cannot be stripped of context, caveats, or uncertainty.
This exists to prevent downstream misuse.

ANALYTICAL BURDEN:
You MUST:
• Require claim → evidence traceability
• Embed caveats and confidence with outputs
• Prevent selective excerpting
• Define how summaries must reference full analysis

MANDATORY ADVERSARIAL TESTS:
You MUST explicitly answer:
• How could these outputs be misused?
• How is overconfidence prevented?
• What context must never be removed?
• What warnings must travel with outputs?

These controls make our deliverable harder to produce. That difficulty is the entire point of constitutional governance. I provide the prompt in its entirety at the end of the article.

Why 73% of AI Initiatives Collapse Before Models Run

The conventional wisdom about AI failure is well-documented and widely accepted across industry research.

MIT’s 2025 study shows ninety-five percent of enterprise AI pilots deliver zero measurable return, S&P Global Market Intelligence reports forty-two percent of companies abandoned most AI initiatives in 2025 up from seventeen percent the year before, and Informatica’s CDO Insights survey identifies the primary culprits as data quality and readiness at forty-three percent, lack of technical maturity at forty-three percent, and shortage of skills at thirty-five percent.

The diagnosis has become remarkably consistent: fix data quality through governance, align stakeholders through communication, clarify use cases through business discipline, and invest in organizational capabilities through training. To be clear, I completely agree that data quality must be paramount.

I have watched organizations follow that exact playbook, spending millions on data quality initiatives and governance frameworks. Has the failure rate stayed constant because the conventional diagnosis treats symptoms as root causes? Let’s see.

Here is what the industry analysis consistently misses: data quality problems are real, but organizations discover them only after committing strategic direction based on foundational assessments that mistook management narrative for enterprise reality. Read that again. They then launch expensive governance initiatives to fix quality issues in systems they should never have selected, for use cases they never properly validated, supporting business models they never stress-tested against contradictory evidence that was available during assessment.

I use CISO disciplines to protect the enterprise and I am building CDAIO capabilities to drive enterprise growth, and that combination is unusual enough that it changes what you can see when you look at organizational failures and where you have leverage to force change before patterns become irreversible. Both roles operate in domains where failure stays completely invisible until it becomes catastrophic enough to force response, where truth is frequently politically inconvenient in ways that create real career risk for people who insist on it. Where institutional incentives systematically reward narrative comfort over analytical accuracy because comfortable narratives enable action while uncomfortable truths force confrontation with constraints leadership would prefer to ignore.

The difference between these domains is timing rather than fundamental challenge. Security failures surface on attacker timelines you cannot control or negotiate—hours or days or weeks that force immediate accountability because external threat actors do not wait for your strategic planning cycle to complete. Bad epistemic governance in foundational strategic assessments hides inside multi-year transformation programs where compounding analytical errors take quarters or years to become visible as destroyed enterprise value that nobody can trace definitively back to the initial assessment work that set everything on the wrong trajectory at the very beginning when course correction would have been relatively inexpensive.

Organizations do not fail at AI implementation because they chose the wrong technology platforms or because their data quality turned out to be insufficient for the models they wanted to deploy. They fail because their foundational assessments—the exact work this first deliverable requires us to produce—systematically mistake management narrative for enterprise reality, and every subsequent recommendation inherits that distortion, compounds it through additional layers of analysis that treat the initial assessment as settled truth, and amplifies it into strategic direction that destroys value at scale. The data quality issues surface later as symptoms because the foundational assessment never properly tested whether proposed use cases could actually be supported by available data, whether the business model is economically viable when you account for actual unit economics, or whether stakeholders claiming alignment actually agree on what constitutes authoritative information when positions conflict during implementation.

My belief is that you cannot fix this pattern by executing harder or by investing more capital in data quality remediation after flawed strategic direction has already been set. The failure is architectural and requires constitutional governance operating at the assessment design level, before anyone commits resources to directions that were never properly validated against evidence.

Nine Stages of Constraint Discovery

This starts a series walking through nine stages of progressive constraint discovery, each designed to surface truths that standard assessments routinely smooth away. Corporate structure and legal constraints. Business model mechanics and value flows. Market forces and competitive dynamics. Organizational execution reality. Financial behavior under stress. Technology constraints and platform dependencies. Information flows and epistemic authority. Risk boundaries and regulatory constraints. Resilience under shock.

The goal is never comprehensive analysis—that is a comfortable fiction. The goal is accurate discovery of what actually constrains outcomes, preserved with uncertainty bounds intact, structured to resist reinterpretation by people who never read the underlying evidence.

Next piece covers Stage 1 and why standard assessments systematically confuse organizational charts with actual power structures, why legal entities map poorly to operational reality, and why formal corporate structure tells you almost nothing about where authority actually lives in practice. Cheers!

Diligence Constitution Stage0 Techref

238KB ∙ PDF file

Download

Download

What matters now is not whether this gap exists, but why it persists even after it is recognized. Most organizations see the problem, acknowledge it (often reluctantly), and attempt to close it by refining metrics, adding dashboards, or aligning more closely to frameworks. And yet, the outcome rarely changes.

From a data leadership perspective, this pattern is all too familiar. When decision support fails consistently, the root cause is almost never the metrics themselves. It is the absence of an underlying data architecture designed to support the decision being asked.

The Structural Cause, Reframed as a Data Problem

I would offer that any Chief Data Officer would not describe most cybersecurity measurement environments as immature because they lack data. They would however describe them as immature because the data has no defined analytical purpose.

In a data-driven enterprise, measurement begins with a decision context. Data is treated as an asset whose value depends on its ability to inform a specific choice. Governance starts by identifying the decision, the consumer, and the consequence of uncertainty. Only then are data sources identified, quality expectations defined, and metrics selected. This is the core logic of enterprise data management and analytics maturity.

Cybersecurity measurement does not follow this pattern. Unfortunately.

Security data is generated opportunistically as a byproduct of technology execution. Controls emit signals. Tools generate telemetry. Processes produce artifacts. I would even go so far as to state that it also is a byproduct of security vendor marketing. Let me know your opinion on this one. Measurement is layered on top to summarize what exists, what ran, and what was completed. From a data governance standpoint, this is unmanaged exhaust or noise. It is data without a declared consumer, without an explicit analytical model, and without accountability for decision impact.

There is no formal decision schema that defines how cybersecurity data should be organized to support governance. There is no clear articulation of which uncertainties matter to executive leadership and which do not. As a result, cybersecurity metrics are governed for completeness, consistency, and external expectation rather than for decision relevance. They simply exist because they can be produced.

A data leader would recognize this immediately. It is the same failure mode seen in early enterprise analytics efforts across other domains: abundant data, increasingly sophisticated reporting, and a persistent inability to translate information into action.

Why Metrics Cannot Repair an Architectural Gap

In analytically mature organizations, measurement is derived from an explicit understanding of what must be explained, predicted, or optimized. Data is collected to support that model. Indicators are retained because they reduce uncertainty for a specific decision. Metrics that do not do this are discarded, regardless of how easy they are to collect or how familiar they appear.

Cybersecurity measurement inverts this logic. Data is collected because it exists. Metrics are defined because they are easy to find. Governance is expected to infer meaning after the fact. When that inference fails, the response is to add more metrics rather than to question whether an analytical model of value exists at all.

From a data maturity standpoint, this guarantees failure. Without an explicit model linking security activity to risk outcomes, metrics cannot be evaluated for usefulness. The system rewards accumulation rather than business value.

This is why cybersecurity measurement environments grow wider but not deeper. Dashboards expand. Domains multiply. Confidence inside the function increases—often without justification. But the analytical capability required to reason about exposure, tradeoffs, and residual risk never materializes, because it was never designed.

I have been able to explain what every cyber metric represents and still be unable to explain what the dataset allows governance to conclude. That is not a communication problem. It is what happens when data was never designed to support decisions.

Cybersecurity as a Failed Data Product

Modern data organizations do not think in terms of reports. They think in terms of data products. A data product has a defined consumer, a defined use case, and a measurable contribution to a business outcome. Its value lies in changing a decision or enabling an action.

Viewed through this lens, cybersecurity measurement is not a data product. It has consumers, but no agreed-upon use case. It produces outputs, but no defined decision impact. It generates activity, but no accountable outcome.

From a Chief Data Officer’s perspective, this is the core failure. Cybersecurity—often reinforced by vendor-driven reporting—has built pipelines and mistaken them for analytical products. Organizations invest in data generation and visualization, but not in modeling, hypothesis testing, or uncertainty reduction. Measurement is delivered, but value is assumed rather than demonstrated.

This is why cybersecurity measurement struggles to show business value in the same way other data initiatives do. There is no value proposition defined at the outset. There is no articulation of how improved measurement should change capital allocation, risk acceptance, or operational constraints. Without that linkage, measurement cannot be evaluated as an asset. It can only be maintained as a cost.

Why This Becomes a Governance and Data Leadership Problem

As long as cybersecurity measurement is treated as internal reporting, these shortcomings are survivable. Reporting does not require causal models or quantified uncertainty. It requires consistency and coverage—areas where most CISOs already excel.

BUT boards are accountable for cyber risk outcomes. Regulatory scrutiny increasingly expects evidence that decisions were informed and deliberate. Insurance, audit, and executive oversight converge at the point where risk must be explicitly accepted, reduced, transferred, or avoided.

From a data leadership standpoint, this is precisely the type of environment that demands structured data, explicit models, and disciplined analytics. When those do not exist, governance still has to act. Decisions are made using judgment, external benchmarks, or precedent. The absence of decision-grade data is masked by the presence of abundant reporting.

This creates a fundamental mismatch. Leadership is accountable for outcomes, but the data systems supporting cybersecurity were never designed to inform the choices leadership is now expected to justify. From an enterprise data governance perspective, this is not a cybersecurity failure. It is an organizational failure to treat cyber risk as a governed analytical domain.

The Governance Obligation We Have Not Met

At this point, the unresolved issue is not whether cybersecurity measurement should support governance decisions. That premise is already accepted. The unresolved issue is whether we have built the data architecture necessary for that support to exist at all.

Decision-grade measurement requires more than dashboards and indicators. It requires a system that defines the decision, models uncertainty, encodes assumptions, and translates operational data into evidence that can be evaluated and compared. Without that foundation, metrics will continue to proliferate without producing insight.

In the final post of this series, I will describe what a decision-centered cybersecurity measurement architecture looks like when approached as a data problem first. Not as a framework, and not as a compliance exercise, but as an analytical system designed to support governance decisions under uncertainty.

As a CISO, my accountability centered around data protection. Risk quantification, compliance, third-party oversight, and incident response depended on how data was classified, governed, measured, and trusted across the enterprise.

I have been writing a standard/book for two years now(whew 😒) to formalize a proper cyber metrics reality and extend it. It focuses on building enterprise data models, metrics, and measurement frameworks that convert technical and operational complexity into proper decision-grade insight. The objective is accountability: enabling leadership to govern risk, allocate capital, and evaluate outcomes based on evidence as analytics and automation scale. The same governance and measurement mechanics apply to AI, where model behavior, training data, and operational impact require defined guardrails and executive oversight.

I am excited to announce that I will be engaging with the prestigious Carnegie Mellon University’s Chief Data and AI Officer (CDAIO) program to deepen this work and formalize the operating models required to govern data and AI at enterprise scale. 🎩 🚀

My career has come full circle from securing systems, to governing data, to overseeing how analytics and automation shape enterprise outcomes. Expanding into data and AI leadership broadens the scope and the failure modes are already familiar; the surface area is just a bit larger. That is why it feels so exciting. It calls for continued learning, operational rigor and restraint, and it is work I am ecstatic to step into.

Wish me luck. Cheers!

Our organizations invest heavily in security controls, tooling, and programs, maybe not as much as we would like, but the notion still holds true. We track coverage across our environments, maturity across all domains, and the activity across business functions. All this so that we can produce regular reports, satisfy audit requirements, and demonstrate alignment(for good or worse) to established frameworks. On paper, our programs appear disciplined and increasingly mature. The important point is this: the failure becomes “more” visible when we and our peers try to use the information to make an actual decision. That is the goal, right?

At that point, the information that we have provided rarely reduces uncertainty in a way that supports real action. Leadership cannot clearly determine how risk has changed, which exposures remain material, or which tradeoffs follow from the data being presented. The discussion shifts toward reassurance rather than resolution, and decisions proceed based on “judgment” rather than “evidence”.

This outcome continues to persist even in organizations with experienced teams, a huge amount of telemetry, and strong external validation. And yet, the same failure is constantly/consistently found. So knowing this, we can say that this rules out isolated causes and more than likely points to a structural one. The underlying issue sits in measurement design. Hard stop.

Most cybersecurity measurement efforts evolve from what is easy to observe rather than from what leadership needs to decide. Metrics describe activity(most of all), completeness, or progress within the security function, but they do not explicitly model how those observations change the likelihood or impact of outcomes that our business cares about. As a result, measurement artifacts accumulate without forming a coherent decision system. Their function becomes presentation rather than decision support. Metric theater.

Measurement has a single purpose in an executive context. It exists solely to reduce uncertainty so that a decision can be made with greater confidence than would otherwise be possible. That is it.

When measurement does not alter a decision, narrow the set of plausible options, or materially change confidence in an outcome, it has failed its purpose regardless of how polished or compliant it appears. That is a problem we needed to fix yesterday.

the decision failure

Let’s consider a routine governance decision that most boards now face:

The board must determine whether to accept, reduce, transfer, or avoid a specific category of cyber risk. Most of us have been through this discussion many times. It influences our capital allocation, our insurance coverage, our contractual commitments, and even our operational constraints. In preparation, the organization points to sustained investment in cybersecurity. Spending has increased year-over-year(yay!), and the security program reports improved maturity(maybe), broader control coverage(not likely), faster response times(probably), and fewer audit findings(i’ll leave this one alone). From an operational standpoint, our program appears to be improving.

During the discussion, a director asks a straightforward question that aligns directly with the decision at hand:

How did this year’s security investments change our exposure to a material cyber event?

I respond with information that is accurate and defensible.

1. New controls were implemented.

2. Known gaps were closed.

3. Programs matured.

4. Metrics improved across several dimensions.

Each statement that I gave reflects real work and real progress by the entire team. However my response does not support the decision the board is trying to make, and that bothers me every time I see it happen, including when I am the one presenting. Sigh.

It does not quantify how our uncertainty changed. It also does not describe whether the likelihood or impact of a relevant loss events moved in a meaningful way. It also does not distinguish which investments mattered more than others. Most importantly, it does not support a clear choice among accepting, reducing, transferring, or avoiding the risk that we had under discussion.

At this point, the board still has to act. So now what?

When measurement does not support the decision, boards follow a very predictable path. I know you have seen this. They defer the decision pending additional analysis. They rely on qualitative judgment and professional intuition. They substitute external opinion through consultants, insurers, or auditors—yes, the odd notion that external input is of higher value than internal input—I don’t understand that either.

Each of these responses allows the governance to proceed, but none of them truly reflected a measurement system that performed its intended function: that a properly functioning measurement system would have reduced uncertainty enough to make one option more defensible than the others. The absence of that outcome is the failure being examined here.

what measurement actually requires

Measurement has never required complete information(I admit that i’ve made this same mistake so many times). Finance, operations, and the risk functions regularly rely on partial data that holds up statistically and supports real decisions. It requires enough information to reduce uncertainty for a defined decision.

In our cybersecurity executive context, measurement earns its value only when it changes how a decision is evaluated. The question is/should never be whether a metric is precise in isolation. The question is whether it meaningfully narrows the range of plausible or realistic outcomes the decision makers are weighing.

Other disciplines internalized this principle long ago. I provide a case in point:

Finance measures variance and exposure because capital allocation decisions depend on understanding the downside, volatility, and sensitivity. Operations obviously measures throughput, capacity, and operational constraints because those measures directly inform tradeoffs between cost, speed, and reliability. Our good friends in Safety measure near-miss incident rates and leading indicators because regulatory thresholds and liability decisions depend on them.

In each case, measurement exists within a clearly defined decision context. Metrics are selected because they influence a choice that must be made, not because they are easy to collect or even widely accepted.

Unfortunately, cybersecurity measurement rarely follows this pattern.

Most cybersecurity metrics describe internal activity within the security function rather than external impact on the organization. I encourage you to prove me wrong. They report counts, percentages, maturity scores, and coverage levels. These measures provide visibility into effort and progress, but they do not specify how observed values affect the likelihood or magnitude of loss events that matter to the business. I am looking directly at you, Microsoft Secure Score! What a bunch of BS.

When a metric does not state how a change in value alters risk exposure, it cannot reduce uncertainty for a decision maker. It is really just THAT simple. It can however inform a status discussion, but it cannot support a choice among possible alternatives.

And knowing all of this, the result is information that is descriptive but not decision-grade or decision-inducing. It explains what the security organization is doing. It DOES NOT explain what leadership should do differently as a consequence.

what this leaves unanswered

Until we answer that question precisely, improvements in our reporting will not translate into better decisions. I feel very strongly about this. We will keep refining dashboards, expanding metrics(because more metrics must mean better understanding—right?), and meeting external expectations, while governance discussions continue to rely on judgment rather than evidence. In my view, this disconnect is a core reason we continue to see posture failures that ultimately lead to higher impact breaches and incidents.

In the next post, I will focus on the question that I keep coming back to when I am in front of a board trying to make or defend a decision: what cybersecurity measurement is actually responsible for delivering in that moment to help reduce uncertainty in a way governance can act on.

Subscribe now

The Hipster CISO isn’t about style or provocation (or trying to be clever for its own sake). It’s simply how I think about the role as a Chief Information Security Officer. I value true judgment over noise, and I’m far more interested in results that hold up under scrutiny than in activity that just looks good on paper. This Substack is where I put that thinking down, without smoothing the rough edges or pretending everything is neatly resolved. Let’s be honest, it definitely is not.

I’m writing for two reasons. First, I needed a space to think and grow (as a leader, and honestly as a person). Second, cybersecurity has become very good at narrating itself and much less disciplined about asking whether it’s actually working. Boo. Hiss. We re-produce (yes, that emphasis is intentional) artifacts, maturity language, and broadly agreeable stories, but we spend far less time examining effectiveness, trade-offs, or whether any of this is materially changing enterprise risk. I’m not particularly interested in preserving that dynamic.

Why “Hipster” Actually Matters

The term fits because it implies intentionality.

It rejects mass production in favor of precision (and yes, I am looking directly at you, AI-musing-generation experts). It favors quality over volume. I will question all defaults instead of inheriting them because “that’s how it’s always been done.” That just feels like nails on a chalkboard just writing it down.

This won’t be a feed of hot takes. It’s going to be a working notebook. Take it or leave it (and that’s perfectly okay).

Some of what shows up here will read like internal strategy memos. Some of it will come from board questions I didn’t have clean answers to at the time. There will be positions I hold with conviction, and others I’m still pressure-testing (or actively developing while I’m writing). The structure will vary, but the underlying premise won’t: cybersecurity is a business function of trust, and it should be governed with the same rigor as capital, safety, or strategy.

I’m not here to sell tools, validate orthodoxy, or soften conclusions for broader public appeal and discourse (although everyone’s voice always matters). If something is widely practiced but simply ineffective, I’ll call it out. If the data is inconclusive, I won’t pretend otherwise or shy away from saying so. I like data. I trust it more than most narratives.

This Substack fits because it reflects how I approach the work, the role, and life in general, to be honest. I’m selective, intentionally and by design. I care far less about how mature a program sounds than how it performs when something actually breaks.

How to Read This All

Read this the way you’d read something written by a peer who has nothing to sell and no interest in oversimplifying the work.

You don’t need to agree with everything here. In fact, disagreement is often a sign the writing is doing its job. If what you’re after is comfort, consensus, or the familiar “influencer” calls-to-action that are everywhere today, this may not be the right place. I wish you well regardless.

If you’re interested in sharper thinking about cybersecurity as governance, as leadership, and as an enterprise discipline that actually earns its seat at the big kid’s table, then you’re exactly who this is for.

Welcome to The Hipster CISO.

Github Source

Canonical Article on Github