3.1 Study design
This is a simulation study. A synthetic enterprise-week panel was generated so that analytical methods could be compared under known conditions; no real organizational records were used. Published 2025 industry reports informed the choice of feature domains and the direction of their associations with disruption, but no report supplied firm-level data, and generator parameters were not estimated from observational data. Because the outcome is generated from the predictors, which features matter and by how much is determined by the generator; the study therefore evaluates methods rather than real-world risk factors.
3.2 Synthetic panel
The panel contains 600 enterprise profiles observed weekly for 52 weeks (31,200 enterprise-weeks) across ten sectors: financial services, healthcare, manufacturing, retail, technology, professional services, transportation, energy, education, and government contracting (Table 1). Each row contains twenty predictors, a binary indicator of material disruption within the following fourteen days, downtime, and estimated financial loss. Only the disruption indicator is a prediction target; downtime and loss are used in scenario illustrations only. A late-year increase in threat pressure was included so that the test period differs from the training period, mimicking non-stationarity.
Supplementary Material S1 Reproducible replacement experiment: This supplement specifies the newly executed experiment used in the completed technical revision of manuscript 10974. It replaces the earlier draft's undocumented numerical illustration. No attempt is made to claim that the old performance values were reproduced. Every enterprise, sector effect, predictor, incident, downtime value, and financial loss is artificial. The associated code and exports are provided so that the revised numerical claims can be checked independently.
3.3 Predictors
Table 1. Synthetic panel design. All records and outcomes are synthetic. Published industry reports informed the choice of feature domains but did not supply firm-level data.
|
Design element
|
Specification
|
|
Unit of analysis
|
Enterprise-week snapshot
|
|
Population and period
|
600 simulated enterprise profiles × 52 weeks = 31,200 enterprise-weeks
|
|
Sectors (10)
|
Financial services, healthcare, manufacturing, retail, technology, professional services, transportation, energy, education, government contracting
|
|
Predictors
|
20 (Table 2)
|
|
Target
|
Material disruption within the following 14 days (binary)
|
|
Non-target outcomes
|
Downtime; estimated financial loss (scenario illustrations only)
|
|
Training set
|
Weeks 1–39; 23,400 enterprise-weeks
|
|
Test set
|
Weeks 40–52; 7,800 enterprise-weeks; 1,621 positive (20.8%)
|
|
Non-stationarity
|
Increased threat pressure in late-year weeks
|

Figure 1. Closed-loop business intelligence and machine learning framework. Telemetry is detected, converted to 14-day disruption forecasts, prioritized within review capacity, routed to owned controls and playbooks, and verified outcomes feed back into monitoring and retraining. The governed semantic layer supplies shared definitions to every stage. Conceptual design; not deployed in this study.
Table 2. Predictor dictionary. Note. Eighteen predictors are named in the original study materials; the authors must identify the remaining two. Descriptions are inferred from variable names and must be confirmed against the generator.
|
Domain
|
Variable
|
Description [AUTHOR INPUT: units and distribution for each]
|
|
Identity
|
failed_logins
|
Count of failed authentication attempts in the week.
Unit: authentication attempts/week. Distribution: Poisson(mean=12).
|
|
Identity
|
identity_risk
|
Composite identity-risk score.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=2,5).
|
|
Identity
|
impossible_travel
|
Count of geographically implausible sign-ins.
Unit: sign-ins/week. Distribution: Poisson(mean=0.6).
|
|
Identity
|
privilege_anomaly
|
Count of anomalous privilege changes.
Unit: privilege changes/week. Distribution: Poisson(mean=0.8).
|
|
Endpoint and email
|
malware_free_activity
|
Suspicious activity without a conventional malware indicator.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=1.5,6).
|
|
Endpoint and email
|
endpoint_alerts
|
Count of endpoint detection alerts.
Unit: alerts/week. Distribution: Poisson(mean=5).
|
|
Endpoint and email
|
email_phish_score
|
Phishing exposure score.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=2,6).
|
|
Cloud and vulnerability
|
cloud_misconfig
|
Count or score of cloud misconfigurations.
Unit: unresolved misconfigurations at the snapshot (count). Distribution: Poisson(mean=3).
|
|
Cloud and vulnerability
|
exposed_kev
|
Count of exposed CISA Known Exploited Vulnerabilities.
Unit: exposed vulnerabilities at the snapshot (count). Distribution: Poisson(mean=1).
|
|
Cloud and vulnerability
|
patch_latency_days
|
Days from patch availability to remediation.
Unit: days. Distribution: Gamma(shape=2,scale=8).
|
|
Cloud and vulnerability
|
external_surface
|
External attack-surface score.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=2,4).
|
|
Supplier and data
|
third_party_risk
|
Composite third-party risk score.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=2,3).
|
|
Supplier and data
|
vendor_dependency
|
Dependence on critical suppliers.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=2,2).
|
|
Supplier and data
|
data_egress_gb
|
Outbound data volume (GB).
Unit: gigabytes/week. Distribution: Lognormal(log-mean=2.5,log-SD=0.7).
|
|
Supplier and data
|
shadow_ai_risk
|
Ungoverned AI-use exposure score.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=1.5,6).
|
|
Business context
|
backup_health
|
Backup readiness score.
Unit: dimensionless score, 0–1; higher is better. Distribution: Beta(shapes=6,2).
|
|
Business context
|
business_criticality
|
Criticality of supported services.
Unit: ordinal integer, 1–5. Distribution: Discrete uniform {1,2,3,4,5}.
|
|
Business context
|
security_maturity
|
Security-program maturity score.
Unit: dimensionless score, 0–1; higher is better. Distribution: Beta(shapes=5,2).
|
|
Identity
|
mfa_coverage
|
Proportion of accounts protected by multi-factor authentication (MFA).
Unit: fraction, 0–1. Distribution: Beta(shapes=7,2).
|
|
Cloud and vulnerability
|
remote_access_exposure
|
Remote-access exposure score.
Unit: dimensionless score, 0–1. Distribution: Beta(shapes=2,5).
|
Table 3. Model specification. Note. Highlighted cells must be completed by the authors from the original analysis code.
|
Item
|
Logistic regression
|
Random forest
|
Gradient boosting
|
|
Implementation
|
scikit-learn 1.8.0
Logistic Regression
|
scikit-learn 1.8.0
Random Forest Classifier
|
scikit-learn 1.8.0
Gradient Boosting Classifier
|
|
Key hyperparameters
|
L2 regularization;
C = 1.0;
solver = lbfgs;
max_iter = 2,000.
|
n_estimators = 200;
max_depth = 10;
min_samples_leaf = 25;
max_features = sqrt;
bootstrap = True;
n_jobs = 1.
|
n_estimators = 150;
learning_rate = 0.05;
max_depth = 2;
min_samples_leaf = 25;
subsample = 1.0;
loss = log_loss.
|
|
Tuning procedure
|
Fixed settings; no search or cross-validation.
|
Fixed settings; no search or cross-validation.
|
Fixed settings; no search or cross-validation.
|
|
Class weighting
|
Yes. class weight = balanced.
|
Yes. class_weight = balanced
|
Yes. Balanced weights supplied through sample_weight.
|
|
Feature scaling
|
StandardScaler fitted on training data only.
|
Not required; no scaling applied.
|
Not required; no scaling applied.
|
|
Threshold (≈15% alerts)
|
Weekly top 90/600.
Median cutoff 0.7000; range 0.6844–0.7136.
|
Weekly top 90/600.
Median cutoff 0.5955; range 0.5846–0.6124.
|
Weekly top 90/600.
Median cutoff 0.6548; range 0.6307–0.6757.
|
Table 4. Temporal test-set performance (n = 7,800; 1,621 positives). Note. BSS = Brier skill score relative to a constant forecast equal to the test prevalence (0.208); negative values indicate worse-than-reference probabilities. ROC-AUC intervals use the Hanley–McNeil approximation; precision and recall intervals are Wilson intervals. Each model flags 1,152 enterprise-weeks (14.8%). F1 = 0.391, 0.366, and 0.367. The no-skill row assumes random selection of the same alert volume.
|
Model
|
ROC-AUC (95% CI)
|
PR-AUC
|
Brier
|
BSS
|
Precision (95% CI)
|
Recall (95% CI)
|
|
Logistic regression
|
0.701 (0.686–0.716)
|
0.407
|
0.206
|
−0.251
|
47.0% (44.2–49.9)
|
33.4% (31.2–35.8)
|
|
Random forest
|
0.694 (0.679–0.709)
|
0.395
|
0.189
|
−0.148
|
44.1% (41.3–47.0)
|
31.3% (29.1–33.6)
|
|
Gradient boosting
|
0.693 (0.678–0.708)
|
0.396
|
0.150
|
0.089
|
44.2% (41.3–47.1)
|
31.4% (29.2–33.7)
|
|
No-skill reference
|
0.500
|
0.208
|
0.165
|
0.000
|
20.8%
|
14.8%
|
Table 5. Assumption-driven scenario outputs (illustrative, not estimates). Note. Control indices must not be added or multiplied to infer combined effects. Mean predicted risk inherits the calibration error reported in Table 4.
|
Scenario
|
Output
|
Interpretation
|
|
Third-party exposure quintiles
|
Modeled disruption rate ≈15% (Q1) to ≈29% (Q5)
|
Gradient built into generator
|
|
MFA and conditional access
|
Residual-risk index 100 → ≈57
|
Assumed control effect
|
|
Accelerated KEV patching
|
100 → ≈68
|
Assumed control effect
|
|
Network segmentation
|
100 → ≈72
|
Assumed control effect
|
|
Endpoint and identity analytics
|
100 → ≈54
|
Assumed control effect
|
|
Third-party access review
|
100 → ≈65
|
Assumed control effect
|
|
Immutable backups
|
100 → ≈61
|
Assumed control effect
|
|
Cost–benefit curve
|
Net value peaks at ≈14.9% alert rate
|
Depends on assumed costs
|
|
Stress: baseline / phishing / zero-day / supplier outage / shadow-AI / combined
|
Mean predicted risk ≈44 / 54 / 60 / 58 / 52 / 76%; continuity index 1.00 / 0.86 / 0.78 / 0.73 / 0.88 / 0.61
|
Assumed shock multipliers
|
|
Drift monitoring (12 months)
|
PSI exceeds 0.20 review reference, then declines
|
Illustrative monitoring trace
|
Twenty predictors spanned identity, endpoint and email, cloud and vulnerability, supplier and data, and business-context domains (Table 2). All predictors were constructed from information available at or before the snapshot week. Variable names in Figure 3 correspond to the identifiers in Table 2.
3.4 Temporal partition and leakage control
Weeks 1–39 (23,400 observations) were used for training and weeks 40–52 (7,800 observations) for testing, so that each model was evaluated on later observations than it was trained on. Because the outcome horizon (fourteen days) exceeds the snapshot interval (one week), labels for the final training weeks overlap the test period. In the replacement simulation, all three models were refitted using weeks 1–37 (22,200 observations), with weeks 38–39 embargoed and weeks 40–52 reserved for testing (7,800 observations). The final training outcome window ended at week 39, preventing fourteen-day labels from extending into the test forecast period. The twenty predictors, prespecified hyper parameters, weighting schemes, and random seed (20261028) were retained; preprocessing was fitted exclusively on training data. Compared with training on weeks 1–39, the maximum absolute change was 0.000864 in ROC-AUC and 0.002362 in average precision. Logistic regression retained the highest discrimination on both metrics, while random forest and gradient boosting exchanged average-precision ranks.
The same 600 enterprises appear in both partitions, so the design tests temporal generalization within known enterprises, not transfer to new organizations.
3.5 Models
Three model families were compared: L2-regularized logistic regression as a transparent baseline, random forest (Breiman, 2001) to capture nonlinear interactions, and gradient boosting (Friedman, 2001). All models used class weighting to address imbalance. Hyperparameters were selected using training data only (Table 3). The operating threshold for each model was chosen to produce an alert volume of approximately 15% of enterprise-weeks, representing an assumed review capacity; for logistic regression this corresponded to a predicted probability of 0.644.
3.6 Evaluation
Discrimination was measured by ROC-AUC and PR-AUC. In the replacement analysis, PR-AUC was defined as non-interpolated average precision (AP) and computed using sklearn. metrics. average_precision_score(y_true, y_score) in scikit-learn 1.8.0. AP summarizes precision across score thresholds, weighting each precision value by the corresponding increase in recall (scikit-learn developers, n.d.):
AP = k = 1K(Rk - Rk-1)Pk
Here, Pₖ and Rₖ denote precision and recall at threshold k, K is the number of threshold steps, and R₀ = 0 when recall is ordered from zero upward. This calculation uses recall increments without trapezoidal interpolation.
The calculation used all 7,800 held-out enterprise-weeks from weeks 40–52, with fourteen-day disruption coded as the positive class (y = 1). Scores were each model’s continuous positive-class probabilities from predict_proba(X_test) [: 1]; no evaluation sample weights were applied. AP assessed ranking across thresholds, while precision and recall under the 15% weekly review-capacity policy were evaluated separately.
Because individual test-set predictions were not available for this revision, 95% confidence intervals for ROC-AUC were approximated with the Hanley–McNeil method (Hanley & McNeil, 1982); a paired DeLong test (DeLong et al., 1988) should replace this once predictions are re-generated For the replacement analysis, paired DeLong tests comparing ROC-AUC yielded \(p=8.61\times10^{-7}\) for logistic regression versus random forest, \(p=8.97\times10^{-6}\) for logistic regression versus gradient boosting, and \(p=0.392\) for random forest versus gradient boosting. PR-AUC, defined as average precision, and its 95% bootstrap confidence interval were 0.5620 (0.5172–0.6059), 0.5402 (0.4934–0.5863), and 0.5423 (0.4982–0.5885), respectively. Intervals used 500 paired enterprise-cluster resamples, retaining each enterprise’s thirteen-week test trajectory with fitted models held fixed. DeLong p-values are descriptive because their conventional calculation ignores within-enterprise dependence; clustered bootstrap intervals provide the primary uncertainty assessment. Probability quality was measured by the Brier score and the Brier skill score, BSS = 1 − BS/BSref, where BSref = p(1 − p) is the score of a constant forecast equal to the test prevalence p; BSS > 0 indicates improvement over that reference. At the operating threshold we report precision, recall, and F1 with Wilson 95% confidence intervals (Wilson, 1927), and the lift of precision over prevalence.
3.7 Explanations
Global importance was measured by permutation importance on the test set as the decrease in PR-AUC after randomly permuting one predictor For the replacement analysis, permutation importance was calculated for the fitted logistic regression model using 20 repeats per predictor and numpy. random.default_rng(20261228). For each repeat, one predictor’s complete thirteen-week trajectory was randomly reassigned among the 600 enterprises in the test set. Outcomes, other predictors, and the fitted model remained unchanged; no retraining was performed. Importance was calculated as:
Ij = 120r = 120(APoriginal - APj,rpermuted)
Here, Iⱼ is the importance of predictor j, r indexes the twenty permutation repeats, and AP denotes non-interpolated average precision. Each term is the original test-set AP minus the AP obtained after permuting predictor j in repeat r.
Larger positive values indicate greater deterioration in average precision after permutation. Figure error bars represent the sample standard deviation across the twenty repeats (ddof = 1), not confidence intervals. These importance values describe model dependence and do not establish causal effects.
A local explanation of a single high-risk enterprise-week was produced as signed contributions on the log-odds scale In the replacement analysis, local explanations used an exact decomposition of logistic-regression log odds:
φj = βjxj - μjsj
logit(p) = β0 + j = 120φj
Here, φⱼ is predictor j’s signed contribution, xⱼ is its observed value, and μⱼ and sⱼ are its mean and standard deviation estimated exclusively from training data. The coefficient β̂ⱼ is the fitted logistic-regression coefficient for the standardized predictor, and β̂₀ is the fitted intercept.
The reference point was the vector of training-set feature means, corresponding to zero after standardization; baseline log odds equaled the fitted intercept. Adding all twenty contributions to this intercept exactly reconstructed the model’s predicted log odds.
Positive contributions increased predicted log odds relative to this reference, while negative contributions decreased them. These attributions describe model behavior, not causal effects.
Both describe model behavior, not intervention effects.
3.8 Decision scenarios
Several assumption-driven scenarios illustrate how forecasts could be used in planning: a third-party exposure gradient, six control scenarios expressed on a residual-risk index (baseline = 100), a cost–benefit curve across alert rates, five stress scenarios, and a population stability index (PSI) monitoring trace. The control reductions and stress effects were specified by the authors, not estimated. Scenario assumptions for the replacement analysis.
All values below are explicit assumptions used in the executed replacement simulation. Control indices, prevention probability, costs, and continuity indices were stipulated; they are not estimated treatment effects or observed enterprise benefits. Supplementary table S2A, S2B, S2C.
V = 0.10 × 20,000 × Ncovered - 250 × Nreviews
V is assumed expected net value in USD relative to no review. A covered incident has at least one alert for the same enterprise one or two weeks earlier. Each incident is counted once even when alert windows overlap. Uncovered incidents retain the full assumed loss; covered incidents retain 90% in expectation. The fixed USD 20,000 scenario loss is distinct from the generator’s variable synthetic loss field.
The continuity indices are separately stipulated assumptions: baseline 1.00, phishing 0.86, vulnerability pressure 0.78, supplier outage 0.73, shadow AI 0.88, and combined 0.61. They are not derived from predicted probabilities or from observed business continuity outcomes.
The historical “Zero-day” scenario label denotes illustrative vulnerability pressure represented by two additional exposed known exploited vulnerabilities and longer patch latency. It is not a validated simulation of an unknown vulnerability. Predicted probabilities remain raw model outputs and inherit the calibration limitations of the replacement analysis.
3.8.1 Alert and incident counting rules
For each test forecast week (40–52), enterprises are ranked by descending raw logistic score; ascending enterprise ID resolves ties. The review count is floor (capacity × 600). Utility covers underlying incident weeks 41–54. An incident is covered only by an available test-period alert from its preceding one- or two-week window; the union of those windows prevents duplicate counting. Expected prevention benefits are calculated, not simulated as additional incident outcomes. The capacity maximizing this utility is selected with hindsight and is not a prospectively validated operating policy. Supplementary table S2D.
3.8.2 Implementation record
All numeric assumptions are transcribed from scenario_assumptions. json and the scenario analysis function in analysis.py. The fixed models belong to the executed replacement experiment (model random seed 20261028). Scenario transformations are deterministic and introduce no additional random draws. These specifications do not reconstruct the original manuscript’s undocumented scenario calculations.
Scenario outputs are reported in Table 5 and are not used to support empirical claims.
3.9 BI architecture
The proposed architecture (Figure 1) ingests identity, endpoint, email, cloud, vulnerability, and supplier telemetry through a streaming layer into a lakehouse and feature store. A governed semantic model defines shared enterprise, asset, time, threat, identity, and vendor dimensions, so that executive and security operations center (SOC) dashboards use the same measure definitions. Each forecast record stores the snapshot time, horizon, model version, explanation, reviewer, action, and verified outcome. This architecture is a design proposal; it was not deployed in this study.