Chelsea AI Ventures The Challenge: Hidden Liability in Safe Segments
The client, a high-volume insurance provider, was bleeding margin in a segment historically considered low risk. Their existing underwriting process relied on static rule-sets which had grown over many years. They were hard to maintain and incapable of detecting complex, non-linear risk factors.
The real economic loss came from a small percentage of high-severity claims slipping through standard filters. They didn’t need a generative chatbot; they needed a precise, interpretable risk compass.
Our Approach: Benchmarks and Interpretability
Instead, we applied White Box models:
We established a Golden Dataset of historical claims to define a quantitative baseline. We found that standard accuracy metrics were misleading due to class imbalance: a model could be 99% accurate by simply rejecting every risk. We shifted the “North Star” metric to Precision at Top Decile (Lift).
We chose gradient boosted decision trees, a model that learns from the table-shaped data an insurer already holds and returns the same answer every time it sees the same case, so an auditor can check it.
We deployed an evaluation pipeline to detect concept drift, ensuring the model’s risk definitions evolved with changing market conditions.
Technical Implementation
Interpretable “Glass-Box” Architecture
In insurance a score nobody can explain is a liability. We used SHAP values, which split a score into the contribution of each individual factor, so an underwriter can see in plain terms why a case scored as it did.
- Constraint-Based Engineering: We purposely limited the feature set to ensure robustness, adhering to the principle that “Constraints force creativity”.
- Human-in-the-Loop Safeguards: The model does not auto-reject. Instead, it acts as a triage layer, routing high-risk applications to senior underwriters with a detailed “risk context” dossier. This raises the ceiling for experts rather than attempting to replace them.
Business Impact
We achieved an improvement in important business metrics:
- Loss Ratio Improvement: The 28% reduction in high-severity claims was achieved not by pricing higher, but by selecting better risks (avoidance).
- Auditability: Unlike vibe-based LLM systems, every decision is backed by a deterministic log of feature contributions, satisfying strict regulatory compliance.
- Operational Efficiency: The system filtered noise, allowing underwriters to focus their “attention budget” on complex cases where human judgment is irreplaceable.
Technology Stack
- Model: XGBoost (Gradient Boosted Trees) - Chosen for performance/auditability balance.
- Explainability: SHAP & ELI5 - For per-prediction transparency.
- Orchestration: Python/SQL pipeline with automated drift detection.
