PK
%\ _rels/PK
%\ docProps/PK
%\ ppt/PK
%\
ppt/_rels/PK
%\ ppt/charts/PK
%\ ppt/charts/_rels/PK
%\ ppt/embeddings/PK
%\
ppt/media/PK
%\ ppt/slideLayouts/PK
%\ ppt/slideLayouts/_rels/PK
%\ ppt/slideMasters/PK
%\ ppt/slideMasters/_rels/PK
%\ ppt/slides/PK
%\ ppt/slides/_rels/PK
%\
ppt/theme/PK
%\ ppt/notesMasters/PK
%\ ppt/notesMasters/_rels/PK
%\ ppt/notesSlides/PK
%\ ppt/notesSlides/_rels/PK
%\yB [Content_Types].xml
PK
%\] ] _rels/.rels
PK
%\! docProps/app.xml
0
0
Microsoft Office PowerPoint
On-screen Show (16:9)
0
8
8
0
0
false
Fonts Used
2
Theme
1
Slide Titles
8
Arial
Calibri
Office Theme
Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8
PptxGenJS
false
false
false
16.0000
PK
%\" " docProps/core.xml
ShieldScore — Technical Model Review
PptxGenJS Presentation
ShieldScore Analytics Team
ShieldScore Analytics Team
1
2026-04-02T04:47:50Z
2026-04-02T04:47:50Z
PK
%\g
ppt/_rels/presentation.xml.rels
PK
%\Oݨ ppt/theme/theme1.xmlPK
%\ڧ
ppt/presentation.xml
PK
%\X ppt/presProps.xml
PK
%\ ppt/tableStyles.xml
PK
%\D
>0 0 ppt/viewProps.xml
PK
%\H7t ! ppt/slideLayouts/slideLayout1.xml
PK
%\ђ7 7 , ppt/slideLayouts/_rels/slideLayout1.xml.rels
PK
%\;1 ppt/slides/slide1.xml
ShieldScoreTechnical Model Review & ValidationBinary classification for auto claims risk | Gradient boosted ensemble | Excel deploymentErie Insurance Group | Model Governance Review | April 2026PK
%\3 ppt/slides/_rels/slide1.xml.rels
PK
%\. ppt/notesSlides/notesSlide1.xml
1PK
%\:A * ppt/notesSlides/_rels/notesSlide1.xml.rels
PK
%\lM
d d ppt/slides/slide2.xml
The model predicts a binary outcome (claim >$5K: yes/no) on a class-imbalanced dataset, requiring cost-sensitive threshold selection rather than accuracy optimization Parameter Specification Rationale Task type Binary classification Predict claim occurrence, not amount Target HIGH_COST_CLAIM (1/0) Threshold at $5K aligns with Erie's intervention cost-benefit Event rate 18.2% Class imbalance → accuracy misleading (82% baseline) Primary metric AUC-ROC Threshold-agnostic; enables post-hoc optimization Decision metric Sensitivity @ FPR < 0.30 250:1 cost ratio favors catching risk over avoiding false alarms Observations 5,000 policyholders Synthetic dataset modeling Erie's 12-state territory Features (raw) 31 columns → 16 inputs + 6 engineered Demographics, policy, vehicle, driving, behavioral Leakage excluded CLAIM_AMOUNT removed Post-hoc variable; perfectly predicts target Partition: 60% train / 20% validation / 20% test — stratified random split preserving 18.2% event rate across all partitions1PK
%\2- ppt/slides/_rels/slide2.xml.rels
PK
%\ ppt/notesSlides/notesSlide2.xml
2PK
%\xշ * ppt/notesSlides/_rels/notesSlide2.xml.rels
PK
%\L L ppt/slides/slide3.xml
Data preparation addressed 8 quality issues and created 6 domain-informed features — missing indicators preserve information that deletion would loseDATA QUALITY REMEDIATION Issue Action n Duplicate POLICY_ID Dropped (keep first) 1 Missing AGE Median impute + indicator 169 Missing CREDIT_SCORE Median impute + indicator 258 Missing MILEAGE Median impute + indicator 199 Missing GENDER Mode impute 97 Negative MILEAGE Set NaN → median impute 2 VEH_YEAR = 1923 Capped at 2005 1 4-digit ZIPs Zero-padded to 5 digits 20 ENGINEERED FEATURES Feature Formula Type CLAIMS_PER_YEAR PRIOR_CLAIMS_3YR / 3 Ratio VEHICLE_AGE 2026 − VEHICLE_YEAR Derived PREMIUM_PER_VEH PREMIUM / NUM_VEH Ratio LOYALTY_INDEX YEARS × BUNDLED Interaction HIGH_SERVICE CALLS > 5 → 1 Binary CLAIM_RECENCY Binned LAST_CLAIM Ordinal Encoding: One-hot for COVERAGE_TYPE (4), RISK_ZONE (3), DIGITAL_ENGAGEMENT (3), GENDER (3), MARITAL (4), CLAIM_RECENCY (4). Final feature matrix: 5,000 × 28.2PK
%\W/ ppt/slides/_rels/slide3.xml.rels
PK
%\K |Ő ppt/notesSlides/notesSlide3.xml
3PK
%\9 Y * ppt/notesSlides/_rels/notesSlide3.xml.rels
PK
%\~?X: : ppt/slides/slide4.xml
Gradient boosting achieves the highest AUC (0.86) but the decision tree is retained as the explainability companion — both deploy together in production Model AUC Sens. Spec. Role Logistic Reg. 0.81 0.68 0.75 Baseline Decision Tree 0.76 0.62 0.72 Explainer Random Forest 0.84 0.70 0.74 Challenger Grad. Boost 0.86 0.72 0.73 Champion Dual deployment pattern: GBM generates the score (accuracy). Decision tree generates the IF-THEN explanation (interpretability). Standard practice for regulated industries where underwriters must justify flagging decisions.3PK
%\iasX X ppt/slides/_rels/slide4.xml.rels
PK
%\vs ppt/notesSlides/notesSlide4.xml
4PK
%\J * ppt/notesSlides/_rels/notesSlide4.xml.rels
PK
%\/ / ppt/slides/slide5.xml
Optimal threshold is 0.35 — catching 72% of high-risk policyholders at a 27% false positive rate, justified by the 250:1 cost asymmetry between missed claims and false alarms Threshold Sensitivity Specificity FPR Precision F1 0.20 0.85 0.58 0.42 0.31 0.45 0.30 0.78 0.68 0.32 0.37 0.50 0.35 ◀ 0.72 0.73 0.27 0.40 0.53 0.50 0.60 0.82 0.18 0.48 0.53 0.70 0.35 0.94 0.06 0.60 0.45 COST-BASED SELECTIONFN cost: $5,000+FP cost: ~$20Ratio: 250:1 At t=0.35, expected cost per1,000 policyholders scored: FN: 50 missed × $5K = $250KFP: 220 × $20 = $4.4KNet cost: $254KTEST SET VALIDATIONAUCTest: 0.85 (Val: 0.86)SensitivityTest: 0.71 (Val: 0.72)SpecificityTest: 0.74 (Val: 0.73)FPRTest: 0.26 (Val: 0.27)✓ No overfitting — performance stable across validation and test sets4PK
%\5 ppt/slides/_rels/slide5.xml.rels
PK
%\W8 ppt/notesSlides/notesSlide5.xml
5PK
%\Q e * ppt/notesSlides/_rels/notesSlide5.xml.rels
PK
%\e) ) ppt/slides/slide6.xml
The model deploys through two channels — nightly batch scoring for the retention team and real-time lookup for agent interactions — with an Excel scorer as the day-one tool Batch Real-Time Excel Scorer Trigger Cron, 11 PM nightly Agent opens account Manual entry Volume ~500K policyholders 1 per interaction 1 per use Latency Hours (overnight) Sub-second Instant (local) Output Score table → CRM Score + tier + actions Score + tier + actions Consumer Retention team dashboard Agent screen in-call Underwriter workstation Model form Full GBM ensemble Simplified tree rules 5-component formula MONITORING PLAN Frequency Metric Threshold Action Weekly AUC on rolling 30-day window < 0.78 Auto-alert to model owner Monthly Population Stability Index PSI > 0.20 Investigate data drift source Quarterly Full retrain + fairness audit Scheduled Rebuild on fresh 12-month window 5PK
%\ج+ ppt/slides/_rels/slide6.xml.rels
PK
%\z ppt/notesSlides/notesSlide6.xml
6PK
%\=| * ppt/notesSlides/_rels/notesSlide6.xml.rels
PK
%\IU ppt/slides/slide7.xml
Three features require ongoing fairness monitoring — credit score, ZIP-derived risk zone, and telematics enrollment all correlate with protected characteristics Feature Concern Mitigation Review Cycle CREDIT_SCORE Correlates with income/race; banned in some states for insurance scoring Quarterly disparate impact analysis; alternative model without credit for ban states Quarterly RISK_ZONE (ZIP) ZIP code is a strong proxy for race and socioeconomic status Evaluate driving-behavior alternatives; monitor flag rates by demographic group Quarterly TELEMATICS Lower-income customers less likely to enroll → model rewards enrollment access, not driving quality Analyze enrollment rates by income quartile; consider enrollment-neutral features Semi-annual Human override process: Underwriters can flag disagreements with model scores. Overrides are logged, reviewed monthly, and used as retraining signal. Model is advisory, not determinative — final intervention decisions remain with licensed underwriters.KNOWN LIMITATIONSExcel scorer uses simplified rules (5 components, max 100 pts) — less precise than full GBM for edge casesModel trained on synthetic data — production deployment requires validation on actual Erie claims dataText analytics (adjuster notes) not yet integrated — planned for Q3 2026 enhancement6PK
%\F ppt/slides/_rels/slide7.xml.rels
PK
%\)l ppt/notesSlides/notesSlide7.xml
7PK
%\|g * ppt/notesSlides/_rels/notesSlide7.xml.rels
PK
%\.!: !: ppt/slides/slide8.xml
TECHNICAL ROADMAPModel is validated, deployment tool is built, monitoring plan is defined — ready for pilot pending governance approvalQ2 2026Pilot deployment (500 policyholders, Erie region)Validate on actual claims dataA/B test: ShieldScore group vs. controlQ3 2026Integrate adjuster notes (text analytics)Add weather-event proactive alert systemFirst quarterly retrain on production dataQ4 2026Full fairness audit on pilot resultsDecision point: scale or iterateEvaluate model for homeowners extensionQ1 2027If pilot succeeds (15% target met):Scale to full 12-state territoryAPI deployment for real-time scoring7PK
%\6 ppt/slides/_rels/slide8.xml.rels
PK
%\iސ ppt/notesSlides/notesSlide8.xml
8PK
%\pO * ppt/notesSlides/_rels/notesSlide8.xml.rels
PK
%\K ! ppt/slideMasters/slideMaster1.xml
PK
%\N) , ppt/slideMasters/_rels/slideMaster1.xml.rels
PK
%\6T T ! ppt/notesMasters/notesMaster1.xml
7/23/19Click to edit Master text stylesSecond levelThird levelFourth levelFifth level‹#›PK
%\s * * , ppt/notesMasters/_rels/notesMaster1.xml.rels
PK
%\RPB PB . ppt/embeddings/Microsoft_Excel_Worksheet1.xlsxPK
%\ _rels/PK
%\ docProps/PK
%\ xl/PK
%\ xl/_rels/PK
%\
xl/tables/PK
%\ xl/theme/PK
%\ xl/worksheets/PK
%\ xl/worksheets/_rels/PK
%\ # # [Content_Types].xml
PK
%\K K _rels/.rels
PK
%\* docProps/app.xmlMicrosoft Macintosh Excel0falseWorksheets1Sheet1falsefalsefalse16.0300
PK
%\2 *i i docProps/core.xmlPptxGenJSPptxGenJS2026-04-02T04:47:50.717Z2026-04-02T04:47:50.717ZPK
%\ՙ xl/_rels/workbook.xml.relsPK
%\<] ]
xl/styles.xml
PK
%\^ xl/theme/theme1.xmlPK
%\T xl/workbook.xml
PK
%\I3j$ $ # xl/worksheets/_rels/sheet1.xml.rels
PK
%\Uaz z xl/sharedStrings.xmlValidation AUCLogistic
RegressionDecision Tree
(depth=5)Random Forest
(n=300)Gradient
Boosting (n=200)
PK
%\tϩ xl/tables/table1.xmlPK
%\7[ [ xl/worksheets/sheet1.xml01
20.81
30.76
40.84
50.86
PK
%\ _rels/PK
%\ $ docProps/PK
%\ K xl/PK
%\ l xl/_rels/PK
%\
xl/tables/PK
%\ xl/theme/PK
%\ xl/worksheets/PK
%\ xl/worksheets/_rels/PK
%\ # # @ [Content_Types].xmlPK
%\K K _rels/.relsPK
%\* docProps/app.xmlPK
%\2 *i i P docProps/core.xmlPK
%\ՙ xl/_rels/workbook.xml.relsPK
%\<] ]
xl/styles.xmlPK
%\^ ` xl/theme/theme1.xmlPK
%\T 1 xl/workbook.xmlPK
%\I3j$ $ # 3 xl/worksheets/_rels/sheet1.xml.relsPK
%\Uaz z O5 xl/sharedStrings.xmlPK
%\tϩ 6 xl/tables/table1.xmlPK
%\7[ [ 8 xl/worksheets/sheet1.xmlPK g= PK
%\(ǰ> > ppt/charts/_rels/chart1.xml.relsPK
%\?'( ( ppt/charts/chart1.xml Sheet1!$B$1 Validation AUC Sheet1!$A$2:$A$5 Logistic
RegressionDecision Tree
(depth=5)Random Forest
(n=300)Gradient
Boosting (n=200) Sheet1!$B$2:$B$5 General 0.810.760.840.86 PK
%\ _rels/PK
%\ $ docProps/PK
%\ K ppt/PK
%\
m ppt/_rels/PK
%\ ppt/charts/PK
%\ ppt/charts/_rels/PK
%\ ppt/embeddings/PK
%\
ppt/media/PK
%\ B ppt/slideLayouts/PK
%\ q ppt/slideLayouts/_rels/PK
%\ ppt/slideMasters/PK
%\ ppt/slideMasters/_rels/PK
%\
ppt/slides/PK
%\ 3 ppt/slides/_rels/PK
%\
b ppt/theme/PK
%\ ppt/notesMasters/PK
%\ ppt/notesMasters/_rels/PK
%\ ppt/notesSlides/PK
%\ ppt/notesSlides/_rels/PK
%\yB P [Content_Types].xmlPK
%\] ] . _rels/.relsPK
%\! docProps/app.xmlPK
%\" " ! docProps/core.xmlPK
%\g
I% ppt/_rels/presentation.xml.relsPK
%\Oݨ - ppt/theme/theme1.xmlPK
%\ڧ
N ppt/presentation.xmlPK
%\X _\ ppt/presProps.xmlPK
%\ ] ppt/tableStyles.xmlPK
%\D
>0 0 ^ ppt/viewProps.xmlPK
%\H7t ! a ppt/slideLayouts/slideLayout1.xmlPK
%\ђ7 7 , d ppt/slideLayouts/_rels/slideLayout1.xml.relsPK
%\;1 bf ppt/slides/slide1.xmlPK
%\3 _w ppt/slides/_rels/slide1.xml.relsPK
%\. ky ppt/notesSlides/notesSlide1.xmlPK
%\:A * 8 ppt/notesSlides/_rels/notesSlide1.xml.relsPK
%\lM
d d K ppt/slides/slide2.xmlPK
%\2- ? ppt/slides/_rels/slide2.xml.relsPK
%\ A ppt/notesSlides/notesSlide2.xmlPK
%\xշ * H ppt/notesSlides/_rels/notesSlide2.xml.relsPK
%\L L J ppt/slides/slide3.xmlPK
%\W/ ݗ ppt/slides/_rels/slide3.xml.relsPK
%\K |Ő ppt/notesSlides/notesSlide3.xmlPK
%\9 Y * ppt/notesSlides/_rels/notesSlide3.xml.relsPK
%\~?X: : ɢ ppt/slides/slide4.xmlPK
%\iasX X 6W ppt/slides/_rels/slide4.xml.relsPK
%\vs Y ppt/notesSlides/notesSlide4.xmlPK
%\J * ` ppt/notesSlides/_rels/notesSlide4.xml.relsPK
%\/ / b ppt/slides/slide5.xmlPK
%\5 ppt/slides/_rels/slide5.xml.relsPK
%\W8 ppt/notesSlides/notesSlide5.xmlPK
%\Q e * ̚ ppt/notesSlides/_rels/notesSlide5.xml.relsPK
%\e) ) ߜ ppt/slides/slide6.xmlPK
%\ج+ ppt/slides/_rels/slide6.xml.relsPK
%\z ppt/notesSlides/notesSlide6.xmlPK
%\=| * ppt/notesSlides/_rels/notesSlide6.xml.relsPK
%\IU ppt/slides/slide7.xmlPK
%\F Y ppt/slides/_rels/slide7.xml.relsPK
%\)l ,[ ppt/notesSlides/notesSlide7.xmlPK
%\|g * a ppt/notesSlides/_rels/notesSlide7.xml.relsPK
%\.!: !: d ppt/slides/slide8.xmlPK
%\6 ` ppt/slides/_rels/slide8.xml.relsPK
%\iސ l ppt/notesSlides/notesSlide8.xmlPK
%\pO * 9 ppt/notesSlides/_rels/notesSlide8.xml.relsPK
%\K ! L ppt/slideMasters/slideMaster1.xmlPK
%\N) , " ppt/slideMasters/_rels/slideMaster1.xml.relsPK
%\6T T ! * ppt/notesMasters/notesMaster1.xmlPK
%\s * * , ppt/notesMasters/_rels/notesMaster1.xml.relsPK
%\RPB PB . 1 ppt/embeddings/Microsoft_Excel_Worksheet1.xlsxPK
%\(ǰ> > + ppt/charts/_rels/chart1.xml.relsPK
%\?'( ( I- ppt/charts/chart1.xmlPK F F @