Customer acquisition costs five to twenty-five times more than retention. Every telecom organization knows this. Most of them still discover churn after it happens a customer who stopped paying, a contract that silently lapsed, a relationship that degraded invisibly until it ended.
The reason is not a lack of data. Telecom companies have extraordinary amounts of customer data. The reason is the gap between having data and having a system that turns that data into timely, specific, actionable intelligence.
That gap is what this platform closes.
The Business Frame First
Before writing a single line of code, I defined what a useful churn prediction system actually needs to deliver. Not to a data scientist to a retention team manager deciding how to allocate five agents' time across 200 flagged customers on a Tuesday afternoon.
That manager needs to know, for each customer: how likely are they to churn, why, what intervention has the highest probability of changing that outcome, and what is the revenue at stake. A probability score alone is useless in that context. A risk segment label without explanation is marginally better. A complete decision package score, segment, drivers, recommended action, intervention simulation is what makes the difference between a model and a tool.
Every technical decision in this project was evaluated against that frame.
The Dataset and Its Real Problem
The IBM Telco Customer Churn dataset contains 7,010 customer records with 21 features spanning demographics, service subscriptions, contract type, payment method, and billing details. The churn rate is 26.5% — imbalanced but not extreme.
The real problem with this dataset is not the class imbalance. It is that the features, taken at face value, tell an incomplete story.
Tenure and contract type are strong predictors. Monthly charges matter. But what the dataset does not contain and what a real telecom system would have is behavioral data: support ticket volume, payment delays, login frequency, service usage patterns. These are the signals that precede churn in practice. They are not in the raw Kaggle data.
I addressed this by engineering synthetic behavioral features derived from observable proxies never from the target label. Support tickets were simulated from tenure and internet service type. Payment delay counts were derived from payment method and charge level. Login frequency was approximated from service count and contract type. The key constraint: every synthetic feature had to be derivable from data a real system would possess, with no information leakage from the churn outcome itself.
This distinction matters. Early versions of the feature engineering used simulated features that inadvertently encoded churn information producing near-perfect AUC scores (0.99) that were meaningless. The data leakage was caught, traced to target-label-driven simulation, and corrected. The lesson: when synthetic features produce suspiciously good performance, investigate before celebrating.
Why the Default Threshold Is Wrong
The default classification threshold of 0.5 is almost never optimal for imbalanced classification problems with asymmetric costs. For churn prediction, it is definitively wrong.
The cost structure is asymmetric by design: a missed churner costs the business their full customer lifetime value approximately $1,541 in this dataset, calculated from average monthly charge times average tenure. An unnecessary retention offer costs roughly $154 in discount expense. The ratio is roughly 10:1.
Setting the threshold at 0.5 implicitly treats these costs as equal. They are not. The optimal threshold, derived by minimizing total dollar cost over the test set, is 0.106 substantially lower than the default, correctly prioritizing recall over precision to capture more true churners at the acceptable cost of more false positives.
This is not a statistical trick. It is a business decision made explicit. The model's job is not to maximize F1 score. It is to minimize the cost the business actually cares about. Those are different objectives, and confusing them produces systems that perform well on benchmarks and poorly in production.
Probability Calibration: Why It Matters
Raw XGBoost probabilities are overconfident. The model learns to produce well-separated scores high for churners, low for retained customers but those scores are not calibrated probabilities in the frequentist sense. A customer assigned a score of 0.7 does not necessarily churn 70% of the time.
For a risk segmentation system one where "High risk" means genuinely high likelihood of churn, not just a relatively high model score this distinction is critical. Risk segments derive their meaning from calibrated probabilities. Without calibration, the segments are ordinal labels, not probability estimates.
Isotonic regression calibration was applied to bring the probability curve closer to the diagonal. The Brier score improved from 0.1479 to 0.1373. More importantly, the risk segment boundaries (High: ≥0.70, Medium: 0.40–0.69, Low: <0.40) became defensible to a retention team manager who asks: what does "High risk" actually mean? The answer, after calibration, is: it means this customer has approximately a 70%+ probability of churning. That is an answer worth acting on.
The SHAP Layer: From Prediction to Explanation
Model accuracy is necessary but not sufficient for a retention platform. A retention agent asking why Customer #82 is flagged at 91.6% churn probability needs more than the number. They need the reasoning.
SHAP (SHapley Additive exPlanations) provides exactly this. For each prediction, the system computes the contribution of each feature to the final probability positive contributions pushing toward churn, negative contributions pushing away from it.
For Customer #82, the SHAP waterfall tells a clear story: month-to-month contract (+0.59 above baseline), short tenure (+0.48), fiber optic internet with no online security, electronic check payment. Each factor is quantified, directional, and actionable.
The SHAP layer also powers the intervention simulation the feature that makes this a decision platform rather than a scoring system.
Intervention Simulation: What Should We Do?
For Customer #82, at 86.9% baseline churn probability, the simulation asks: what happens to this customer's churn probability under different retention interventions?
Upgrading to a two-year contract reduces churn probability by 66.5 percentage points from 86.9% to 20.4%. Upgrading to a one-year contract reduces it by 48.0 points. Resolving all outstanding payment delays reduces it by 5.9 points. Increasing login frequency (engagement re-activation) has almost no effect: +3.0 points in the wrong direction.
This last finding is counterintuitive and important. Re-engagement campaigns, the instinct of many retention teams have near-zero impact on high-risk churners whose churn is driven by contract structure and pricing dissatisfaction. The model's intervention simulation identifies where to spend retention resources and, equally importantly, where not to.
The SHAP-powered intervention approach is mechanistically honest: changing a feature value and re-running inference gives you the model's prediction under that scenario. It is not a causal model correlation is not causation, and changing a customer's contract type does not control all the other variables that correlate with contract type. But it is far more useful than a black-box score, and it produces recommendations that align with domain expertise: contract conversion is the most powerful retention lever in telecom, which is exactly what the model discovers independently.
The Revenue Impact Quantification
The platform's analytics tab surfaces the business impact in terms any executive can evaluate:
1,857 at-risk customers. $138,540 in monthly recurring revenue at risk. $448,870 in estimated annual retention opportunity, calculated assuming 30% successful conversion at a 10% discount cost.
These are not marketing projections. They are derived directly from the model's predictions, the customer-level revenue data, and a conservative retention scenario. The assumptions are transparent and the numbers are auditable.
This is what a retention intelligence platform looks like when it is designed for business consumption rather than academic evaluation.
Architecture: Built for Production
The system's four-layer architecture reflects production ML engineering discipline:
The data layer ingests the IBM Telco CSV, runs the feature engineering pipeline, and stores artifacts in Parquet format. The model layer runs XGBoost tuned with Optuna across 50 Bayesian optimization trials, applies isotonic calibration, computes SHAP values, and exposes predictions through a clean model interface. The API layer serves FastAPI endpoints single prediction, batch prediction, model info, and a feedback endpoint for ground truth collection. The dashboard layer connects the Streamlit four-tab UI single prediction with risk gauge and SHAP waterfall, batch CSV upload with segment distribution, analytics with intervention ROI, model performance with ROC/PR curves directly to the FastAPI backend.
The entire stack deploys via Docker Compose with a GitHub Actions CI/CD pipeline that gates on linting and unit tests before building.
The Engineering Decision That Mattered Most
Storing the model as four separate artifacts base XGBoost estimator, isotonic calibrator, fitted scaler, decision threshold rather than one monolithic pickle was the decision that made the system production-grade rather than production-adjacent.
Each component can be independently updated. The threshold can be recalibrated as the cost structure changes without retraining. The scaler can be refit to new data without touching the model. The calibrator can be swapped when recalibration is needed. This is not over-engineering for a portfolio project it is the minimum viable architecture for a system that will need to evolve after deployment.
The alternative one large pickle works until someone needs to update the threshold, at which point they discover they need to retrain everything.
Full source code, API documentation, and dashboard are available on GitHub.