Delivery Delay Risk & Value Optimization

Delivery delays at Olist are rare but disproportionately costly, making blanket operational intervention inefficient. This project built a decision system to predict delivery delay risk at order creation and determine whether proactive intervention creates net business value.

Why it mattered: Using 100K+ real marketplace orders, delay risk was shown to be highly concentrated across distance, cross-state logistics, and a small subset of sellers. A ranking-based machine learning model doubled Precision@10% versus heuristic rules, enabling intervention on only the top 10% highest-risk orders. Crucially, model performance was reframed in economic terms, identifying break-even intervention cost thresholds and supporting a defensible operational rollout via an executive dashboard.

Can delivery delays be predicted early enough to enable targeted intervention — and does acting on those predictions create positive economic value?

• Delivery delays are a rare event (high class imbalance)
• Predictions must rely only on information available at order creation
• Operational teams have limited capacity to intervene

Hypothesis: Delivery delay risk is driven by observable structural factors such as distance, cross-state logistics, and seller behavior, and can be concentrated into a small subset of orders.

1. Constructed a canonical order-level fact table with strict grain control
2. Diagnosed structural delay drivers using geography and seller behavior
3. Benchmarked simple rule-based heuristics as operational baselines
4. Trained ranking-based ML models optimized for Precision@K
5. Translated model output into a cost-based decision framework
6. Operationalized insights via an executive dashboard and playbook

Methods: Order-level ETL and data validation, Geographic and logistics enrichment, Seller concentration (Pareto) analysis, Rule-based baselines vs ML ranking models, Cost-sensitive threshold evaluation

Metrics: Precision@10%, ROC-AUC

Data: 100K+ Brazilian e-commerce orders (2016–2018), enriched with seller, customer, product, review, and geolocation data.

• Delivery delay risk is structural, increasing sharply with shipment distance.
• Cross-state logistics introduce meaningful administrative and operational friction.
• Delay risk is highly concentrated: ~13% of sellers account for ~80% of delays.
• Rule-based heuristics capture some risk but are operationally inefficient.
• A ranking-based model materially improves intervention precision at the margin.

• Enabled selective intervention instead of blanket operational action
• Improved efficiency by focusing only on the highest-risk orders
• Aligned analytics output directly with economic decision-making

• Precision@10% improved to ~5.4% (≈2× uplift vs heuristic baselines)
• Identified clear break-even intervention cost thresholds

• Optimized for ranking precision rather than binary accuracy
• Selected a Top-10% operating threshold to match capacity
• Prioritized interpretable, business-grounded features

• External logistics variability introduces unavoidable noise
• Economic outcomes are sensitive to intervention cost discipline
• Causal impact requires controlled experimentation

Delivered a deployable, economically defensible delivery-delay decision system supported by a six-page executive dashboard and a clear operational playbook.

• Run randomized A/B tests to measure causal intervention impact
• Integrate predictions into routing and seller SLA workflows
• Monitor data drift, seller mix shifts, and economic ROI over time