How pricing works
A calibrated model writes each contract
For every request, a time-to-cancel model estimates how likely the booking is to be cancelled in each window before check-in. Each cancellation contract is then priced so the hotel expects the same revenue from this guest as under its usual policy. Here is how that works, how well the model does on bookings it never saw, and where it falls short.
From a request to a price
- Hazard per window. A gradient-boosted model scores six windows before check-in (30+ days, 14–30, 7–14, 3–7, 1–3, last day) from what is known at quote time: lead time, dates, nights, party size, meal plan, hotel type and — if a wallet is connected — how many stays it completed before.
- Chance of cancelling in each window (q). The hazards combine into q for each window and P(cancel) for the booking.
- Risk cost of a contract. For each window: q × (refund the contract pays − the chance the hotel resells the freed night), never below zero. A cancellation that refunds nothing earns the hotel no credit.
- Price. The hotel’s usual contract costs its base rate. Any other contract costs base × (1 − usual risk) ÷ (1 − its risk), within the hotel’s bounds, rounded up to a cent. More refund costs more; less refund costs less.
Every quote carries this breakdown: open “Why this price” next to any option at checkout. The prices and refund schedule are signed and written into the booking’s escrow on Solana; the probabilities are not.
How good is it?
Model hazard-gbdt-2026-09-25: 44 trees trained on 41,062 bookings, evaluated on 36,294 held-out bookings arriving January–August 2017 (Hotel Booking Demand dataset). 30.9% of them cancelled; the model predicted 29.9% on average.
0.713
Ranking (AUC)
vs 0.634 ranking by lead time alone. Modest, but clearly better than the strongest rule of thumb.
0.1917
Brier score
vs 0.2136 for always predicting the true cancel rate (lower is better).
0.038
Calibration error
Average gap between predicted and observed rates across ten groups (0.041 before calibration).
Calibration: do the probabilities mean what they say?
Held-out bookings sorted by predicted P(cancel) into ten equal groups. A well-calibrated model sits on the dashed line: of bookings given a 30% chance, about 30% cancel.
- Platt-calibrated (used for prices)
- Uncalibrated model
- Perfect calibration
Show the numbers
| Group (~3,630 bookings) | Uncalibrated: predicted / observed | Platt: predicted / observed |
|---|---|---|
| 1 | 5.3% / 6.0% | 5.1% / 6.0% |
| 2 | 12.4% / 10.8% | 12.0% / 10.8% |
| 3 | 17.7% / 17.4% | 17.2% / 17.4% |
| 4 | 21.9% / 19.0% | 21.5% / 18.9% |
| 5 | 28.9% / 27.8% | 28.9% / 27.9% |
| 6 | 35.2% / 38.9% | 35.3% / 38.6% |
| 7 | 38.0% / 41.6% | 38.3% / 41.6% |
| 8 | 39.3% / 47.2% | 39.7% / 47.2% |
| 9 | 43.0% / 52.9% | 43.6% / 52.6% |
| 10 | 56.6% / 47.5% | 57.7% / 48.2% |
Why Platt scaling and not isotonic regression
Isotonic regression calibrates slightly better on paper (error 0.029 vs 0.038) but it is a step function: nearby requests would share identical prices and prices would jump at step edges. Platt scaling is smooth and strictly increasing, so a slightly riskier request always costs slightly more. We accept the small, disclosed calibration cost (isotonic also ranks worse here: AUC 0.709).
Per-window accuracy
Prices use the per-window chances, so this is where drift matters. Mean predicted vs observed chance of cancelling in each window, on the same held-out bookings:
| Before check-in | Predicted | Observed | Predicted ÷ observed |
|---|---|---|---|
| 30+ days | 15.40% | 19.76% | 0.78× |
| 14–30 days | 5.02% | 5.29% | 0.95× |
| 7–14 days | 3.70% | 3.17% | 1.17× |
| 3–7 days | 3.25% | 1.25% | 2.61× |
| 1–3 days | 1.40% | 0.78% | 1.81× |
| last day | 1.15% | 0.69% | 1.67× |
Honest limitations
- It is a prior from elsewhere. The model learned from two Portuguese hotels in 2015–2017. Tbilisi guesthouses in 2026 differ in guests, seasons and channels. Every hotel shares this prior during the pilot and tunes it only through its own settings; recalibrating on each hotel’s own outcomes (recorded on-chain) is on the roadmap.
- Timing drift. Cancellations moved between windows from 2016 to 2017: the 3–7 days window is over-predicted 2.61× on 2017 arrivals, while overall P(cancel) stays close because errors partly offset. Per-hotel recalibration corrects exactly this.
- Request-time information only. No price, nationality or deposit features, and wallet history can only lower a guest’s estimated risk, never raise it.
- AI agents are priced as direct bookers. An agent booking for its own user is treated like a direct booking rather than an online travel agency.
- Prices target, not guarantee, revenue. Each option is priced to target the same expected revenue as the hotel’s usual policy; actual revenue depends on who really cancels.
Full report and reproduction steps: ml/reports/metrics.md in the Stumari repository. Back to Stumari