Next steps

Next steps

Both tracks start this week. Dates are proposed, to agree together.

Track 1 · policybazaar.ai

WhenStepWhat happensOwner
By 18 Nov
1Data sync and KPIs*
Data systems, SOPs and past pb.ai chats shared; KPIs and today's baseline agreedPolicybazaar, with Dvar
By 25 Nov
2Post-train PB Flash and PB Pro
Router, PB Flash and PB Pro trained on pb.ai's chats; scored on your evals against today's Gemini stackDvar
By 5 Dec
3Pilot
1.5 weeks on a share of live pb.ai traffic, with a control group; quality, cost and latency tracked dailyBoth
15–30 Dec
4Iterate and full rollout
Fixes from the pilot, then all pb.ai traffic; evals and observability stay onBoth

* Dates assume Policybazaar's data and access are ready by 18 Nov. Data sync can take up to twice as long; each later date moves with it.

Track 2 · Shifting to open source models

Target: Start saving 50% on 1–2 high-volume workflows within 2–3 weeks

WhenStepWhat happensOwner
By 18 Nov
1Pick 1–2 high-volume workflows
Where tokens go today, by workflow: volume, model, cost and owner; the 1–2 with the biggest saving go firstPolicybazaar, with Dvar
By 24 Nov
2Evals and the open model
Evals on the owner's own cases; the cheapest open model that matches today's quality and speedDvar, with each owner
By 1 Dec
3Live and saving 50%
Traffic moves over with evals and observability on; spend tracked against today's bill every weekBoth
From Dec
4Next workflows, same playbook
Ranked by saving; each moves only when it passes its evalsDvar, with each owner