Auditing Inference Latency Budget

Audits the inference-latency budget for a deployed ML model against a stated SLO — measures P50, P95, and P99 latency end-to-end and per-stage (tokenization or feature lookup, preprocessing, model forward, postprocessing, serialization), attributes the tail to the dominant per-stage contributor, and recommends ranked optimizations by latency-saved-per-engineering-hour. Triggers whenever a real-time inference service is breaching or about to breach its P95 / P99 SLO, whenever latency tail behavior is unexplained, whenever an input-shape distribution change is suspected of inflating the tail, or whenever pre-launch SLO sizing must be derived from a measured profile. Refuses…