Glossary term

Cost per inference

Total inference cost divided by the number of model requests or completed inference operations.

A coarser unit than cost per token — useful when comparing across models or providers with different tokenization schemes, since it prices the request rather than an internal implementation detail of how the model counts its own input and output.