Chapter 3. About the model performance view

The Model performance view displays only validated models in the model catalog, and enables you to filter models based on advanced workload and hardware constraints. The recommended hardware configurations, throughput (output tokens), and latency metrics are derived from Pareto-optimal filtering, which provides non-dominated options across all performance dimensions.

When you select a validated model card in the catalog, the active performance filter constraints are transferred to the Performance Insights tab on the model details page. You can immediately compare compression variants and hardware options against the specified constraints in the catalog. This streamlines the process of moving from initial discovery to selecting the optimal model and hardware combination for a specific production workload.

Modifying the performance filter settings changes all the displayed model cards and their content to align with the applied constraints. Disabling and enabling the Model performance view resets the performance filter to the default settings.

When you enable the Model performance view view, models are sorted by lowest to highest latency and the model cards show validated models only and performance data with labels instead of description and labels. In addition, the time to first token, inter-token, and end-to-end latency metrics change according to the filters that you select in the workload and performance constraints.