Evaluation instance details

View the status, outcome, evaluator pass rates, and test case results for an evaluation instance on the Instance details page. You can also open the associated agentflow, dataset, and evaluation, and review the input, actual output, and evaluator results for each test case.

Jump to: 

Access the Evaluation instance details page

  1. Go to Agents > Evaluations.

  2. Select the Evaluation instances tab.

  3. Click the name of the evaluation instance that you want to view.
    The Instance details page is displayed.

View the associated assets

The Asset details section provides links to the assets used by the evaluation instance.

  • Click the Agentflow name to open the agentflow in the Agentflow designer.

  • Click the Dataset name to open the dataset.

  • Click the Evaluation name to open the evaluation.

Review the evaluation instance results

The Instance details section shows when the evaluation instance started and finished, its current status, its overall outcome, and its unique instance ID.

The Evaluator pass rates section shows the percentage of test cases that passed for each evaluator included in the evaluation.

An evaluation instance can have a Completed status and an overall outcome of Fail. This occurs when the evaluation finishes successfully, but one or more evaluator pass rates do not meet the configured pass rate thresholds.

View test case results

The test case table shows the result of each evaluator for every test case included in the dataset. Each row represents a test case. The evaluator columns are based on the evaluators configured for the evaluation and show a result of Pass or Fail.

View test case details

  • Click a test case number in the test case table.
    The test case panel opens on the right side of the page and displays the following information:

    • The agentflow instance created for the test case.

    • A summary of the score and result for each evaluator.

    • The input values provided to the agentflow.

    • The actual output produced by the agentflow.

    • Detailed results for each evaluator.

  • Click the Agentflow instance name to open the associated agentflow instance details.

  • Under Evaluator results, expand an evaluator to view its result. The information displayed depends on the evaluator type. For example, an Exact match evaluator displays the expected output that was compared with the actual output.

  • Click Close to close the test case panel.

The following fields or columns are displayed when you click an instance name.

Section Field or column Description
  Evaluations Click to return to the Evaluations page.

 

Evaluation Instances

Click to return to the Evaluation instances page.
  (Evaluation instance name)

Name of the evaluation instance. The name includes the evaluation name and the date and time when the instance started.
  Refresh Click to refresh the evaluation instance details and test case results.

Instance details

Initiated

Date and time when the evaluation instance started.
  Finished Date and time when the evaluation instance finished. This field is not populated while the instance is still running.

 

Instance ID

Unique identifier for the evaluation instance.
  Status Current status of the evaluation instance. Possible values include Queued, Running, Completed, Failed, and Terminated.
  Overall outcome Overall evaluation result. Possible values are Pass and Fail. The outcome is determined by the configured evaluator pass rate thresholds.
  Evaluator pass rates Percentage of test cases that passed for each evaluator included in the evaluation.
Asset details Agentflow

Name of the agentflow used by the evaluation instance. Click the name to open the agentflow in the Agentflow designer.
  Dataset Name of the dataset used by the evaluation instance. Click the name to open the dataset.
  Evaluation Name of the evaluation associated with the instance. Click the name to open the evaluation.

Test case results

Test case Number of the test case from the dataset. Click the number to open the test case panel.

 

Evaluator columns

Shows whether the test case passed or failed for each evaluator configured in the evaluation. The column names are based on the evaluator names.
Test case panel Agentflow instance Agentflow instance created for the selected test case. Click the name to open its instance details.

Summary

Evaluator Name of the evaluator used to assess the test case. Click the evaluator name to expand its detailed result.

 

 Score (%) Score assigned to the test case by the evaluator.

 

 Result Evaluator result for the test case. Possible values are Pass and Fail.
Input Input variables Input values from the selected dataset row that were provided to the agentflow.

Actual output

Output variables Values produced by the agentflow for the selected test case.

Evaluator results

Evaluator name Expand to view detailed information about how the evaluator assessed the test case.

 

 Evaluator type Type of evaluator used, such as Exact match.

 

 Result Indicates whether the selected test case passed or failed the evaluation.
  Expected output Expected value used to assess the actual output. The fields displayed depend on the evaluator type.