> ## Documentation Index
> Fetch the complete documentation index at: https://docs-platform.arklex.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Metric Alignment

> Use human annotations to automatically improve a metric's scoring prompt.

Metric Alignment uses your team's human annotations to automatically improve a metric's scoring prompt. When the LLM judge and your reviewers consistently disagree on a metric, run Alignment to distill those disagreements into a refined rubric — without writing the new prompt by hand.

<Note>
  Alignment builds on resolved annotations, so annotate and resolve at least one evaluation first. See [Annotations](/annotations) for that workflow, and [Metrics](/metrics) for how metrics are defined.
</Note>

Alignment works on both built-in and custom metrics, and accepting a result behaves differently for each:

* **Built-in metric** — creates a new custom metric (a fork) containing the distilled guidelines. The original built-in metric is unchanged.
* **Custom metric** — bumps the metric to a new version with the refined prompt.

***

## Start an alignment run

Open a metric's detail page, find the **Alignment** section at the bottom, and click **Start Alignment**. Configure the run:

<ParamField path="Source evaluations" type="array" required>
  One or more completed evaluations that have resolved annotations for this metric. Only evaluations with resolved annotation data appear in the list.
</ParamField>

<ParamField path="Reflection LM" type="string">
  The provider and model used to analyze disagreements and distill guidelines. Uses the platform default if left blank.
</ParamField>

<ParamField path="Judge model" type="string">
  The model used to verify the distilled guidelines. Optional.
</ParamField>

<ParamField path="Delta threshold" type="number">
  The minimum score gap between the judge and a human annotation for the pair to count as a disagreement.
</ParamField>

Click **Start** — the run enters the queue immediately and moves through four phases: loading annotations, building disagreement traces, distilling guidelines, and finalizing. Most runs finish within a few minutes, depending on the number of conversations and the reflection model.

| Status        | Meaning                                                  |
| ------------- | -------------------------------------------------------- |
| **Pending**   | Queued, not yet started.                                 |
| **Running**   | In progress.                                             |
| **Aligned**   | Completed successfully — guidelines are ready to review. |
| **Error**     | The run failed; view the detail and retry.               |
| **Cancelled** | Cancelled before completion.                             |

<Tip>
  Cancel any in-flight run with **Cancel**. Cancellation is idempotent — cancelling an already-completed run does nothing.
</Tip>

***

## Review and accept results

When a run reaches **Aligned**, expand it to see the distilled guidelines (in plain language), how many disagreements were found across the source evaluations, and how many agreeing rows were skipped.

If the guidelines look good, click **Accept**:

* On a **built-in metric**, a new custom metric is created with the distilled guidelines as its scoring prompt — the built-in stays untouched.
* On a **custom metric**, the metric updates to a new version, and all future evaluations using it apply the new rubric.

If a run found zero disagreements or the output is too generic, discard it and try again with different source evaluations or a different reflection model.

***

## Alignment history

The Alignment section lists all past runs for the metric, newest first, with each run's status, creation time, and source evaluations. Expand any completed run to view its guidelines — even after accepting.

***

## FAQ

<AccordionGroup>
  <Accordion title="How is Alignment different from editing the scoring prompt myself?">
    Editing means writing the new rubric by hand. Alignment derives it automatically by analyzing disagreements between the judge's scores and your team's resolved annotations — most useful when you see a pattern of disagreement but aren't sure how to express the correction in the prompt.
  </Accordion>

  <Accordion title="How is Alignment different from Annotation Calibration?">
    **Annotation Calibration** (on the evaluation detail page) *shows* you where the judge and your reviewers agree or disagree, per metric. **Metric Alignment** goes a step further and *acts* on those disagreements, rewriting the metric's rubric to close the gap. Use Calibration to spot the problem, Alignment to fix it.
  </Accordion>

  <Accordion title="Can I run Alignment on a built-in metric?">
    Yes. Accepting the result forks it into a new custom metric containing the distilled guidelines; the original built-in metric isn't modified.
  </Accordion>

  <Accordion title="How many resolved annotations do I need?">
    No hard minimum, but more resolved annotations give the reflection model more signal. Fewer than 5–10 resolved disagreements tends to produce generic, low-confidence guidelines — aim to annotate and resolve at least one full evaluation first.
  </Accordion>
</AccordionGroup>