DSPy

DSPy optimizes programs against a metric, and an optimizer calls that metric many times. Drex can be the judge two ways: through DSPy's own TypeSafe language model, which works today with only a base URL, a key and an explicit model, or through the optional dspy-nace package, which adds a ready-made metric and a classifier module. See Questions and Reading answers.

Use DSPy's TypeSafe modelCopy Anchor Link

DSPy 3.4.1 ships dspy.experimental.TypeSafe, a language model for decision models that sends TypeSafe's /v1/systemone request. Drex answers it. Create a key as shown in Authentication, set TYPESAFE_API_KEY to it and TYPESAFE_BASE_URL to https://console.nace.ai (the host, without /v1; for Drex on your own infrastructure, that deployment's URL).

import os
import dspy
from dspy.experimental import Choice, Noul

lm = dspy.experimental.TypeSafe(
    "drex-max",
    api_key=os.environ["TYPESAFE_API_KEY"],
    base_url=os.environ["TYPESAFE_BASE_URL"],
    timeout=60,
)

class Triage(dspy.Signature):
    """Judge a customer message."""

    text: str = dspy.InputField()
    refund: Noul = dspy.OutputField(desc="Is the customer asking for a refund?")
    team: Choice[("billing", "payments"), ("technical", "bugs"), ("other", None)] = dspy.OutputField(
        desc="Who handles this?"
    )

result = dspy.Predict(Triage)(text="I was charged twice for March.", lm=lm)
result.refund.probability          # probability of yes
result.team.value                  # "billing"
result.team.probabilities          # a probability per label

Each output field becomes one question. The model id is required: name a Drex model such as drex-max or drex-v2.0.

What to know:

• Pass dspy.experimental.TypeSafe(...), not dspy.LM("typesafe:drex-max", ...). The dspy.LM form didn't work against Drex in the same test, so use the class above.
• timeout is in seconds. TypeSafe's default of 10 seconds is shorter than Drex can take on a large request; see Migrate from TypeSafe.
• box questions aren't supported by Drex, which answers 422.
• A Score output failed when it was tried: Drex returned the answer's legend as a list of level labels, and DSPy's TypeSafe client validates it as an object keyed by level, so it raised a validation error even though the request succeeded. Drex now sends that client an object keyed by level, recognized by its X-TypeSafe-SDK header, and X-Nace-Legend: array or object overrides it; see Reading answers. That is available once the change is deployed. An object legend validates against the real typesafe-sdk ScoreAnswer model offline, but a live Score call through DSPy has not been run yet, so test one before relying on it. Until then, Noul or Choice (a graded scale can be a Choice over its levels), the Python Nace SDK, dspy-nace below or plain HTTP are the checked routes.
• dspy.experimental can change between DSPy releases.

The dspy-nace packageCopy Anchor Link

dspy-nace adds a DrexMetric, which judges a program's outputs with Drex, and a DrexClassify module, a dspy.Predict-like module that returns Drex's answers with their probabilities. It uses the Nace Python SDK, so it doesn't depend on how DSPy's TypeSafe client reads a Score output.

SetupCopy Anchor Link

pip install dspy-nace
export NACE_API_KEY=nace_sk_...

Create a key as shown in Authentication. NACE_BASE_URL (or base_url) overrides the base URL, which is https://console.nace.ai by default. If Drex runs on your own infrastructure, set it to that deployment's URL: any host that isn't OpenRouter is treated as Drex and uses NACE_API_KEY.

Judge with DrexCopy Anchor Link

A DrexMetric takes a list of criteria. One request judges every criterion of one output, which uses far fewer input tokens than separate requests.

import dspy
from dspy_nace import Criterion, DrexMetric

metric = DrexMetric(
    [
        Criterion.noul("grounded", "Does the output state the reference fact?", threshold=0.7),
        Criterion.noul("filler", "Is the output padded with filler?", want=False),  # a defect
    ]
)

dspy.Evaluate(devset=devset, metric=metric, max_errors=1)(program)
dspy.GEPA(metric=metric, reflection_lm=dspy.LM("..."), auto="light").compile(program, trainset=train)
dspy.BootstrapFewShot(metric=metric, max_errors=1).compile(program, trainset=train)
metric.raise_if_errors()  # after a run: a failed request is not a quality score

What the metric returns follows DSPy's conventions:

Feedback is facts, not explanationsCopy Anchor Link

Drex returns probabilities and no rationale, so the feedback GEPA reads lists the failed criteria with their pass score, threshold and probabilities, and names the ones that passed. A model writes none of it:

1 of 2 criteria failed (score 0.57):
- grounded: pass score 0.20 < threshold 0.70; P(yes)=0.20, wanted yes
Passed: filler.

Direction and thresholdsCopy Anchor Link

Each criterion's pass score runs from 0 to 1, and higher is better. A yes/no criterion wants yes (want=True) or no (want=False). A choice criterion passes on the total probability of its good labels. A score rubric is normalized to its level divided by the top level, and invert=True flips it when level 0 is best. The thresholds are yours: choose them from a labeled sample.

What Drex seesCopy Anchor Link

By default Drex reads the input, the output and the reference. Pass state=lambda gold, pred: ... to send only what the criteria need. metric.usage totals the judge requests and input tokens, counted once per request, and metric.last holds the latest verdict.

Classify with DrexCopy Anchor Link

DrexClassify asks typed questions and returns each answer next to its probabilities:

from nace_sdk import Choice, Noul
from dspy_nace import DrexClassify

triage = DrexClassify(
    {
        "refund": Noul(instructions="Is the customer asking for a refund?"),
        "team": Choice(instructions="Who handles this?", criteria={"billing": None, "tech": None}),
    }
)
result = triage(state="I was charged twice for March.")
result.refund                  # probability of yes
result.team, result.team_probabilities
result.drex["request_id"]

Models and imagesCopy Anchor Link

The default model is drex-max. Pass model or set NACE_DEFAULT_DECISION_MODEL to use another id; any id Drex serves works. Pin a model when you tune thresholds, because probabilities can shift between models. Drex enforces the per-request question limit; see Limits. Per million input tokens, Drex Max costs $0.03 and Drex 2.0 costs $0.06; output tokens aren't billed.

Drex Max reads images, which Drex calls "Image 1", "Image 2" and so on. Give the metric an images function, or pass dspy.Image, bytes, a path or an uploaded file to DrexClassify:

DrexMetric(
    [Criterion.noul("ok", "Does Image 1 show the confirmation page?")],
    images=lambda gold, pred: [pred.screenshot],
)

Images are refused before sending for a text-only model and never dropped. Drex never fetches image URLs, so download a dspy.Image that holds only a URL first. Drex enforces the image limits, so upload large images with nace-sdk. The shared request limits are:

ErrorsCopy Anchor Link

A bad key, no available funds, a rejected request or an outage raises DrexAPIError or DrexConnectionError. The metric never scores it as a failed criterion, but DSPy catches an exception from a metric and scores it as failure_score (0) until max_errors. Pass max_errors=1 to Evaluate and BootstrapFewShot so the first failed request stops the run, and call metric.raise_if_errors() after a run. GEPA's error limit isn't configurable: after a 401, 402 or 403 from Drex the metric raises without sending more requests, and a non-zero metric.errors means some scores aren't quality judgments. A 422 lists each issue's path and message, and a 402 links to the Billing page of the Drex host you configured (https://console.nace.ai only when you set none; there is no link on OpenRouter). The client retries 429, 529 and 5xx responses after retry-after-ms within a bounded total wait, so a longer wait raises the error. It waits at least 60 seconds for each request, and by default has a small number of requests in flight at once. See Errors and retries and Limits.

OpenRouterCopy Anchor Link

Pass base_url="https://openrouter.ai/api/v1" to run through OpenRouter. A bare model id, the default included, is sent as nace-ai/<id> (drex-max becomes nace-ai/drex-max), and an id that already has a prefix is sent as given. drex-latest is Drex's own alias, not an OpenRouter model, so it is refused: pin nace-ai/drex-max or nace-ai/drex-v2.0. Use your OpenRouter key in api_key or OPENROUTER_API_KEY. NACE_API_KEY is never sent there. The request is shaped for it: instructions are always a string, images are content parts of the state, and the referer and title headers are sent. OpenRouter takes inline images only, so an uploaded file is refused rather than dropped.

Every request sends User-Agent and X-Nace-Client headers that name dspy-nace and its version.