Oh My Pi

Oh My Pi (omp) is a coding agent for the terminal. Besides its chat model, it has a judge model role: a model that answers small typed questions instead of writing text. Set that role to Drex, Nace's decision models, and these questions go to Drex.

Drex support in Oh My Pi was contributed by the community.

What you getCopy Anchor Link

• The judge() and judgeBatch() helpers in omp's eval tool (judge_batch() in Python) are answered by Drex. They return calibrated probabilities, as described in Reading answers.
• With Drex Max, a judgment can read screenshots and other images.
• omp's own small decisions use the same role. See What in omp uses the judge.

Your chat model stays the same. Drex only takes the judge role.

Install ompCopy Anchor Link

Drex support is in omp v18.20.0 and later. Install omp on macOS or Linux with:

curl -fsSL https://omp.sh/install | sh

To pin a release, pass its version:

curl -fsSL https://omp.sh/install | sh -s -- --binary --ref v18.20.0

Homebrew, Bun and Windows installs are listed in the omp README. Run omp --version to check which release you have; it prints omp/18.20.0 or later.

An older omp, before v18.20.0, has no Drex support. It doesn't fail when you set the judge role below. It answers with another judge instead, so check which model answers, as shown in Check that Drex answers.

Connect your keyCopy Anchor Link

Create a key in the console under API keys, as shown in Authentication. Then give it to omp in one of two ways:

• In omp, run /login nace and paste the key.
• Or set it in your shell before you start omp:

export NACE_API_KEY=nace_sk_...

Set the judge roleCopy Anchor Link

omp lists three Drex models in its nace provider:

Assign one to the judge role in ~/.omp/agent/config.yml:

modelRoles:
  judge: nace/drex-max-v1.1

Keep any other roles you already have, such as default, next to it. To check the setting, run:

omp config get modelRoles

{"default":"fireworks/glm-5.3","judge":"nace/drex-max-v1.1"}

Check that Drex answersCopy Anchor Link

When omp can't use Drex, because the omp build has no Drex support or NACE_API_KEY isn't set in omp's environment, a judgment doesn't fail. omp answers it with the next judge it can use, such as TypeSafe or your own chat model, and the answer looks plausible. In our test without the key, the build log question below was answered by the chat model.

List the Drex models omp can use:

omp models nace --kind judge

nace (3)
┌──────────────────┬─────────┬─────────┬──────────┬────────┐
│ model            │ context │ max-out │ thinking │ images │
├──────────────────┼─────────┼─────────┼──────────┼────────┤
│ drex-max-finance │    262K │       - │ -        │ yes    │
│ drex-max-v1.1    │    262K │       - │ -        │ yes    │
│ drex-v2.0        │    262K │       - │ -        │ no     │
└──────────────────┴─────────┴─────────┴──────────┴────────┘

Without a key it prints No models available. After a judgment, the session file under ~/.omp/agent/sessions/ has a model_usage entry with "provider": "nace" for each Drex request; see Cost.

Ask a question about textCopy Anchor Link

omp's eval tool runs JavaScript or Python cells, and judge(state, questions) sends one state and its questions to the judge. omp's question types are choice, score and bool; bool is what Drex calls noul. Save this as build-log.js:

const r = await judge(
  "Build log: 412 tests passed, 3 failed (checkout.spec.ts: timeout after 30s).",
  { broken: { type: "bool", instructions: "Did the build fail?" } }
);
console.log(JSON.stringify(r));

Ask omp to run it. %load runs a file as one cell, and -p runs omp once without its interface. Name the language, since eval also runs Python. < /dev/null matters in scripts and CI: without a terminal, omp otherwise reads standard input as part of the prompt and waits for it to end:

omp -p "Run %load build-log.js in JavaScript eval. Reply with its output only, as plain text." < /dev/null

{"broken":{"type":"bool","bool":0.9969429682459251}}

bool is the probability that the answer is yes. In a normal session you don't need a file: ask omp in plain words, for example "use judge() to check which of these logs show a failed build", and it writes the cell itself. Python cells take the same call, with await judge(state, questions), and judge_batch() in place of judgeBatch().

Ask a question about a screenshotCopy Anchor Link

Drex Max reads images. Pass file paths, or image objects, in images; the questions call them "Image 1", "Image 2" and so on. Save this as checkout.js, next to two screenshots of a checkout page, error.png (a "500 Internal Server Error" page) and ok.png (an "Order confirmed" page):

for (const shot of ["./error.png", "./ok.png"]) {
  const r = await judge(
    "A screenshot of the checkout page after a test run.",
    {
      page: { type: "choice", instructions: "What does Image 1 show?",
              criteria: { error: "An error page", success: "A successful order", other: null } },
      broken: { type: "bool", instructions: "Is the checkout broken in Image 1?" }
    },
    { images: [shot] }
  );
  console.log(shot, JSON.stringify(r));
}

omp -p "Run %load checkout.js in JavaScript eval. Reply with its output only, as plain text." < /dev/null

./error.png {"page":{"type":"choice","choice":"error","confidence":0.9945615818598971,"probabilities":{"error":0.996374387906598,"success":0.0021766292912109063,"other":0.001448982802191049}},"broken":{"type":"bool","bool":0.9971428448520602}}
./ok.png {"page":{"type":"choice","choice":"success","confidence":0.9895471047073818,"probabilities":{"error":0.0048951675267121545,"success":0.9930314031382546,"other":0.0020734293350333373}},"broken":{"type":"bool","bool":0.004796215933822115}}

Both questions were sent in one request per screenshot, so each image was read and billed once. Each of these requests used 486 input tokens. To judge many states with the same questions, use judgeBatch() (judge_batch() in Python), which runs them in parallel.

What in omp uses the judgeCopy Anchor Link

The judge role answers these. Several are on by default once the judge role is a native decision model such as Drex, so expect Drex requests from them as well as from your own judge() calls:

• judge() and judgeBatch() (judge_batch() in Python) in the eval tool. The jevify keyword in a prompt tells omp to use them for long lists, such as a commit audit or log triage.
• The find tool, omp's search by description, which is on with find.enabled: auto, the default.
• Rules with a question, which omp asks about finished replies, reasoning and tool calls, with ttsr.judge: auto, the default.
• Charts under numeric tables in answers, on terminals that show graphics, with tui.autoGraph: auto, the default.
• The auto thinking level. With omp --thinking auto or defaultThinkingLevel: auto in config.yml, Drex rates how hard each prompt is, and omp picks the thinking level for that turn.
• Detection of a turn that stopped before it was done, only with features.unexpectedStopDetection: smart. The default, mechanical, doesn't use the judge.
• AI staging in the git interface (omp git).

Set find.enabled, ttsr.judge or tui.autoGraph to off to keep those features from calling Drex.

Each judgment is a billed Drex request, unless omp answers it from its local cache (see Limits).

LimitsCopy Anchor Link

• Images need a Drex Max model, with at most 8 images per judgment. omp skips nace/drex-v2.0 for any judgment with images, and skips every Drex model for one with more than 8. Such a judgment goes to the next judge that reads images, such as a vision chat model, without an error, so keep image judgments to 8 images on a Drex Max model.
• Images can be PNG, JPEG or WebP. omp scales down an image that is over 40 million pixels, or over its share of about 3 MB of image data per request, before sending it. Drex's own image limits apply as well.
• Drex refuses text that holds a base64 data URL of an image, audio, video or PDF. omp replaces each data URL in the state, instructions and criteria with a short description, such as [image/png base64 data URL]. It doesn't rewrite media content parts, such as an OpenAI image_url object, which Drex still refuses with a 422. Raw base64 media without the data: prefix is refused only when the string is at least 512 characters long and starts like a known media file; shorter base64 passes as text and is billed as tokens. Pass images in images instead; see Send text, not media.
• omp keeps a local cache of answers. The same question about the same state and images is answered from the cache, without a new Drex request.

CostCopy Anchor Link

Drex bills input tokens only; output tokens are free:

The examples above used 106 input tokens for the build log and 486 for each screenshot. omp records each Drex request in the session file under ~/.omp/agent/sessions/, as a model_usage entry with "provider": "nace", its token counts and a cost estimate from omp's price list. omp stats --summary counts the requests per model. Your Drex usage and billing in the console is the actual charge.