Skip to content

AI

The AI Grading Your Investments Agrees With Seniors 88.6% of the Time

Terminal X's Eval Agent runs GPT-6.1 Sol against a 54-criterion rubric and, per the company, catches 25% more fund-mandate deviations than senior investors. The red pen has changed hands.

Cover photo: Jernej Furman from Slovenia / CC BY 2.0

TL;DR

  1. Terminal X's Eval Agent grades investment outputs on a 54-criterion rubric using GPT-6.1 Sol.
  2. It matches senior investor judgment 88.6% of the time, per the announcement.
  3. It catches 25% more deviations from a fund's mandate than senior investors do.

What happened

The Eval Agent grades investment outputs against a 54-criterion rubric, with GPT-6.1 Sol doing the grading. Terminal X says the system matches senior investor judgment 88.6% of the time.

The second number is the spicy one. Terminal X says the agent catches 25% more deviations from a fund's mandate than senior investors do. A fund mandate is the rulebook for what a fund is supposed to do, so a deviation is the fund wandering off its own script.

Both figures come from the announcement, so treat them as vendor claims until someone outside the building reruns the test.

Why it matters

Most AI-in-finance headlines are about generating things: analysis, summaries, answers. This one is about checking things. A model that grades output at scale is a different job from a model that writes it, and arguably a more useful one.

The 88.6% figure deserves a careful read. It says the grader agrees with senior investors most of the time, which is a decent sign the rubric reflects how experts think. It does not say the grader is right when the two disagree.

That is where the 25% claim comes in. If the agent really flags more mandate deviations than seniors do, the gap is the product. Humans reading long outputs get tired and skim. A rubric does neither.

The scope is narrow. This is one company's tool, aimed at enterprise investment teams, and the public detail is a single announcement with three numbers. That is enough to take seriously and not enough to call a revolution.

Still, the pattern is worth noticing, sapiens. The expert's role is shifting from doing the checking to defining what gets checked.

What to do with it

  • Steal the rubric idea.Break "good output" into explicit criteria, as Terminal X did with 54, before you ask any model to judge anything.
  • Ask for the test.If a vendor quotes an agreement rate, ask what it was measured against and who the reviewers were.
  • Watch the disagreements.An 88.6% match means the interesting cases are the rest, so have a human review those.
  • Check your mandates.If you run money, see how easily your own rules could be written as a checklist a machine can read.

Ciao, sapiens. The rubric has 54 criteria. Your performance review has fewer, for now.