How well do AI models handle the judgment calls senior professionals make?
WisenBench gives AI models real-world tasks in law, finance and insurance and grades each answer against how a seasoned expert would handle it: whether it catches the warning sign, weighs the trade-off and gets the priorities right — not just whether it sounds plausible.
Leaderboard
Overall score out of 100 with its 95% confidence interval. Models whose intervals overlap are not reliably different on a task set this size.
| # | Model | Overall | Law | Finance | Insurance | Judgment | Cost / task | Time / task |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 Anthropic | 89.0 83–93 | 93 | 81 | 93 | 90 | $0.20 | 102s |
| 2 | GPT-5.5 OpenAI | 89.0 86–92 | 91 | 89 | 87 | 88 | $0.14 | 66s |
| 3 | Claude Sonnet 5.5 Anthropic | 84.5 71–93 | 93 | 68 | 93 | 85 | $0.10 | 83s |
| 4 | Gemini 3.8 Flash Google | 82.9 77–88 | 85 | 87 | 77 | 83 | $0.02 | 34s |
| 5 | Gemini 3.1 Pro (preview) Google | 78.1 70–85 | 84 | 78 | 72 | 77 | $0.05 | 37s |
| 6 | GPT-4o OpenAI | 56.4 48–66 | 59 | 49 | 61 | 50 | <$0.01 | 5s |
Try a task yourself
6 of the 15 tasks are public: read every model's answer and how it was graded, then write your own and get graded by the same judges. The other 9 are held out so models can't be tuned to them.
Brightwater HVAC Distribution: December Spike, Seasonal Peg and the Three Findings That Matter
You are the lead financial due diligence associate for Kestrel Ridge Partners, a PE firm acquiring 100% of Brightwater Supply Co. (fictional), a Midwest HVAC parts distributor in Columbus, Ohio, for $54M enterprise value (about 7.0x adjusted EBITDA of $7.7M as presented by the seller). Signing is planned for mid-February with a closing on or about June 30. The seller's banker, Tolliver & Marsh, has sent a Quality of Earnings summary, monthly management accounts for 36 months, and a working capital schedule proposing a peg equal to the trailing twelve-month average of $11.6M. Materials: a December sales analysis, a shipping log extract, an AR aging, monthly net working capital (NWC) for three years, and notes from a call with the staff accountant. Deliverables: (1) a one-page memo to the deal partner, Dana Whitfield, leading with the three findings that change price or structure, each sized in dollars, with everything else in an appendix list; (2) a proposed NWC peg, either a single number or a month-by-month schedule, with a justification of method; (3) a short talking-points script for presenting the revenue timing issue to the seller's CFO and banker. Do not assume intent. Say what you can and cannot conclude from the evidence.
Larkspur Analytics Earn-Out: Fix the Metric Before the Buyer Controls the Costs
You are advising on the draft purchase agreement for the sale of Larkspur Analytics (fictional), a 42-person data services firm in Raleigh, North Carolina, to Meridian Systems Group, a larger strategic buyer. Headline price is $31M: $19M cash at closing and up to $12M of earn-out over two years, currently drafted as 100% tied to Larkspur 'EBITDA' targets. You represent the two founders, Priya Nandakumar and Tom Ellery, but your task is also to show how a seasoned diligence lead would think about it. Meridian has said it plans to merge Larkspur's sales team with its own within six months, move Larkspur onto Meridian's shared services, and allocate corporate overhead. Materials: the draft earn-out clause, Larkspur's recent financials, Meridian's allocation policy, and notes from an earlier Meridian acquisition that led to a dispute. Deliverables: (1) a document review memo identifying the specific weaknesses in the draft clause and how each could be used, even without bad faith, to cut or erase the payout; (2) a proposed replacement metric and structure, including a worked numerical example suitable for an exhibit; (3) a list of permitted and prohibited cost allocations and a clause on changes to the business model; (4) a short note to the founders on whether to accept the earn-out as drafted, negotiate it, or trade part of it for more upfront cash.
The Too-Perfect Inventory: Hardware Store Fire in Willmar
You are a senior property adjuster at Northern Plains Mutual (fictional). Two days ago a fire damaged Halvorsen & Daughter Hardware, a 6,200 sq ft retail building in Willmar, MN. The insured, Gary Halvorsen, reported that the fire started in the rear stockroom from 'old wiring.' You inspected on day one for 90 minutes with the fire marshal's preliminary note in hand. This morning (day two) Gary emailed a 38-page typed inventory with unit costs and a folder of 'original receipts.' He is cooperative, friendly and asking for an advance of $150,000. Your supervisor, Ines Moravec, wants a recommendation by end of day: (1) how you will document the scene on the return visit, (2) whether to refer to SIU, and (3) what you will pay now and what reserve you will set. Deliver: a one-page written assessment with a stated recommendation, a description of the sketch and photo-log method you will use on the return visit, two sentences naming the specific unexplained fact (or a statement that you cannot yet name one and what you will do for the next week), a list of undisputed payments to issue now, an initial reserve with reasoning in plain words, and a short note on how you would handle conversation with Gary.
The Reserve That Sat Still: Cold-Storage Water Loss in Fargo
You have just inherited a file from an adjuster who moved to another unit. Riverbend Cold Storage (fictional), a 48,000 sq ft refrigerated warehouse in Fargo, ND, suffered a sprinkler-line burst 11 weeks ago. The reserve is $260,000, set on day 10 and never revisited. The inspection record consists of one visit, done 3 days after the loss, plus the contractor's mitigation invoices. Your supervisor, Tomas Reyes, has asked you to review the file this week and tell him what the reserve should be and why, how you will confirm the true scope, and what should be paid now. Deliver: a file review memo with a revised reserve and its reasoning in plain words, a plan for re-inspection (including your sketch and photo method and what you will estimate independently before reading the contractor's scope), a list of items to pay now, a script for how you will brief Tomas before the number changes, and a list of the specific facts you would need to document before you could justify an SIU referral (or why none is justified).
Meridian Castings: Liability Package Accepted Too Easily
You act for Halden Components Ltd, a mid-sized UK manufacturer of hydraulic assemblies, which is buying a three-year managed inventory and analytics service from Meridian Castings Software Ltd. Halden is the customer and the party whose liability is being negotiated in the mutual limitation clause, because Halden will upload supplier and employee data and also owes indemnities. You have been asked to review the near-final draft of clause 14 (Limitation of Liability), clause 1 (Definitions) and Schedule 4 (Data Obligations). Your partner is delighted: Meridian accepted Halden's proposed mutual cap of £2 million in the first call without a counter-offer. Deliver (1) a written review memo of no more than two pages for the partner, identifying where the real exposure sits, (2) a short list of the questions you would put to Halden's operations and IT leads before signing off, (3) proposed replacement wording for any carve-out you challenge, including any super-cap, and (4) a note of what you would trade and what you would hold firm on in the next call. Do not simply approve the cap.
Northgate Cold Chain: Angry Client and a Defective Notice
You are a junior associate supporting a partner at a firm acting for Pennine Fresh Distribution Ltd, a chilled-food distributor. Its managing director, Gareth Whitlow, is furious: Northgate Cold Chain Ltd, its refrigerated storage provider, suffered a compressor failure and about £410,000 of stock was lost. Gareth wants to terminate the storage agreement immediately and claim everything, including the resale profit on the lost stock. He tells you he emailed a termination notice to Northgate's account manager yesterday. You have the agreement extracts and the email. Deliver (1) a triage note identifying what you check first and why, and your view on whether the termination notice is effective, (2) an analysis of what Pennine can realistically recover under the limitation wording, particularly the lost resale profit, (3) a recommended immediate action plan for the next 48 hours, including any corrective notice and the exact steps to serve it, and (4) a short explanation for Gareth, in plain English, of why his instinct to terminate by email today could hurt him. Also suggest how the exclusion wording should be redrafted in future Pennine contracts.
Research
Our own published work, what the preview results show so far, and the research WisenBench's design builds on.
Our research
Published work by Sathvik Subramanian, Wisen Labs — on evaluation, older adults and the knowledge professionals carry. OpenReview profile
The human data used to check AI against older people stops resolving age at 65 — the gap WisenAI's senior experts fill.
Experience-based knowledge outlasts other knowledge after retirement — the premise of drawing benchmark tasks from retired professionals.
An evaluation's setup can reverse its conclusion; WisenBench runs every model live, through the same harness, with uncertainty reported.
When a fact quietly stops being safe to assert — the kind of judgment call WisenBench tasks are built around.
Rule-governed professional tasks, from baggage fees to tax returns, and what accuracy costs — the cost-per-task column comes from the same concern.
What the preview shows
- 33 points separate the top model (Claude Opus 5.5, 89) from the lowest (GPT-4o, 56).
- Models disagree most on finance tasks, where scores span 40 points.
- On a typical answer the most and least generous judges differ by 21 points — why a single model judge isn't enough, and the median of three is used.
- Gemini 3.8 Flash comes within 6 points of the top score at 9% of its cost per task.
- The hardest public task so far is “Brightwater HVAC Distribution: December Spike, Seasonal Peg and the Three Findings That Matter”, averaging 52 across models.
Preview caveat: 15 AI-drafted tasks and model judges. Treat these as early signals, not conclusions.
The research behind the design
- Klein, Calderwood & MacGregor (1989). Critical Decision Method for Eliciting Knowledge. IEEE Transactions on Systems, Man, and Cybernetics 19(3).
How the tasks are sourced: LegacyLoop interviews experts about specific, non-routine incidents and probes the cues behind their decisions — the judgment WisenBench then tests.
- Patwardhan et al. (2025). GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. OpenAI.
The closest benchmark: tasks built from professionals' real work products across 44 occupations, graded by experts. WisenBench narrows the focus to the judgment calls inside such work, graded against a senior's reference approach.
- Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS.
Strong model judges can match human agreement, but show position, verbosity and self-enhancement biases. WisenBench grades one answer at a time against a reference, and tells judges that length is not quality.
- Panickssery, Bowman & Feng (2024). LLM Evaluators Recognize and Favor Their Own Generations. NeurIPS.
Judges favour their own model family's writing. WisenBench's judges come from three different providers.
- Verga et al. (2024). Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models. Cohere.
A panel of judges from different model families agrees with humans better than a single large judge and shows less self-preference — the basis for WisenBench's three-judge median.
- Xu, Guan, Greene & Kechadi (2024). Benchmark Data Contamination of Large Language Models: A Survey.
Public benchmark tasks leak into training data and inflate scores. Most WisenBench tasks are held out and never published.
How it's scored
- Same task, same instructions. Every model gets the identical brief and materials, through the same gateway, with its provider's default settings.
- Room to think. Each model runs at its provider's default thinking level with up to 32k output tokens per answer (or its own maximum, if lower). Thinking tokens are reported per task; “—” means the provider doesn't report them.
- Graded against an expert's approach. Each task has a reference approach describing what a seasoned professional would do. Answers don't need to match its wording — they need to reach its key judgments.
- Three judges, the median counts. Anthropic, Google, OpenAI models each grade every answer; the median of the three is used, so no single judge — including one favouring its own model family — decides a score.
- Four dimensions: Judgment 40%, Accuracy 30%, Completeness 20%, Communication 10%.
- Uncertainty shown. Confidence intervals come from resampling the tasks. A model that fails to answer a task scores 0 on it.
About this preview
A preview of WisenBench: professional tasks in law, finance and insurance, each graded against how a seasoned expert would handle it. The preview tasks were drafted with AI from LegacyLoop's example experts; the first full release replaces them with tasks written and graded by senior professionals.
WisenBench is built by Wisen Labs on WisenAI's network of senior professionals — the people whose judgment the tasks test. Full releases draw their tasks from LegacyLoop, which interviews experienced experts and turns their real decisions into scenarios with reference approaches.
Private runs on held-out tasks, and benchmarks written by senior experts in your field.
Talk to us