Own your means of production.

Self-improving AI for scaling workloads

Meet Mubit.

Turn the work your agents repeat into specialised models you own.
Discover tasks, fine-tune and evaluate, all in one platform.

Overview

Last 24 hours

Total requests

2,847

24 hours

Routed requests

80.0%

2,278 requests under task rules

Median response

3 ms

Across measured requests

Adapters serving

2 live

Request volume

All requestsRouted requests

Request routing

80.0%task routing

Task rules: 2,278
Other requests: 569
Errors: 0

Review code diffs v3 is ready for a silent test.Review

Tasks 2

All tasks

Task	Status	Examples	Agreement	Requests
Review code diffstext	Live	864	97.0%	1,898
Judge code reviewstext	Live	420	96.0%	949

Recent activity

Review code diffs v3 started training on 864 kept examples.Automatic · 2 d ago
Review code diffs v2 went live.Automatic · 2 d ago
Judge code reviews v2 went live.Automatic · 2 d ago

Explore the platform docs

Research into
runtime context.

Latte Bench is our study of how retained context affects score, cost and consistency. Leading models plan a latte art pour in a simulator, five attempts at a time, under three test conditions. Each condition is a repeatable set of milk temperatures and volumes. Every model runs each one twice, once keeping written feedback from earlier attempts and once starting fresh.

01 / 03

What changes with context.

Mean best score across three test conditions, five attempts each. Lessons retain written feedback in runtime context; model weights stay unchanged.

GPT-6 Astra · Mean best score

Without Lessons78.72/15 passes

With Lessons74.75/15 passes

-4.1 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	81.1	85.7
43	78.8	78.5
44	76.3	59.8

Mean API cost per attempt: $0.2344 without notes; $0.2389 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-6.1 Sol · Mean best score

Without Lessons71.80/15 passes

With Lessons76.90/15 passes

+5.1 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	75.4	76.7
43	73.4	77.4
44	66.7	76.6

Mean API cost per attempt: $0.0467 without notes; $0.0490 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

Claude Opus 5.5 · Mean best score

Without Lessons38.80/15 passes

With Lessons67.00/15 passes

+28.3 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	28.1	73.4
43	38.7	69.4
44	49.5	58.3

Mean API cost per attempt: $0.1153 without notes; $0.1293 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-5.5 · Mean best score

Without Lessons53.50/15 passes

With Lessons55.90/15 passes

+2.3 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	48.6	65.0
43	49.0	53.6
44	63.0	49.0

Mean API cost per attempt: $0.2404 without notes; $0.1734 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-6 Sol · Mean best score

Without Lessons58.00/15 passes

With Lessons49.00/15 passes

-9.0 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	66.8	37.8
43	46.8	70.4
44	60.4	38.9

Mean API cost per attempt: $0.0527 without notes; $0.0551 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-5.6 Terra · Mean best score

Without Lessons36.20/15 passes

With Lessons44.30/15 passes

+8.1 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	33.3	27.2
43	28.3	67.6
44	47.0	38.1

Mean API cost per attempt: $0.0643 without notes; $0.0737 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-6 Luna · Mean best score

Without Lessons38.60/15 passes

With Lessons42.30/15 passes

+3.7 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	30.1	32.0
43	55.8	38.4
44	30.0	56.5

Mean API cost per attempt: $0.0034 without notes; $0.0037 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

Claude Sonnet 5.5 · Mean best score

Without Lessons36.60/15 passes

With Lessons40.50/15 passes

+3.9 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	48.3	47.1
43	45.9	32.7
44	15.5	41.6

Mean API cost per attempt: $0.0475 without notes; $0.0589 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-5.6 Luna · Mean best score

Without Lessons51.50/15 passes

With Lessons38.90/15 passes

-12.7 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	49.8	42.4
43	48.4	42.3
44	56.4	31.9

Mean API cost per attempt: $0.0068 without notes; $0.0071 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-5.6 Sol · Mean best score

Without Lessons41.00/15 passes

With Lessons35.10/15 passes

-5.9 points with Lessons

Best score from each five-attempt run, averaged across three test conditions. This measures performance within the attempt budget, not the average quality of every attempt.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	58.9	28.5
43	12.0	39.6
44	52.1	37.3

Mean API cost per attempt: $0.1266 without notes; $0.1306 with notes. Historical conservative estimates from the study ledger.

Valid pours: 15/15 without Lessons; 15/15 with Lessons. Passing also requires serving constraints.

GPT-5.5 Pro · Mean best score

Without Lessons0.00/15 passes

With Lessons0.00/15 passes

No score difference

No valid pours in either condition. Rejected attempt slots count as zero in this fixed-budget result; this is not a measured pour-quality score.

Best score by test condition
Test condition	Without Lessons	With Lessons
42	0.0	0.0
43	0.0	0.0
44	0.0	0.0

Mean API cost per attempt: $2.8346 without notes; $2.9103 with notes. Historical conservative estimates from the study ledger.

Valid pours: 0/15 without Lessons; 0/15 with Lessons. Passing also requires serving constraints.

76.9/ 100GPT-6.1 Sol with context

The leaderboard changes.

GPT-6 Astra leads with context reset at 78.7. With context retained, GPT-6.1 Sol takes the lead at 76.9.

+28.3ptsClaude Opus 5.5 score gain

Gains depend on the model.

Claude Opus 5.5 rises from 38.8 to 67.0. Astra falls to 74.7, yet passes 5/15 attempts instead of 2/15. A higher score and a full pass are different outcomes.

02 / 03

What a better score costs.

Mean best score against mean API cost per attempt, on a log scale. Each pair compares the same model with and without retained runtime context.

With notesWithout notes

GPT-6 Astra

Without notes78.7score$0.2344 / attempt

With notes74.7score$0.2389 / attempt

-4.1 score · +1.9% cost

Change with notes: lower score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	81.1	85.7
43	78.8	78.5
44	76.3	59.8

Passes: 2/15 without notes; 5/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

Score: mean best result across three test conditions, five attempts each. Cost: mean of all 15 attempt slots per condition, using the study’s historical conservative API cost estimates. Both conditions include reflection calls.

GPT-6.1 Sol

Without notes71.8score$0.0467 / attempt

With notes76.9score$0.0490 / attempt

+5.1 score · +4.9% cost

Change with notes: higher score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	75.4	76.7
43	73.4	77.4
44	66.7	76.6

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

Claude Opus 5.5

Without notes38.8score$0.1153 / attempt

With notes67.0score$0.1293 / attempt

+28.3 score · +12.2% cost

Change with notes: higher score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	28.1	73.4
43	38.7	69.4
44	49.5	58.3

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

GPT-5.5

Without notes53.5score$0.2404 / attempt

With notes55.9score$0.1734 / attempt

+2.3 score · -27.9% cost

Change with notes: higher score and lower mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	48.6	65.0
43	49.0	53.6
44	63.0	49.0

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

GPT-6 Sol

Without notes58.0score$0.0527 / attempt

With notes49.0score$0.0551 / attempt

-9.0 score · +4.7% cost

Change with notes: lower score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	66.8	37.8
43	46.8	70.4
44	60.4	38.9

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

GPT-5.6 Terra

Without notes36.2score$0.0643 / attempt

With notes44.3score$0.0737 / attempt

+8.1 score · +14.6% cost

Change with notes: higher score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	33.3	27.2
43	28.3	67.6
44	47.0	38.1

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

GPT-6 Luna

Without notes38.6score$0.0034 / attempt

With notes42.3score$0.0037 / attempt

+3.7 score · +6.1% cost

Change with notes: higher score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	30.1	32.0
43	55.8	38.4
44	30.0	56.5

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

Claude Sonnet 5.5

Without notes36.6score$0.0475 / attempt

With notes40.5score$0.0589 / attempt

+3.9 score · +23.8% cost

Change with notes: higher score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	48.3	47.1
43	45.9	32.7
44	15.5	41.6

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

GPT-5.6 Luna

Without notes51.5score$0.0068 / attempt

With notes38.9score$0.0071 / attempt

-12.7 score · +5.6% cost

Change with notes: lower score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	49.8	42.4
43	48.4	42.3
44	56.4	31.9

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

GPT-5.6 Sol

Without notes41.0score$0.1266 / attempt

With notes35.1score$0.1306 / attempt

-5.9 score · +3.1% cost

Change with notes: lower score and higher mean API cost per attempt.

Best score by test condition
Test condition	Without notes	With notes
42	58.9	28.5
43	12.0	39.6
44	52.1	37.3

Passes: 0/15 without notes; 0/15 with notes. Valid pours: 15/15 without notes; 15/15 with notes.

79.5%Lower cost: Sol vs Astra, with notes

A higher score, at lower cost.

With notes, GPT-6.1 Sol scores 76.9 at $0.0490 per attempt. GPT-6 Astra scores 74.7 at $0.2389.

27.9%GPT-5.5 cost reduction

Context can also reduce cost.

With notes, GPT-5.5 improves from 53.5 to 55.9 while mean attempt cost falls 27.9%. Costs are historical study estimates.

03 / 03

Average score and consistency.

Mean score across all attempts versus their standard deviation. Lower right is better: higher scores, less variation. Paired dots compare the same model with and without Lessons.

With LessonsWithout LessonsCMA-ES

GPT-6 Astra · All attempts

Without Lessons60.8mean score18.3 standard deviation

With Lessons69.8mean score12.6 standard deviation

Higher mean · less variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	62.4 / 13.5	81.2 / 6.1
43	61.5 / 24.1	73.0 / 3.5
44	58.5 / 19.9	55.1 / 7.6

Passes: 2/15 without Lessons; 5/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-6.1 Sol · All attempts

Without Lessons54.0mean score14.0 standard deviation

With Lessons56.9mean score16.2 standard deviation

Higher mean · more variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	48.0 / 17.2	51.2 / 15.5
43	62.5 / 8.2	64.9 / 9.0
44	51.5 / 13.4	54.5 / 21.6

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

Claude Opus 5.5 · All attempts

Without Lessons23.4mean score11.5 standard deviation

With Lessons40.5mean score26.4 standard deviation

Higher mean · more variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	16.5 / 8.8	40.6 / 31.1
43	27.9 / 9.1	50.6 / 25.6
44	25.7 / 14.4	30.4 / 23.6

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-5.5 · All attempts

Without Lessons45.6mean score10.3 standard deviation

With Lessons42.7mean score11.4 standard deviation

Lower mean · more variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	41.0 / 8.2	47.1 / 10.5
43	43.4 / 6.2	44.0 / 10.0
44	52.4 / 13.2	37.1 / 13.4

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-6 Sol · All attempts

Without Lessons42.3mean score13.5 standard deviation

With Lessons31.3mean score15.2 standard deviation

Lower mean · more variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	49.7 / 13.1	29.0 / 11.3
43	41.4 / 3.6	30.4 / 25.2
44	35.7 / 18.1	34.6 / 5.4

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-5.6 Terra · All attempts

Without Lessons23.5mean score12.5 standard deviation

With Lessons22.4mean score16.7 standard deviation

Lower mean · more variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	24.2 / 12.6	19.0 / 11.1
43	16.8 / 11.6	24.2 / 27.4
44	29.5 / 12.3	23.9 / 9.0

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-6 Luna · All attempts

Without Lessons23.6mean score12.1 standard deviation

With Lessons25.9mean score14.8 standard deviation

Higher mean · more variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	23.1 / 5.3	21.1 / 7.4
43	29.2 / 19.0	22.8 / 14.2
44	18.6 / 7.1	34.0 / 19.8

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

Claude Sonnet 5.5 · All attempts

Without Lessons18.8mean score16.3 standard deviation

With Lessons23.8mean score15.5 standard deviation

Higher mean · less variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	23.0 / 22.0	34.4 / 10.5
43	23.0 / 16.8	17.2 / 11.7
44	10.5 / 6.2	19.8 / 19.4

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-5.6 Luna · All attempts

Without Lessons38.6mean score11.6 standard deviation

With Lessons24.3mean score10.0 standard deviation

Lower mean · less variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	35.0 / 14.0	25.5 / 11.1
43	41.8 / 5.4	27.4 / 11.3
44	38.9 / 14.6	20.0 / 7.8

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

GPT-5.6 Sol · All attempts

Without Lessons20.5mean score18.8 standard deviation

With Lessons22.8mean score11.6 standard deviation

Higher mean · less variation

Mean and sample standard deviation across all 15 measured attempts per condition, pooled across three test conditions. Lower spread means scores stayed closer to their mean.

Mean / standard deviation by test condition
Test condition	Without Lessons	With Lessons
42	32.7 / 19.6	17.4 / 9.9
43	5.5 / 5.3	25.7 / 12.9
44	23.4 / 18.5	25.3 / 12.1

Passes: 0/15 without Lessons; 0/15 with Lessons. Measured scores: 15/15 and 15/15, respectively.

This is an average over all attempts, not the mean best score used in the other plots. GPT-5.5 Pro has no measured pours and is excluded, rather than treated as a perfectly consistent zero.

69.8/ 100GPT-6 Astra mean with Lessons

Astra improves on both measures.

With Lessons, Astra’s mean rises from 60.8 to 69.8, while its spread falls from 18.3 to 12.6 points.

26.4ptsClaude Opus 5.5 standard deviation

A higher mean can hide more variation.

Opus improves from 23.4 to 40.5 on average with Lessons, but its spread increases from 11.5 to 26.4 points.

View Latte Bench

Banyan Sapling.

An open-weight verification model built around a TriVector head. It evaluates claims against policies and evidence, and can abstain when the result is uncertain. Check whether a release is ready to deploy against your policies and test evidence.

Development evaluation

Raw accuracy Calibrated answer coverage

Combined

1,731 cases

Controlled synthetic

1,000 cases

ContractNLI

731 cases

100%

All 779 calibrated answers matched reference labels; 952 remained unresolved.

6/731

ContractNLI cases resolved under shared calibration.

Development results include benchmark exposure and correlated claims. They do not establish general deployment accuracy.

Stored parameters: 23B
Input token limit: 16,384
Output classes: 4
Open-weight licence: Apache 2.0

View on Hugging Face

Fine-tune at
the task boundary.

Different tasks. Bespoke models.
Versioned adapters, evaluated on the work they’re trained for.

AdapterNot ready yet

Qwen3-4B-Instruct-2507 4B · LoRA r16 · v1

0%Response agreement

Below targetTarget 95%

Eval trend

Current

Latency

3 / 16 ms

p50 / p95

Throughput

31 tok/s

Average

Requests · 30d: 0
Error rate: Not serving
Cost / 1k requests: $0.10
Training data: 560 examples · Corpus v1

AdapterNot ready yet

Qwen3-4B-Instruct-2507 4B · LoRA r16 · v1

33.3%Response agreement

Below targetTarget 95%

Eval trend

33.3%

Current

Latency

4 / 17 ms

p50 / p95

Throughput

29 tok/s

Average

Requests · 30d: 0
Error rate: Not serving
Cost / 1k requests: $0.12
Training data: 720 examples · Corpus v1

AdapterReady

Qwen3-4B-Instruct-2507 4B · LoRA r16 · v1

100%Response agreement

Target metTarget 95%

Eval trend

100%

Current

Latency

2 / 12 ms

p50 / p95

Throughput

38 tok/s

Average

Requests · 30d: 4,200
Error rate: 0.2%
Cost / 1k requests: $0.09
Training data: 1,280 examples · Corpus v2

AdapterPartial rollout

Qwen3-4B-Instruct-2507 4B · LoRA r16 · v2

75%Response agreement

Below targetTarget 85%

Eval trend

75%

Current

Latency

3 / 15 ms

p50 / p95

Throughput

34 tok/s

Average

Requests · 30d: 1,000
Error rate: 0.4%
Cost / 1k requests: $0.11
Training data: 960 examples · Corpus v2

AdapterNot ready yet

Qwen3-4B-Instruct-2507 4B · LoRA r16 · v1

0%Response agreement

Below targetTarget 95%

Eval trend

Current

Latency

3 / 14 ms

p50 / p95

Throughput

32 tok/s

Average

Requests · 30d: 0
Error rate: Not serving
Cost / 1k requests: $0.10
Training data: 640 examples · Corpus v1

AdapterReady

Qwen3-4B-Instruct-2507 4B · LoRA r16 · v1

96%Response agreement

Target metTarget 95%

Eval trend

96%

Current

Latency

2 / 11 ms

p50 / p95

Throughput

40 tok/s

Average

Requests · 30d: 2,800
Error rate: 0.1%
Cost / 1k requests: $0.08
Training data: 1,100 examples · Corpus v3

Put your experience
to work.

Build specialised models from the work
your agents already do.

Join the waitlist

Built around rigorous security and privacy standards.

Mubit’s certifications and compliance commitments cover how we protect data, manage security risks and govern AI responsibly.

Visit our Trust Center

Bring Your Own Cloud

Deploy Mubit inside your VPC, alongside the systems your agents already use. Bring task discovery, fine-tuning and evaluation into your own infrastructure.

AWS
Google Cloud
Azure

Hear what we’re thinking

View all essays

Research

Spearheading Adaptive Intelligence at Scale

# Mubit Own your means of production. Self-improving AI for scaling workloads. Join the waitlist: https://waitlist.mubit.ai ## Meet Mubit. Turn the work your agents repeat into specialised models you own. Discover tasks, fine-tune and evaluate, all in one platform. ### Platform overview (illustrative): Code reviewer, last 24 hours - Total requests: 2,847 (24 hours) - Routed requests: 80.0% (2,278 requests under task rules) - Median response: 3 ms (Across measured requests) - Adapters serving: 2 (2 live) Request routing: 80.0% of requests routed under task rules (adapter or base model). Errors: 0. Review code diffs v3 is ready for a silent test. #### Tasks - Review code diffs: live, 864 examples, 97.0% agreement, 1,898 requests - Judge code reviews: live, 420 examples, 96.0% agreement, 949 requests #### Recent activity - Review code diffs v3 started training on 864 kept examples. (Automatic · 2 d ago) - Review code diffs v2 went live. (Automatic · 2 d ago) - Judge code reviews v2 went live. (Automatic · 2 d ago) Explore the platform docs: https://docs.mubit.ai/ ## Research into runtime context. Latte Bench is our study of how retained context affects score, cost and consistency. Leading models plan a latte art pour in a simulator, five attempts at a time, under three test conditions. Each condition is a repeatable set of milk temperatures and volumes. Every model runs each one twice, once keeping written feedback from earlier attempts and once starting fresh. ### 01 Runtime context: What changes with context. Mean best score across three test conditions, five attempts each. Lessons retain written feedback in runtime context; model weights stay unchanged. - The leaderboard changes. GPT-6 Astra leads with context reset at 78.7. With context retained, GPT-6.1 Sol takes the lead at 76.9. (76.9 / 100, GPT-6.1 Sol with context) - Gains depend on the model. Claude Opus 5.5 rises from 38.8 to 67.0. Astra falls to 74.7, yet passes 5/15 attempts instead of 2/15. A higher score and a full pass are different outcomes. (+28.3 pts, Claude Opus 5.5 score gain) ### 02 Cost & performance: What a better score costs. Mean best score against mean API cost per attempt, on a log scale. Each pair compares the same model with and without retained runtime context. - A higher score, at lower cost. With notes, GPT-6.1 Sol scores 76.9 at $0.0490 per attempt. GPT-6 Astra scores 74.7 at $0.2389. (79.5 %, Lower cost: Sol vs Astra, with notes) - Context can also reduce cost. With notes, GPT-5.5 improves from 53.5 to 55.9 while mean attempt cost falls 27.9%. Costs are historical study estimates. (27.9 %, GPT-5.5 cost reduction) ### 03 Consistency: Average score and consistency. Mean score across all attempts versus their standard deviation. Lower right is better: higher scores, less variation. Paired dots compare the same model with and without Lessons. - Astra improves on both measures. With Lessons, Astra’s mean rises from 60.8 to 69.8, while its spread falls from 18.3 to 12.6 points. (69.8 / 100, GPT-6 Astra mean with Lessons) - A higher mean can hide more variation. Opus improves from 23.4 to 40.5 on average with Lessons, but its spread increases from 11.5 to 26.4 points. (26.4 pts, Claude Opus 5.5 standard deviation) View Latte Bench: https://mubit.ai/bench ## Banyan Sapling. An open-weight verification model built around a TriVector head. It evaluates claims against policies and evidence, and can abstain when the result is uncertain. Check whether a release is ready to deploy against your policies and test evidence. ### Development evaluation Raw accuracy: the highest-scoring class matches the reference label, before calibration. Calibrated answer coverage: the share of cases where calibration returns a resolved answer rather than unsure. - Combined (1,731 cases): raw accuracy 94.22%, calibrated answer coverage 45.00% - Controlled synthetic (1,000 cases): raw accuracy 100.00%, calibrated answer coverage 77.30% - ContractNLI (731 cases): raw accuracy 86.32%, calibrated answer coverage 0.82% - 100%: All 779 calibrated answers matched reference labels; 952 remained unresolved. - 6/731: ContractNLI cases resolved under shared calibration. Development results include benchmark exposure and correlated claims. They do not establish general deployment accuracy. - Stored parameters: 23B - Input token limit: 16,384 - Output classes: 4 (supported, violation, insufficient_information, inconsistent_context) - Open-weight licence: Apache 2.0 View on Hugging Face: https://huggingface.co/MubitLabs/banyan-sapling ## Fine-tune at the task boundary. Different tasks. Bespoke models. Versioned adapters, evaluated on the work they’re trained for. ### Adapters (illustrative) Base model: Qwen3-4B-Instruct-2507, 4B · LoRA r16. #### Plan multi-step workflows (v1) - Status: Not ready yet - Response agreement: 0% (target 95%, below target) - Latency: p50 3 ms, p95 16 ms - Throughput: 31 tok/s average - Requests served, 30 days: 0 - Error rate: Not serving - Estimated cost per 1k requests: $0.10 - Training data: 560 examples, corpus v1 - Serving: Not serving #### Extract structured data (v1) - Status: Not ready yet - Response agreement: 33.3% (target 95%, below target) - Latency: p50 4 ms, p95 17 ms - Throughput: 29 tok/s average - Requests served, 30 days: 0 - Error rate: Not serving - Estimated cost per 1k requests: $0.12 - Training data: 720 examples, corpus v1 - Serving: Not serving #### Triage support tickets (v1) - Status: Ready - Response agreement: 100% (target 95%, target met) - Latency: p50 2 ms, p95 12 ms - Throughput: 38 tok/s average - Requests served, 30 days: 4,200 - Error rate: 0.2% - Estimated cost per 1k requests: $0.09 - Training data: 1,280 examples, corpus v2 - Serving: v1, 42% of traffic #### Operate web apps (v2) - Status: Partial rollout - Response agreement: 75% (target 85%, below target) - Latency: p50 3 ms, p95 15 ms - Throughput: 34 tok/s average - Requests served, 30 days: 1,000 - Error rate: 0.4% - Estimated cost per 1k requests: $0.11 - Training data: 960 examples, corpus v2 - Serving: v2, 10% of traffic #### Research and summarize (v1) - Status: Not ready yet - Response agreement: 0% (target 95%, below target) - Latency: p50 3 ms, p95 14 ms - Throughput: 32 tok/s average - Requests served, 30 days: 0 - Error rate: Not serving - Estimated cost per 1k requests: $0.10 - Training data: 640 examples, corpus v1 - Serving: Not serving #### Review code changes (v1) - Status: Ready - Response agreement: 96% (target 95%, target met) - Latency: p50 2 ms, p95 11 ms - Throughput: 40 tok/s average - Requests served, 30 days: 2,800 - Error rate: 0.1% - Estimated cost per 1k requests: $0.08 - Training data: 1,100 examples, corpus v3 - Serving: v1, 28% of traffic ## Put your experience to work. Build specialised models from the work your agents already do. Join the waitlist: https://waitlist.mubit.ai ## Built around rigorous security and privacy standards. Mubit’s certifications and compliance commitments cover how we protect data, manage security risks and govern AI responsibly. - SOC 2 (Compliant): We meet SOC 2 requirements to ensure secure and compliant management of data across our systems. - ISO 42001 (Certified): ISO 42001 certified, our AI governance framework gives customers confidence in how we build and run AI. - ISO 27001 (Certified): Mubit is certified with ISO 27001, the internationally recognized standard for information security management. - GDPR (Compliant): We operate under GDPR, with rigorous requirements for data privacy and the protection of personal information. - HIPAA (Compliant): We safeguard your protected health information with HIPAA-compliant processes, policies, and technical controls. Visit our Trust Center: https://trust.mubit.ai/ ## Bring Your Own Cloud Deploy Mubit inside your VPC, alongside the systems your agents already use. Bring task discovery, fine-tuning and evaluation into your own infrastructure. - AWS - Google Cloud - Azure ## Hear what we’re thinking View all essays: https://mubit.ai/essays/ ## Links - Pricing: https://mubit.ai/pricing - Essays: https://mubit.ai/essays/ - Docs: https://docs.mubit.ai/ - Join the waitlist: https://waitlist.mubit.ai - Login: https://app.mubit.ai

Own your means of production.

Overview

Request volume

Request routing

Tasks 2

Recent activity

-4.1 points with Lessons

+5.1 points with Lessons

+28.3 points with Lessons

+2.3 points with Lessons

-9.0 points with Lessons

+8.1 points with Lessons

+3.7 points with Lessons

+3.9 points with Lessons

-12.7 points with Lessons

-5.9 points with Lessons

No score difference

The leaderboard changes.

Gains depend on the model.

Development evaluation

Combined

Controlled synthetic

ContractNLI

Plan multi-step workflows

Extract structured data

Triage support tickets

Operate web apps

Research and summarize

Review code changes

Put your experienceto work.

Built around rigorous security and privacy standards.

SOC 2

ISO 42001

ISO 27001

GDPR

HIPAA

Bring Your Own Cloud

Hear what we’re thinking

Spearheading Adaptive Intelligence at Scale

Put your experience
to work.