We just wrapped a full 445-task Terminal-Bench sweep using our Harbor-based Blobfish agent pointed at a locally hosted MiniMax-M2.5 Q8_0 model. Below is the first look at how the Mac Studio rig held up and what the reward distribution tells us about the remaining work on the agent side.

Setup at a glance

• Model + host: MiniMax-M2.5 Q8_0 running on an M3 Ultra Mac Studio (512 GB unified). llama.cpp server exposed on localhost:8081 and bridged into Harbor via ANTHROPIC_BASE_URL=http://host.docker.internal:8081.
• Agent: blobfish_harbor:BlobfishAgent (prompt variant auto, so MiniMax-specific CLAUDE.md + hook stack). Agent metadata: agent_name=cchuter, agent_org=teamblobfish.com.
• Benchmark harness: ./scripts/run-terminal-bench.sh --backend claude --model minimax/minimax-m2.5 -k 5 -n 1 with retries disabled for timeout-class exceptions, Docker host networking enabled, and the standard CLAUDE hook suite writing ATIF trajectories.
• Run envelope: 2026-03-08 22:02 UTC → 2026-03-14 20:07 UTC. One long soak to validate MiniMax stability under a continuous multi-day agent workload.

Outcome snapshot

┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Metric                   ┃ Value                  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Agent                    │ cchuter (minimax-m2.5) │
│ Dataset                  │ terminal-bench         │
│ Trials                   │ 441                    │
│ Errors                   │ 178                    │
│                          │                        │
│ Mean                     │ 0.422                  │
│                          │                        │
│ Reward Distribution      │                        │
│   reward = 0.0           │ 253                    │
│   reward = 1.0           │ 188                    │
│                          │                        │
│ Exception Distribution   │                        │
│   AgentTimeoutError      │ 174                    │
│   AgentSetupTimeoutError │ 1                      │
│   RuntimeError           │ 2                      │
│   VerifierTimeoutError   │ 1                      │
└──────────────────────────┴────────────────────────┘

Key takeaways:

1. Mean reward landed at 0.422. That’s in line with the best MiniMax runs we’ve seen to date and confirms the local inference stack is stable enough for leaderboard submissions.
2. Agent timeouts dominate error budget. 174 out of 178 recorded failures were straight AgentTimeoutErrors at the 900 s mark. The model rarely crashes, but it can stall when Claude takes a wrong branching turn. Since task timeouts are fixed on the public bench, the best lever is higher effective token throughput: keep nudging llama.cpp performance (Metal kernels, cache types, context sizing) so MiniMax gets more reasoning done inside the existing window.
3. Verification issues are “userland.” 253 tasks ran to completion and failed tests, meaning there’s still plenty of work to do on reasoning quality, artifact handling, and task-specific strategies—not on the infrastructure path.

What’s next

• Triage the worst-offending tasks. Focus first on the timeout-heavy categories (gpt2-codegolf, path-tracing*, crack-7z-hash, etc.) where Claude often wanders until the watchdog fires.
• Keep squeezing more tokens per second. Because timeout budgets are fixed, every kernel or caching improvement that boosts throughput directly reduces the number of agent stalls.
• Keep the MiniMax infra lean. llama.cpp 8240 (or HEAD) plus Metal’s latest kernels are performing well—stick with the current --ctx-size 131072, --cache-type-k v q8_0, --flash-attn on recipe unless a future release gives us back more reward.

We’ll post follow-ups as we drive the mean closer to 0.5, but for now this gives Team Blobfish a baseline reference when we talk about MiniMax M2.5 performance on Terminal-Bench. Photos, plots, and the raw JSON live in the repo under docs/results for anyone who wants to dig deeper.