Training Data
Factory's Grinberg: a model-agnostic harness beats one co-designed with a model
The counterintuitive claim, aimed squarely at friends at OpenAI and Anthropic: building the model and the harness together does not make them better together. "What data is to a model, models are to a harness." Exposing a harness to many models stops it overfitting to one model's quirks, and Factory saw new Opus and GPT releases score better on TerminalBench inside Droid than in Claude Code or Codex. He also says being two years early is just being wrong, and backs it with the story of refunding nearly $2M of enterprise revenue because the product was not making developers happy. Open models went from under 1% of their customers' tokens at the start of the year to double digits now, with GLM 5.2 the standout, and he expects 90% of tokens to be asynchronous within 12 to 24 months.
- #agents
- #open-source
- #evals
