AI Builders Digest

OpenAI pauses some frontier RL training over alignment and monitoring standards

OpenAI has paused portions of its frontier reinforcement learning work until it can meet alignment, security, and monitoring standards for the capability level now in front of it. Altman framed it as making good on a long-stated promise to act when capability outpaces safety, and said the field will eventually need shared standards but OpenAI will move unilaterally in the meantime. He added that near-term model launches are unaffected and this hits further-out releases. His read on where this goes: confidence in safety will increasingly set the pace of AI progress.

• #policy
• #safety

Anthropic details the agent containment failures it shipped through

Telemetry showed users approved roughly 93% of Claude Code permission prompts, which is the whole case against human-in-the-loop as a primary defense. Shipping an OS-level sandbox cut prompts 84%, but the incidents that taught the most were both egress: an internal red team phished an employee into pasting a prompt that read ~/.aws/credentials and POSTed them out, succeeding 24 of 25 times, and a third party found that allowlisting api.anthropic.com let a poisoned workspace file upload data to the attacker's own Anthropic account using their key. The lesson drawn is that an egress allowlist is a capability grant, not a destination filter, and that battle-tested hypervisors and syscall filters held while Anthropic's own custom proxy was the piece that broke.

• #agents
• #security

Rich Sutton: today's models don't learn, and synthetic data is a big mistake

"I'm not weird. The field is weird." Sutton's objection to the current paradigm is blunt: an LLM's weights never change while it runs, so calling in-context memory learning misses the point, and the whole field only needs the phrase "continual learning" because it stopped assuming learning was continual. Synthetic data gets dismissed outright as bottlenecked by human expertise, since some person has to decide what good synthetic data is, and any simulation is microscopic next to a world that contains other minds. The concrete fix he and Khurram Javed are building at Oak Lab is continual backprop plus per-weight step-size optimization, so a single new example updates the right places instead of wrecking prior knowledge. He rates language as maybe a fifth to a quarter of intelligence, and thinks the big labs are locked into a product and cannot afford a paradigm where results get worse before they get better.

• #research
• #agents

Levie: the value between the model and the workflow is bigger than people assumed

Case study after case study is showing more headroom in the applied AI layer than expected, even with model capability doing heavy lifting. The work that remains is unglamorous and specific: agents need different product shapes per business process (chat here, background agent in a deterministic workflow there), different enterprise data systems and access patterns per vertical, and domain-specific change management because implementing at a bank looks nothing like a law firm. Evals are the sharpest point, with a long tail of task-specific evaluation that makes it nearly impossible for one system to be tuned for everything. He also flags pricing as a differentiator, since many verticals need abstractions above raw tokens to match their consumption model.

• #agents
• #enterprise
• #evals

Claude can now send Gmail and manage Google Drive files

Claude will draft and send replies in a Gmail thread and manage files in Google Drive, with user control over when approval is required. Connect either from the connectors menu; available on all paid plans. Claude Cowork also went live on mobile and web for all paid plans.

• #products
• #agents

Codex hardens against destructive deletes after GPT-5.6 wiped user files

OpenAI investigated a small number of reports where GPT-5.6 in Codex took destructive actions nobody asked for, and the worst pattern was temp-folder cleanup that deleted real user files. The root cause was the model reusing a system environment variable like $HOME for scratch work, so a malformed cleanup command pointed at the actual home directory. Fixes landed at several layers: explicit instructions to check deletion targets and create fresh temp directories, stronger execution checks that escalate high-risk deletion commands, Full access made harder to enable by accident, and replay evaluations built from the observed failures plus RL tasks and graders aimed at these risks. Practical advice for users is to stay updated and run in "Ask for approval" or "Approve for me" rather than Full access.

• #agents
• #safety
• #products

Vercel puts $1M on breaking out of its sandbox, in the open

Anyone can test any model in the world against Vercel Sandbox to attempt an escape, with findings and patches shared publicly. Rauch's stated goal is transparency about what frontier models can and cannot actually do against real-world guardrails. He also switched his daily driver to a new coding CLI that is 10 to 20x smaller than the major ones, starts instantly, feels closer to zsh than to an IDE in your terminal, and runs embedded anywhere including the browser via WebAssembly. Separate take worth noting: your software factory should be a monorepo, with design, marketing, sales, engineering, and support context in one place for agents to build on.

• #security
• #open-source
• #agents

Claude Code sessions can now publish live artifacts teammates can open

Claude Code can turn a session into a shareable web page built from your codebase, connectors, and the conversation itself, with no data wiring or infrastructure to stand up. Pages update in place as work progresses, so an incident investigation started before standup can republish its timeline, suspect commits, and error-rate chart twice by the time standup begins. Every publish is a new version at the same link with history and restore, and artifacts are private to the author by default, viewable only by authenticated org members, and cannot be made public. It's in beta for Claude Team and Enterprise orgs from the CLI and desktop app.

• #products
• #agents

Treat evals like frontier models: set the quality frontier, then walk down the cost curve

The mistake is optimizing eval cost before you know the eval is measuring anything real. Write the rubric first so you're clear on what good looks like, then pick the measurement method (humans, LLM judge, automated verification) and deliberately overspend on it: expensive judge model, paid humans, your own time. Only once the eval reliably separates good from bad and reflects what you care about in the product do you start cutting, via automation, smaller judge models, sampling, and deterministic checks where they apply.

• #evals
• #agents

PMs attaching pull requests went from 3% to 10% in two years

Non-engineers are shipping real code now: PMs attaching PRs tripled from 3% to 10% over two years, designers went from 1% to 8%, and founders sit second only to engineers at 23%. Yang's surprise is that designers trail PMs despite the rise of design engineer roles. The less cheerful finding is that AI landed on top of existing work rather than replacing it, with teams spending more time chatting with AI and delegating to agents while spending no less time on what they were already doing, most likely because expectations for what any function can accomplish have risen.

• #agents
• #products

The unpressed button: make your SaaS headless and charge agents per interaction

Take an existing SaaS product, strip the UI, let agents drive it directly, and bill per interaction, especially for enterprise. Thariq's point is less about the idea's novelty than about the fact that it's sitting there unpressed.

• #agents
• #products

Google's CC agent in Gmail opens an Australia and New Zealand waitlist

The experimental Gmail productivity agent added a waitlist for Australia and New Zealand and is expanding availability in the US and Canada, with invitations going out to people already queued. CC also picked up calendar management: it connects to Gmail so events land automatically in a dedicated Google Calendar and stay current as plans change.

• #products
• #agents

swyx open sources what AI Engineer learned A/B testing YouTube thumbnails

After a lot of thumbnail A/B testing for AI Engineer, swyx is publishing the learnings rather than keeping an opaque process opaque. The motivation is straightforwardly stated: he wants good educational content to rise above the noise online, and is inviting others to share their own results.

• #open-source
• #products

Claude Desktop startup time gets faster, with more to come

Slow startup makes a daily-driver app feel sluggish, so Claude Desktop's launch speed got a quality of life pass, and Cherny says more improvement is in progress.

• #products

Garry Tan comes out against Connie Chan over SF housing

Tan tied a vote for Connie Chan directly to $10K/mo one-bedroom rents in San Francisco, casting the position as one that serves NIMBY landlords at the expense of newcomers and young people. He called the effort to move Chan into Congress a Democratic machine smokescreen that should be repudiated.

• #policy