The $104 experiment I deleted

This past weekend, I ran an experiment that cost me $104.

I test new AI features regularly. I want to know what actually has ROI for my business, and what might be worth exploring with my clients.

Anthropic released Managed Agents on April 9. My Instagram feed filled up with people claiming the new feature let them run their entire businesses autonomously. I was skeptical and didn't try it.

Last week, Anthropic announced new Managed Agent features: Outcomes (a quality rubric that grades the agent's output and asks for revisions if it doesn't meet the bar) and Dreams (a memory feature that analyzes up to 100 prior sessions and reorganizes itself for better performance over time).

I've been refining my own 2-step intelligence layer that I posted about a few weeks ago. Step 1 pulls weekly content from dozens of podcasts, newsletters, and research feeds, then Step 2 synthesizes it against my strategic positioning and the questions I'm working through.

I was curious if I could improve step 2 with Managed Agents.

So on Saturday I built one (spoiler: what it looked like was me chatting with Claude Code and having it walk me through the Python script to copy into my Terminal).

Step 1 ran as usual in Claude Routine. Step 2 (synthesis) ran twice, in parallel, with identical inputs:

1️⃣ One Claude Routine that read the gathered content and wrote a 3,500-word weekly brief.

2️⃣ One Managed Agent with the same content, the same prompts, and an Outcomes rubric defining what "good" looked like.

When I woke up Sunday I had two 3,500-word briefs and a comparison document. I read both in detail.

They were both excellent. The Claude Routine was actually slightly stronger, but neither was significantly better than the other.

What was very different was the cost.

Managed Agents run on Anthropic's pay-as-you-go API. I'd set my account to auto-refill so I could see the real spend. Sunday morning, my invoice history showed eight separate top-ups totalling $104. That was for one synthesis run for Step 2 alone, which is less than half the actual work in the full pipeline.

It's possible the Dreams feature would meaningfully improve the output if I kept this running week after week and let the memory reorganize itself over time. But that's not something I can afford to test.

So I deleted the Managed Agent. I'll keep running my weekly brief on Claude Routines, which is included in the $200/month Max plan I already pay for and already adding significant value to my work.

I will continue to test new AI features as they become available. Through trying new tools and systems, I've built a variety of systems that have literally transformed how I work. But they're not always useful, and that's how I continue to learn.

I'm sure there are a lot of use cases where Managed Agents are adding significant value to different kinds of businesses. For me, this isn't one.

Has anyone else built with Anthropic's Managed Agents? Where have you found a use case with clear ROI?