Previously we published Pls Fix Bench, which tested 195 targeted PowerPoint edits in a terminal environment. For this benchmark we compared how PowerPoint add-ins using the same underlying model handle an identical prompt.
Defining the task
Corporate presentations are rarely built from a blank canvas. Consultants, investment bankers, and other professionals build slides by recycling components: a table from last week's market update, a chart from a recent board meeting, and a layout that delighted a client two months ago. They pull in different pieces and adjust them for the new context and data.
Our goal was to replicate that workflow with AI.
Assembling the building blocks
We took three slides from Pls Fix Bench:
- An income statement table
- A stacked revenue chart
- A two-chart slide
We pulled summary financials from SpaceX's S-1 and saved them to a simple Excel sheet.
The prompt
"Using the attached data, add a new slide with a layout like slide 3. On the left side add an income statement table, and on the right side a stacked chart showing revenue by segment (in billions). Follow the style on slide 1 for the table and slide 2 for the chart."
The first run
editide completed the first run in 75 seconds, and cost 71 cents. Claude finished in 222 seconds and cost $1.63. That means editide was 3x faster and 2.3x cheaper.
What about the slides themselves?
Take a closer look at the Claude slide with the eyes of an investment banking vice president. I would have gotten yelled at for turning in Claude's slide, and would have gotten an email with these comments:
- Fix the title, why does it say ACME corp?
- The subtitle has weird margin on the left, pls fix
- "Figures in $MM" too hard to read, use darker font
- In the table, use different color highlights (see original)
- Add borders for the table subtotals (see original table)
- Stacked chart missing totals data labels
- Stacked chart missing units
The editide slide would get a "Looks good, pls send. Thx".
A chance at redemption
At this point, I thought we were done but I knew we'd get allegations of cherry-picking the runs. "Claude got unlucky." "editide got lucky."
So we repeated the exercise two more times. Note: I'd love to do these until hitting a statistically significant number, but setting up the screen recordings and tracking down the token counts took me longer than I'd like to admit.
The results across three runs
| Metric | editide | Claude | Difference |
|---|---|---|---|
| Cost | $0.71 | $1.63 | 2.3x cheaper |
| Latency (seconds) | 91.1s | 244.9s | 2.7x faster |
This is a small sample, but the direction was consistent: editide finished faster in all three runs, cost less in all three runs, and more faithfully reproduced the source slides' styles.
The underlying model matters. But so does the entire infrastructure built around it.
Appendix
Raw run data 6 measured Opus 5 runs
Claude for PowerPoint
| Run | Latency | Output tokens | Cache read | Cost |
|---|---|---|---|---|
| 1 | 221.6s | 14,259 | 1,882,327 | $1.63 |
| 2 | 297.7s | 16,226 | 2,062,299 | $1.72 |
| 3 | 244.9s | 14,408 | 1,687,474 | $1.48 |
editide
| Run | Latency | Output tokens | Cache read | Cost |
|---|---|---|---|---|
| 1 | 75.1s | 6,608 | 668,053 | $0.71 |
| 2 | 91.1s | 4,791 | 682,219 | $0.68 |
| 3 | 110.2s | 6,396 | 797,443 | $0.80 |