Blog / Engineering

Our benchmark was wrong three ways — all in our favour

September 5, 2026 · engineering · performance

performance react engine

We publish a side-by-side performance comparison between Grafloria and React Flow. You should not trust it — not because the numbers are wrong today, but because vendor benchmarks are exactly the genre of document where wrong numbers look most authoritative. Ours did. The first version of our harness was broken in three separate ways, and every one of the three flattered us. This post is the anatomy of how that happens, what a benchmark that cannot lie looks like, and what ours prints now — including the row we still lose.

The three lies

Nothing was fabricated. Every number was faithfully measured — of the wrong thing. That is what makes benchmark bugs worse than ordinary bugs: the output still looks like evidence.

• The pan gesture never panned us. The script pressed at a fixed (1000, 500) and dragged. In React Flow that landed on empty pane, so it timed a pan. In Grafloria the same pixel happened to be on a link — so it timed a link hit-test and a selection, while the camera never moved. The table compared them as if they were the same gesture.
• The drag gesture never moved a node. It pressed where node n0 had been before an earlier gesture moved the camera 2,400 pixels away, and then metered the resulting do-nothing frames. Sixty frames of nothing are very fast.
• The two libraries weren't looking at the same scene. React Flow's default minZoom stopped its fit-to-view early, so it drew a slice of the mesh while we drew all of it. And our bundler, walking up to the repo's tsconfig.json, silently resolved the packages to the working tree while the README claimed the published npm builds were being measured.

The fix is not "we corrected the numbers"

Corrected numbers rot the same way the originals did. The fix is a harness where each of those failures is a thrown error instead of a plausible row:

• Every pan starts on verified empty canvas in both libraries, and the run throws if the camera did not actually move.
• The drag re-frames first, finds its node by id, and throws if that node's position is unchanged afterwards.
• Both cameras are pinned to the same world rectangle, the framing of each run is recorded into results.json, and a fairness gate throws if the two libraries were not shown the same scene. The build mode (published packages vs working tree) is chosen explicitly and printed.

A benchmark that cannot fail cannot be trusted. Ours now fails loudly, which is the only reason the rest of this post is worth reading.

What the honest numbers bought us

The repaired harness immediately demoted us: at 2,000 nodes we were dropping frames on gestures the broken version had scored at a smooth 60fps. That hurt, and it was the most useful performance data we ever collected, because each ugly row pointed at a real defect:

• Our A* router scanned its whole frontier for the minimum on every iteration — 1,097ms of pathfinding inside one 40-move drag of a long edge. A binary heap and an adaptive search grid took it to 29ms, and fixed a worse bug the cost had been hiding: past ~2,200 world units the router used to give up and draw the edge straight through the obstacles it was avoiding.
• The engine handed out a fresh copy of its interaction config on every call — and it is called per port, per link, per frame. ~590ms of a drag gesture, gone by caching one frozen snapshot.
• A camera-only frame rebuilt the whole virtual-DOM tree. Since the SVG draws in world coordinates, a pan can be two attribute writes: a 100-frame pan now performs one real render instead of one hundred.

All of it shipped — the current npm releases carry every fix above.

Today's table

Apple M1 Pro, published packages (engine 0.3.6 / renderer 0.4.6), React Flow 12.11.3, both measured in the same session by the same scripted mouse:

Read it honestly: we mount ~1.8× faster and hold 60fps on both pan gestures. Node drag is the row neither library gets to 60fps — 25.6ms a frame for us against 38.5, and both of us drop the same worst-case frames. Dragging a node re-routes its edges, and that work is still on the frame. When that row reaches 16.7 we will say so; until then it says what it says.

One more way benchmarks lie: yesterday

Between two consecutive days, React Flow's drag row moved from 39.4ms to 40.7ms — on identical, untouched library code. That was the machine, not the library. Absolute milliseconds are only comparable within one run, which is why the harness always measures both libraries in the same session and why we never compare a today-number against a yesterday-table. If you see a vendor doing that, close the tab.

Run it yourself

git clone https://github.com/grafloria/grafloria
cd grafloria/benchmarks
npm install && node run.mjs

That reproduces the table against the published npm packages on your machine — the framings, build mode and environment land in results.json alongside the numbers. The benchmark README documents the three defects above in full, and the React Flow comparison page puts the numbers in context of everything that isn't performance. If you find a fourth way the harness can lie, we genuinely want the issue.