Performance
Threads, without changing the answer
pond-rs uses several cores through the parallel Cargo feature, which is on
by default. Two kinds of work are split across threads:
- the output columns of one
rollingoraggregatecall, and - the partitions of a
PartitionedSeries.
Each column, and each partition, is still computed start to finish by one thread, in order. That matters because adding the same numbers in a different order can change the last digit of the result. Because pond-rs never splits one column's rows between threads, the answer is the same, to the bit, on 1 thread or 16. A test checks this on 1, 4 and 7 threads.
The thread pool is rayon's. Set
RAYON_NUM_THREADS to choose its size. To build without threads:
pond-rs = { version = "0.1", default-features = false }
The limit of this design: a call with a single output column, on a series that is not partitioned, uses one core.
What was measured
The numbers below come from the pond-rs plan
(docs/plans/PND_RUST_PLAN.md)
and can be re-run from crates/pond-rs/bench/ and
crates/pond-rs/examples/. All were taken on the same 4-core machine.
Times are milliseconds of computing, not counting reading the file; lower is
better. Runs varied by roughly ±20%, so treat small differences as noise.
Rolling windows and buckets on 1 million rows
Workload A is a general one: 1 million rows one second apart, four number columns with about 5% gaps (and three 10-minute outages where everything is missing). It runs per-minute buckets with average, minimum, maximum and count, and a 5-minute rolling average and standard deviation. Workload B buckets the same file by calendar day in New York; the data crosses the 8 March 2026 clock change, so one day is 23 hours long. Each cell is the range over three full runs, each run the median of seven.
| Workload | pond-ts | pond-rs, 1 thread | pond-rs, 4 threads | polars, 1 thread | polars, 4 threads |
|---|---|---|---|---|---|
| A | 2901–3412 | 438–504 | 141–159 | 519–590 | 209–231 |
| B | 77–115 | 31–47 | 11–14 | 22–28 | 11–14 |
What this shows:
- On one core, pond-rs is about 6.5× faster than pond-ts on A and about 2.4× on B.
- On 4 cores, pond-rs is faster than polars on A and level with it on B.
- The one case where pond-rs loses: B on one core, where polars is faster (22–28 against 31–47 ms).
- Every pond-rs result here is identical, bit for bit, to pond-ts's.
Memory added during one run (MB, lower is better): on A, pond-rs 192, polars on one thread 256, pond-ts 362 (which includes memory its garbage collector had not yet freed); on B, pond-rs 78, polars 110.
Per-entity work: 500 symbols
500 symbols × 1,000 one-minute bars (500,000 rows, symbols interleaved, 2%
gaps), run through partition_by("symbol") and then put back together.
Middle (median) time of several runs:
| Task | pond-rs, 4 threads | pond-rs, 1 thread | pond-ts |
|---|---|---|---|
| split into partitions | 17.5 | 29.0 | 28.7 |
diff, then collect | 45.4 | 89.6 | 637 |
fill (linear), then collect | 45.6 | 107 | 811 |
| rolling 60-minute mean, stdev, last, then collect | 67.7 | 143 | 992 |
| rolling, then the last value per symbol, top 20 | 36.4 | 84.5 | 1072 |
Most of pond-ts's time here goes into putting the partitions back together (about 450 ms of it), which is logged as a pond-ts issue to fix.
Regular grids and duplicates: 1 million rows
From the step 5 pull request (#782); middle (median) time, every result checked against pond-ts.
| Task | pond-rs, 4 threads | pond-rs, 1 thread | pond-ts |
|---|---|---|---|
align hold onto a 1-second grid | 64 | 64 | 4565 |
align linear onto a 1-second grid | 127 | 113 | 1215 |
materialize last, 1-second grid | 72 | 75 | 990 |
dedupe last, 10% repeated times | 42 | 30 | 2314 |
| 500 symbols × 1,000: align, then collect | 127 | 145 | 3232 |
| 500 symbols × 1,000: dedupe, then collect | 76 | 135 | 2264 |
On a single series, more threads did not make these faster (on dedupe,
one thread was quicker); on the 500-symbol panel they did. Step 6 later
made copying text columns faster, which helps these: in that run,
materialize last went from 73 to 27 ms and align hold from 62 to 45 ms
(one thread).
Joins: 1 million rows
From the step 6 pull request (#785). Series a has 1 million rows a second apart; series b has 1 million rows every 1.5 seconds, so a third of a's times are in b. Each has a number column and a text column. Middle (median) time; pond-ts is the range over two separate runs; every result checked against pond-ts.
| Task | pond-rs | pond-ts |
|---|---|---|
| left join, a with b | 13–15 | 27–28 |
| inner join, a with b | 11–12 | 26–36 |
| outer join, a with b (1.67M rows out) | 87–89 | 67–75 |
| left join, a with a 10,000-row series | 10–11 | 16–18 |
| outer join, two series with the same times | 2.3–3.0 | 7–9 |
join_many of 3 with the same times | 5.8–6.1 | 16–23 |
join runs on one thread, so thread count does not change these. pond-ts's
join was already fast, so the gap is smaller than elsewhere. The outer join
is the one place pond-rs is slower: it builds about 100 MB of new output,
and asking the operating system for fresh memory costs about 37 ms of its 87. Told to keep and reuse freed memory, the same join took 50 ms. JavaScript
reuses its own memory between runs, so pond-ts does not pay that cost again.
JSON: 100,000 rows
From the JSON pull requests (#791,
#793): 100,000 rows of six
number columns, text in and out, middle (median) time. pond-ts's times
include JSON.stringify / JSON.parse.
| Task | pond-rs | pond-ts |
|---|---|---|
to_json, array rows | 34 | 148 |
to_json, object rows | 30 | 128 |
from_json | 45 | 152 |
to_columns_json | 30 | 72 |
from_columns_json | 31 | 46 |
to_points_json | 32 | 147 |
from_points_json | 65 | 175 |
One case was slower in pond-rs: reading columns JSON for a wide table (10,000 rows × 41 columns of small whole numbers) took 14 ms against pond-ts's 6.
List columns: 100,000 rows
From the step 8e pull request (#794): 100,000 rows of text lists (0 to 6 words) and number lists (0 to 8 values), a gap every tenth row. Middle (median) time.
| Task | pond-rs | pond-ts |
|---|---|---|
array_contains | 4.1 | 44 |
array_contains_any | 5.7 | 71 |
array_aggregate Avg | 6.9 | 366 |
array_explode, text | 38 | 891 |
to_json | 53 | 103 |
from_json | 162 | 445 |
pond-ts works through these row by row, which is logged as a pond-ts issue to fix.
The newer reducers: 1 million rows
From the step 8f pull request (#797):
1 million rows one second apart; a number column with a gap every 17th row
that repeats in runs of 40, and a host column, one of 8 names in runs of 90. Middle (median) time,
pond-rs then pond-ts. "Buckets" is aggregate into 1-second buckets (one
row each, so the per-bucket cost shows); "rolling" is a 5-minute window.
| Reducer | Buckets | Rolling | reduce |
|---|---|---|---|
Min (for comparison) | 35 vs 413 | 16 vs 209 | 1.1 vs 2 |
Keep, numbers | 35 vs 561 | 21 vs 190 | under 0.1 vs 102 |
Difference | 37 vs 816 | 27 vs 690 | 2.2 vs 353 |
Keep, text | 69 vs 415 | 49 vs 159 | 6.1 vs 71 |
Unique, text | 89 vs 1,792 | 105 vs 1,383 | 10 vs 94 |
Top(3), text | 89 vs 2,920 | 163 vs 2,619 | 11 vs 121 |
Samples, numbers | 74 vs 2,441 | 14,400 vs 97,800 | 22 vs 194 |
A longer window does not slow pond-rs's rolling Difference: 25 ms over a
whole day (86,400 rows in each window). pond-ts used to keep each window
sorted, so its time grew with the window: measured again on the same rows,
354 ms over 5 minutes, 557 ms over an hour and 11.3 seconds over a day. (The
690 ms in the table above is from the first run, which these re-runs did not
reproduce; the cause was not found.) pond-ts now works the way pond-rs does
(#799, released in pond-ts
v0.76.0), and takes about 250 ms at every window
length. Rolling Samples is slow in both because each of the million rows
gets a list of about 280 values; writing that out is the cost.
Rolling options: 1 million rows
From the step 8g pull request (#798): 1 million rows one second apart, a number column with a gap every 17th row. Middle (median) time in milliseconds; both sides give the same answers.
| Call | pond-rs | pond-ts |
|---|---|---|
5-minute Avg, trailing | 33 | 377 |
5-minute Avg, leading | 36 | 474 |
5-minute Avg, centred | 36 | 345 |
5-minute Avg, at least 300 rows | 30 | 576 |
20 rows Avg | 26 | 31 |
20 rows Stdev | 62 | 179 |
20 rows Avg, per host | 169 | 3,252 |
5-minute Avg once a minute, 100,000 rows | 0.7 | 12,856 |
5-minute Avg once a minute, 1M rows | 11 | not run |
5-minute Avg once a second, 1M rows | 352 | not run |
Row-count windows of Avg are only about 1.2× faster than pond-ts.
pond-ts already has a fast path for them, and both libraries add up the
running sum the same way, one step after another, so that they give
exactly the same answer; that sum sets the speed. pond-ts's grid windows
(rolling_over) look through every row for every grid point, which is why
it takes 12.9 seconds on 100,000 rows; that is logged as a pond-ts issue.
The once-a-second grid is pond-rs's slowest case: each of its million
windows (300 rows each) is added up from scratch.
Smoothing and bands: 1 million rows
From the step 8h pull request (#800): 1 million rows one second apart, a gap every 17th row (loess on smaller series, since its time grows with the square of the row count). Middle (median) time in milliseconds, 4 cores; both sides give the same answers.
| Call | pond-rs | pond-ts |
|---|---|---|
| EMA, alpha 0.1 | 5.6 | 12 |
| moving average, 5 minutes | 18 | 1,967 |
| moving average, 5 minutes, centred | 20 | 1,509 |
| loess, span 0.05, 20,000 rows | 16 | 113 |
| loess, span 0.3, 5,000 rows | 5.5 | 32 |
baseline, 5 minutes | 54 | 3,862 |
outliers, 5 minutes | 49 | 2,007 |
scan, running sum | 7.0 | 80 |
sample, every 10th row | 1.1 | 32 |
sample, reservoir of 1,000 | 1.0 | 17 |
The EMA is only about 2× faster: pond-ts's EMA already reads the column
straight through. Loess spreads its rows over the cores; on one core it
takes 56 and 20 ms. pond-ts's moving average, baseline and outliers
build an object per row, which is most of their time; that is logged as a
pond-ts issue.
Stacking and reshaping: 1 million rows
From the step 8i pull request (#802): 1 million rows in all. Middle (median) time in milliseconds, 4 cores, on a busy machine (runs varied by about ±30%); both sides give the same answers.
| Call | pond-rs | pond-ts |
|---|---|---|
concat of 10 series × 100,000 rows | 49 | 279 |
concat of 1,000 series × 1,000 rows | 50 | 432 |
collapse, sum of two columns | 24 | 145 |
pivot_by_group, 100 hosts × 10,000 times | 47 | 553 |
pivot_by_group, 4 hosts × 250,000 times | 39 | 751 |
pivot_by_group, 2 rows per cell, Avg | 72 | 619 |
Along a number column: 1 million rows
From the step 8i2 pull request (#803): 1 million rows. Middle (median) time in milliseconds, 4 cores, on a busy machine (runs varied by about ±30%); both sides give the same answers.
| Call | pond-rs | pond-ts |
|---|---|---|
by_column, 1,000 bins of width 10 | 26 | 93 |
by_column, 6 zones | 20 | 97 |
by_column, width 10, 5th and 95th percentiles | 38 | 415 |
rolling_by_column, ~100 rows a window, Avg | 35 | 265 |
rolling_by_column, ~100 rows a window, 5th and 95th | 94 | 843 |
rolling_by_column_at, 1,000 centers | 22 | 119 |
Value series
From the step 8i3 pull request (#804). Middle (median) time in milliseconds, 4 cores, on a busy machine (runs varied by about ±30%); both sides give the same answers.
| Call | pond-rs | pond-ts |
|---|---|---|
by_value, 1M rows | 3 | 34 |
nearest_index, 1M lookups | 166 | 357 |
slice_by_value, 100,000 slices | 90 | 401 |
to_json rows, 300,000 rows | 59 | 178 |
from_json rows, 300,000 rows | 100 | 129 |
by_value checks the axis once and copies nothing, which is why it is so
quick.
Arrow, from the step 8i4 pull request (#806): 1 million rows with a number axis, a number column with gaps and a text column.
| Call | pond-rs | pond-ts |
|---|---|---|
to_arrow | 1 | 0 |
from_arrow | 5 | 420 |
write_ipc | 10 | 27 |
from_ipc_bytes | 13 | 402 |
pond-ts's toArrow hands back views of its own memory, so it costs
nothing; it has no file writer of its own, so its write_ipc figure is the
apache-arrow library's. Most of pond-ts's reading time goes on the text
column, which is logged as a pond-ts issue.
Agents writing code against it
pond-rs is built mainly for AI agents writing Rust, so one test was about them. Twenty fresh agents were given the same task: daily average, minimum, maximum and count by New York calendar day over the benchmark file. Ten used pond-rs, ten used polars. None had web access; each could only read the library's source.
| pond-rs | polars | |
|---|---|---|
| right answer | 10 / 10 | 10 / 10 |
| right on the first run | 9 / 10 | 6 / 10 |
| failed runs before the first good one | 2 | 4 |
| failed runs in total | 2 | 10 |
Both sides got every answer right, so this does not show that pond-rs makes
agents more correct, only that they needed fewer tries. Six of polars'
failed runs came after a right answer, while agents were adding their own
checks. What the pond-rs agents struggled with (no way to rename or drop
fields on output, and having to add the Arrow crate themselves) has since
been fixed with select, rename, as_time and the Arrow re-exports.
A second trial (2026-10-29) did the same for the live layer. Ten agents used pond-rs and ten used plain Rust with no library (polars has no way to push one event at a time). The task: CPU readings from 13 hosts, arriving out of order, into one-minute buckets per host that wait 5 seconds for late events. All 20 got it right first time, matching pond-ts to the last bit, with no failed runs on either side. So this task does not separate the two: it spelled out every rule, and written out like that the rules come to about 60 lines of plain Rust.
The pond-rs agents took longer (69–110 s against 40–62 s) and used more tokens. Nine of ten also wrote the rules a second time by hand, to check the library against them. What they ran into is now covered on the live page (Per-host buckets over late events):
- closed buckets do not say which host they belong to (all ten);
- the series' grace window is easy to confuse with the buckets' delay (five);
- it was not clear whether the gathered series needs
Reorder(seven).
The trials' setups, tasks, graders and results are in
crates/pond-rs/bench/agent-trial/.
From JavaScript: @pond-ts/rs
The same work called from JavaScript through
@pond-ts/rs, against pond-ts (and @pond-ts/financial)
on the same data. Measured on 31 October 2026 on a 4-core Intel Xeon at
2.10 GHz, Node 22.22, one thread on both sides; each figure is the median
of several runs after warm-ups, in milliseconds, lower is better. The
scripts are packages/rs/scripts/perf-*.mjs. The operations themselves
are checked equal to pond-ts's, bit for bit, by the parity tests in CI.
| 1 million rows | @pond-ts/rs | pond-ts | Faster |
|---|---|---|---|
filter(col('v').gt(0)) (an expression) | 11 | 25 | 2.2× |
filter of v / w > 10 and host = 'h3' | 13 | 27 | 2.1× |
mapColumns with an expression | 5 | 18 | 3.5× |
withColumn with an expression | 3 | 10 | 3.1× |
filter with a JavaScript function | 623 | 35¹ | 0.06× |
sma({ period }) | 4.3 | 44.6 | 10× |
ema | 2.6 | 7.8 | 3× |
rsi | 9.2 | 36.6 | 4× |
bollinger | 22.7 | 84.2 | 3.7× |
macd | 17.4 | 87.6 | 5× |
atr | 8.5 | 24.6 | 2.9× |
sessionVwap (NYSE sessions) | 7.3 | 77.4 | 11× |
toJSON(), one number column | 41 | 175 | 4.3× |
toJSON(), number + text + true/false | 70 | 216 | 3.1× |
toJSON({ rowFormat: 'object' }), the same columns | 72 | 189 | 2.6× |
LiveSeries: push per event, nothing built on it | 273 | 1510 | 5.5× |
LiveSeries: pushMany of 1,000, nothing built on it | 42 | 73 | 1.7× |
rolling average + 1-minute buckets, push | 564 | 2830 | 5× |
rolling average + 1-minute buckets, pushMany | 337 | 944 | 2.8× |
| the same, plus a listener on every rolling update | 1329 | 2673 | 2× |
the same with pushMany | 606 | 829 | 1.4× |
¹ pond-ts with the same function (e => e.get('v') > 0), timed apart
from the scripts. The function row is the one to avoid: each row crosses
from WebAssembly into JavaScript and back. Use an expression (col,
lit, when) wherever the rule can be written as one.
What this shows: @pond-ts/rs is faster wherever the work stays inside
it, and the gain shrinks as more results cross back to JavaScript (a
listener on every update, many small outputs). The .wasm file is about
7.5 MB, 1.4 MB with brotli.