Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration branch to check engine publishing for strangle-nix #4717

Closed
wants to merge 50 commits into from

Conversation

miguelff
Copy link
Contributor

@miguelff miguelff commented Feb 12, 2024

QA testing of #4713

@miguelff miguelff requested a review from a team as a code owner February 12, 2024 11:35
@miguelff miguelff requested review from jkomyno and Druue and removed request for a team February 12, 2024 11:35
@miguelff miguelff mentioned this pull request Feb 12, 2024
Copy link
Contributor

github-actions bot commented Feb 12, 2024

WASM Size

Engine This PR Base branch Diff
Postgres 2.073MiB 2.073MiB 0.000B
Postgres (gzip) 815.762KiB 815.762KiB 0.000B
Mysql 2.055MiB 2.055MiB 0.000B
Mysql (gzip) 807.253KiB 807.252KiB 1.000B
Sqlite 2.015MiB 2.015MiB 0.000B
Sqlite (gzip) 794.301KiB 794.303KiB -2.000B

Copy link
Contributor

github-actions bot commented Feb 12, 2024

✅ WASM query-engine performance won't change substantially (1.003x)

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.0 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - 25000)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  300.63 ms/iter (298.71 ms … 303.54 ms) 302.61 ms 303.54 ms 303.54 ms
Web Assembly: Latest    382.68 ms/iter (381.15 ms … 385.34 ms) 385.29 ms 385.34 ms 385.34 ms
Web Assembly: Current   384.15 ms/iter (382.14 ms … 389.24 ms) 384.93 ms 389.24 ms 389.24 ms
Node API: Current        220.9 ms/iter (216.93 ms … 224.66 ms) 222.91 ms 224.66 ms 224.66 ms

summary for movies.findMany() (all - 25000)
  Web Assembly: Current
   1.74x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.02 ms/iter   (11.78 ms … 13.71 ms)  11.92 ms  13.71 ms  13.71 ms
Web Assembly: Latest     15.82 ms/iter    (15.3 ms … 18.91 ms)   15.7 ms  18.91 ms  18.91 ms
Web Assembly: Current    15.87 ms/iter   (15.61 ms … 17.16 ms)  15.82 ms  17.16 ms  17.16 ms
Node API: Current        8,590 µs/iter   (8,476 µs … 8,809 µs)  8,640 µs  8,809 µs  8,809 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.85x slower than Node API: Current
   1.32x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,934 µs/iter   (1,843 µs … 3,165 µs)  1,922 µs  2,696 µs  3,165 µs
Web Assembly: Latest     2,485 µs/iter   (2,385 µs … 3,506 µs)  2,485 µs  3,087 µs  3,506 µs
Web Assembly: Current    2,528 µs/iter   (2,400 µs … 3,891 µs)  2,520 µs  3,623 µs  3,891 µs
Node API: Current        1,466 µs/iter   (1,396 µs … 1,875 µs)  1,481 µs  1,650 µs  1,875 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.72x slower than Node API: Current
   1.31x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    12.1 ms/iter   (11.99 ms … 12.91 ms)  12.13 ms  12.91 ms  12.91 ms
Web Assembly: Latest     15.69 ms/iter   (15.58 ms … 16.27 ms)   15.7 ms  16.27 ms  16.27 ms
Web Assembly: Current    15.74 ms/iter   (15.65 ms … 15.92 ms)  15.77 ms  15.92 ms  15.92 ms
Node API: Current        8,656 µs/iter   (8,463 µs … 9,083 µs)  8,735 µs  9,083 µs  9,083 µs

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.82x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,908 µs/iter   (1,824 µs … 2,972 µs)  1,907 µs  2,564 µs  2,972 µs
Web Assembly: Latest     2,485 µs/iter   (2,376 µs … 3,627 µs)  2,459 µs  3,433 µs  3,627 µs
Web Assembly: Current    2,481 µs/iter   (2,389 µs … 3,322 µs)  2,481 µs  2,955 µs  3,322 µs
Node API: Current        1,465 µs/iter   (1,384 µs … 1,673 µs)  1,481 µs  1,633 µs  1,673 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.69x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.09 ms/iter   (12.01 ms … 12.36 ms)  12.12 ms  12.36 ms  12.36 ms
Web Assembly: Latest     15.74 ms/iter   (15.61 ms … 15.95 ms)  15.78 ms  15.95 ms  15.95 ms
Web Assembly: Current    15.78 ms/iter   (15.66 ms … 15.92 ms)  15.81 ms  15.92 ms  15.92 ms
Node API: Current        8,792 µs/iter   (8,542 µs … 9,095 µs)  8,874 µs  9,095 µs  9,095 µs

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.8x slower than Node API: Current
   1.31x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,864 µs/iter   (1,798 µs … 2,228 µs)  1,872 µs  2,097 µs  2,228 µs
Web Assembly: Latest     2,449 µs/iter   (2,380 µs … 3,122 µs)  2,449 µs  2,784 µs  3,122 µs
Web Assembly: Current    2,453 µs/iter   (2,393 µs … 2,969 µs)  2,454 µs  2,802 µs  2,969 µs
Node API: Current        1,454 µs/iter   (1,383 µs … 1,736 µs)  1,471 µs  1,688 µs  1,736 µs

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.69x slower than Node API: Current
   1.32x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  886.97 µs/iter   (840.1 µs … 1,482 µs) 892.18 µs  1,261 µs  1,482 µs
Web Assembly: Latest     1,203 µs/iter   (1,145 µs … 1,802 µs)  1,207 µs  1,522 µs  1,802 µs
Web Assembly: Current    1,216 µs/iter   (1,154 µs … 1,866 µs)  1,221 µs  1,587 µs  1,866 µs
Node API: Current       814.67 µs/iter  (749.45 µs … 1,014 µs) 829.18 µs 948.15 µs  1,014 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.49x slower than Node API: Current
   1.37x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  907.22 µs/iter  (870.08 µs … 1,452 µs) 912.13 µs  1,162 µs  1,452 µs
Web Assembly: Latest     1,212 µs/iter   (1,153 µs … 2,050 µs)  1,208 µs  1,978 µs  2,050 µs
Web Assembly: Current    1,216 µs/iter   (1,161 µs … 1,726 µs)  1,222 µs  1,585 µs  1,726 µs
Node API: Current       829.24 µs/iter  (742.71 µs … 1,537 µs)  835.7 µs  1,175 µs  1,537 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.47x slower than Node API: Current
   1.34x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

After changes in d771f85

Copy link

codspeed-hq bot commented Feb 12, 2024

CodSpeed Performance Report

Merging #4717 will not alter performance

Comparing integration/strangle-nix (d771f85) with main (9878210)

Summary

✅ 11 untouched benchmarks

@miguelff miguelff force-pushed the integration/strangle-nix branch 2 times, most recently from bb0e934 to d2631c5 Compare February 12, 2024 12:47
@miguelff miguelff closed this Feb 12, 2024
@miguelff miguelff deleted the integration/strangle-nix branch February 12, 2024 12:56
@miguelff miguelff restored the integration/strangle-nix branch February 12, 2024 17:55
@miguelff miguelff reopened this Feb 12, 2024
@miguelff miguelff force-pushed the integration/strangle-nix branch from 169c448 to d771f85 Compare February 13, 2024 10:00
@aqrln aqrln closed this Feb 13, 2024
@aqrln aqrln deleted the integration/strangle-nix branch February 13, 2024 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants