Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use an array for the relation maybe stack while stack is small #58852

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Use an array for the relation maybe stack while stack is small #58852

wants to merge 4 commits into from

Conversation

jakebailey
Copy link
Member

Profiling our slowest compiler tests shows a lot of time spent adding/deleting from the set.

Try using an array while the stack is small.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 13, 2024
@jakebailey

This comment was marked as outdated.

Sign in to view
@typescript-bot

This comment was marked as outdated.

Sign in to view
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 13, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,360k (± 0.09%) 192,405k (± 0.10%) ~ 192,262k 192,691k p=1.000 n=6
Parse Time 1.58s (± 0.98%) 1.58s (± 0.35%) ~ 1.57s 1.58s p=0.859 n=6
Bind Time 0.86s (± 0.97%) 0.87s (± 1.21%) ~ 0.85s 0.88s p=1.000 n=6
Check Time 11.40s (± 0.39%) 11.31s (± 0.44%) -0.09s (- 0.79%) 11.25s 11.40s p=0.024 n=6
Emit Time 3.27s (± 0.81%) 3.27s (± 0.51%) ~ 3.25s 3.30s p=0.681 n=6
Total Time 17.11s (± 0.34%) 17.02s (± 0.27%) -0.10s (- 0.56%) 16.98s 17.11s p=0.024 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,104 944,104 ~ ~ ~ p=1.000 n=6
Types 407,044 407,044 ~ ~ ~ p=1.000 n=6
Memory used 1,218,546k (± 0.01%) 1,218,671k (± 0.00%) +125k (+ 0.01%) 1,218,603k 1,218,744k p=0.013 n=6
Parse Time 7.96s (± 0.17%) 7.97s (± 0.43%) ~ 7.91s 8.01s p=0.373 n=6
Bind Time 2.24s (± 0.40%) 2.24s (± 0.49%) ~ 2.23s 2.25s p=1.000 n=6
Check Time 36.24s (± 0.24%) 36.21s (± 0.25%) ~ 36.09s 36.34s p=0.748 n=6
Emit Time 17.86s (± 0.35%) 17.84s (± 0.24%) ~ 17.80s 17.89s p=0.627 n=6
Total Time 64.30s (± 0.15%) 64.26s (± 0.16%) ~ 64.15s 64.43s p=0.422 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 2,132,301 2,132,301 ~ ~ ~ p=1.000 n=6
Types 926,081 926,081 ~ ~ ~ p=1.000 n=6
Memory used 2,114,978k (± 0.01%) 2,115,242k (± 0.01%) +264k (+ 0.01%) 2,115,099k 2,115,440k p=0.005 n=6
Parse Time 7.85s (± 0.38%) 7.86s (± 0.29%) ~ 7.82s 7.88s p=1.000 n=6
Bind Time 2.76s (± 0.72%) 2.74s (± 0.38%) ~ 2.73s 2.76s p=0.301 n=6
Check Time 84.20s (± 0.59%) 84.12s (± 0.47%) ~ 83.73s 84.70s p=0.936 n=6
Emit Time 0.16s (± 3.95%) 0.16s (± 2.52%) ~ 0.16s 0.17s p=0.673 n=6
Total Time 94.97s (± 0.50%) 94.88s (± 0.41%) ~ 94.51s 95.46s p=1.000 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,572 1,230,848 +276 (+ 0.02%) ~ ~ p=0.001 n=6
Types 261,228 261,292 +64 (+ 0.02%) ~ ~ p=0.001 n=6
Memory used 2,377,401k (± 2.61%) 2,377,618k (± 2.57%) ~ 2,351,232k 2,502,496k p=0.689 n=6
Parse Time 4.93s (± 0.77%) 4.94s (± 0.69%) ~ 4.89s 4.98s p=0.575 n=6
Bind Time 1.90s (± 0.92%) 1.90s (± 0.43%) ~ 1.89s 1.91s p=0.730 n=6
Check Time 34.01s (± 0.48%) 34.03s (± 0.39%) ~ 33.85s 34.15s p=1.000 n=6
Emit Time 2.59s (± 3.62%) 2.60s (± 2.37%) ~ 2.53s 2.69s p=0.689 n=6
Total Time 43.44s (± 0.44%) 43.51s (± 0.34%) ~ 43.34s 43.71s p=0.689 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,572 1,230,848 +276 (+ 0.02%) ~ ~ p=0.001 n=6
Types 261,228 261,292 +64 (+ 0.02%) ~ ~ p=0.001 n=6
Memory used 2,479,602k (± 3.13%) 2,429,341k (± 0.05%) ~ 2,427,138k 2,430,345k p=0.575 n=6
Parse Time 6.22s (± 0.88%) 6.21s (± 1.24%) ~ 6.14s 6.36s p=1.000 n=6
Bind Time 2.03s (± 0.44%) 2.04s (± 0.51%) ~ 2.03s 2.05s p=0.453 n=6
Check Time 40.54s (± 0.12%) 40.36s (± 0.37%) ~ 40.20s 40.58s p=0.054 n=6
Emit Time 3.10s (± 3.08%) 3.09s (± 3.94%) ~ 2.98s 3.32s p=0.471 n=6
Total Time 51.90s (± 0.25%) 51.70s (± 0.50%) ~ 51.47s 52.18s p=0.093 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 259,198 259,342 +144 (+ 0.06%) ~ ~ p=0.001 n=6
Types 105,191 105,255 +64 (+ 0.06%) ~ ~ p=0.001 n=6
Memory used 428,899k (± 0.01%) 429,034k (± 0.01%) +135k (+ 0.03%) 428,973k 429,099k p=0.005 n=6
Parse Time 4.07s (± 0.51%) 4.08s (± 0.59%) ~ 4.06s 4.12s p=0.330 n=6
Bind Time 1.64s (± 0.81%) 1.65s (± 0.91%) ~ 1.62s 1.66s p=0.492 n=6
Check Time 22.39s (± 0.47%) 22.31s (± 0.44%) ~ 22.17s 22.43s p=0.199 n=6
Emit Time 1.73s (± 1.44%) 1.73s (± 1.46%) ~ 1.69s 1.76s p=0.872 n=6
Total Time 29.82s (± 0.45%) 29.78s (± 0.27%) ~ 29.65s 29.87s p=0.378 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,528k (± 0.01%) 369,577k (± 0.04%) ~ 369,488k 369,860k p=0.689 n=6
Parse Time 2.77s (± 0.56%) 2.76s (± 0.40%) ~ 2.74s 2.77s p=0.359 n=6
Bind Time 1.59s (± 0.73%) 1.59s (± 0.62%) ~ 1.58s 1.61s p=1.000 n=6
Check Time 15.65s (± 0.38%) 15.65s (± 0.37%) ~ 15.60s 15.73s p=0.936 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.01s (± 0.31%) 20.00s (± 0.30%) ~ 19.94s 20.08s p=0.467 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,853,506 2,853,506 ~ ~ ~ p=1.000 n=6
Types 968,049 968,049 ~ ~ ~ p=1.000 n=6
Memory used 3,020,457k (± 0.00%) 3,020,815k (± 0.00%) +358k (+ 0.01%) 3,020,751k 3,020,923k p=0.005 n=6
Parse Time 11.39s (± 0.29%) 11.39s (± 0.26%) ~ 11.36s 11.42s p=0.871 n=6
Bind Time 3.49s (± 0.22%) 3.49s (± 0.26%) ~ 3.48s 3.50s p=0.798 n=6
Check Time 64.38s (± 1.97%) 63.54s (± 0.19%) -0.85s (- 1.32%) 63.37s 63.72s p=0.013 n=6
Emit Time 19.68s (± 6.73%) 20.21s (± 0.48%) ~ 20.14s 20.40s p=0.810 n=6
Total Time 98.94s (± 0.16%) 98.63s (± 0.19%) -0.31s (- 0.32%) 98.45s 98.93s p=0.030 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,914 266,914 ~ ~ ~ p=1.000 n=6
Types 108,685 108,685 ~ ~ ~ p=1.000 n=6
Memory used 411,372k (± 0.01%) 411,397k (± 0.01%) ~ 411,363k 411,424k p=0.230 n=6
Parse Time 3.81s (± 0.72%) 3.81s (± 0.64%) ~ 3.78s 3.85s p=0.871 n=6
Bind Time 1.69s (± 0.45%) 1.69s (± 0.45%) ~ 1.68s 1.70s p=1.000 n=6
Check Time 16.96s (± 0.42%) 16.93s (± 0.42%) ~ 16.82s 17.02s p=0.748 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.46s (± 0.26%) 22.43s (± 0.29%) ~ 22.35s 22.52s p=0.298 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 523,429 523,429 ~ ~ ~ p=1.000 n=6
Types 177,955 177,955 ~ ~ ~ p=1.000 n=6
Memory used 461,902k (± 0.02%) 461,763k (± 0.05%) ~ 461,314k 461,938k p=0.298 n=6
Parse Time 2.63s (± 0.34%) 2.63s (± 0.29%) ~ 2.62s 2.64s p=0.798 n=6
Bind Time 1.00s (± 0.55%) 0.99s (± 0.41%) ~ 0.99s 1.00s p=0.282 n=6
Check Time 15.34s (± 0.42%) 15.30s (± 0.34%) ~ 15.23s 15.37s p=0.332 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.96s (± 0.37%) 18.91s (± 0.29%) ~ 18.84s 18.99s p=0.295 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,287ms (± 0.69%) 2,291ms (± 0.48%) ~ 2,274ms 2,299ms p=0.748 n=6
Req 2 - geterr 5,144ms (± 0.68%) 5,125ms (± 0.26%) ~ 5,102ms 5,141ms p=0.298 n=6
Req 3 - references 265ms (± 0.50%) 262ms (± 1.79%) ~ 258ms 271ms p=0.076 n=6
Req 4 - navto 227ms (± 0.18%) 229ms (± 0.66%) +2ms (+ 0.95%) 228ms 231ms p=0.007 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 83ms (± 5.29%) 80ms (± 3.49%) ~ 77ms 83ms p=0.290 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,593ms (± 0.53%) 3,581ms (± 1.09%) ~ 3,531ms 3,640ms p=0.378 n=6
Req 2 - geterr 5,690ms (± 0.68%) 5,699ms (± 0.58%) ~ 5,659ms 5,753ms p=0.936 n=6
Req 3 - references 410ms (± 0.40%) 409ms (± 0.20%) ~ 408ms 410ms p=0.869 n=6
Req 4 - navto 340ms (± 0.96%) 339ms (± 0.12%) ~ 339ms 340ms p=1.000 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 108ms (± 0.83%) 108ms (± 1.17%) ~ 106ms 109ms p=0.933 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,160ms (± 0.48%) 6,180ms (± 1.30%) ~ 6,110ms 6,296ms p=0.810 n=6
Req 2 - geterr 1,639ms (± 7.28%) 1,661ms (± 7.99%) ~ 1,391ms 1,729ms p=0.078 n=6
Req 3 - references 131ms (± 9.67%) 130ms (± 9.51%) ~ 107ms 145ms p=1.000 n=6
Req 4 - navto 582ms (± 3.25%) 586ms (± 1.15%) ~ 575ms 595ms p=0.423 n=6
Req 5 - completionInfo count 3,412 3,412 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,247ms (± 1.61%) 1,232ms (± 1.01%) ~ 1,214ms 1,243ms p=0.229 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 228.46ms (± 0.23%) 228.49ms (± 0.18%) ~ 226.54ms 231.93ms p=0.181 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 361.74ms (± 0.27%) 361.66ms (± 0.27%) -0.08ms (- 0.02%) 353.31ms 371.26ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 237.49ms (± 0.13%) 237.19ms (± 0.16%) -0.29ms (- 0.12%) 235.68ms 242.15ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 236.99ms (± 0.17%) 237.21ms (± 0.17%) +0.23ms (+ 0.10%) 235.53ms 241.55ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 24, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,582k (± 0.94%) 193,581k (± 0.91%) ~ 192,299k 195,949k p=0.810 n=6
Parse Time 1.58s (± 0.93%) 1.57s (± 0.80%) ~ 1.55s 1.58s p=0.459 n=6
Bind Time 0.86s (± 1.04%) 0.85s (± 1.42%) ~ 0.84s 0.87s p=0.362 n=6
Check Time 11.34s (± 0.29%) 11.20s (± 0.58%) -0.14s (- 1.22%) 11.11s 11.30s p=0.006 n=6
Emit Time 3.26s (± 0.88%) 3.25s (± 1.08%) ~ 3.22s 3.30s p=0.571 n=6
Total Time 17.04s (± 0.20%) 16.88s (± 0.54%) -0.16s (- 0.91%) 16.79s 16.99s p=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors 7 7 ~ ~ ~ p=1.000 n=6
Symbols 945,532 945,532 ~ ~ ~ p=1.000 n=6
Types 409,507 409,507 ~ ~ ~ p=1.000 n=6
Memory used 1,221,098k (± 0.00%) 1,221,250k (± 0.00%) +152k (+ 0.01%) 1,221,204k 1,221,313k p=0.005 n=6
Parse Time 7.91s (± 1.00%) 7.87s (± 0.49%) ~ 7.82s 7.93s p=0.418 n=6
Bind Time 2.22s (± 0.40%) 2.21s (± 0.60%) ~ 2.19s 2.23s p=0.111 n=6
Check Time 36.42s (± 0.48%) 36.29s (± 0.49%) ~ 36.01s 36.53s p=0.230 n=6
Emit Time 17.86s (± 0.34%) 17.82s (± 0.68%) ~ 17.61s 17.96s p=0.630 n=6
Total Time 64.42s (± 0.41%) 64.18s (± 0.45%) ~ 63.66s 64.46s p=0.298 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,324,872 2,324,872 ~ ~ ~ p=1.000 n=6
Types 949,556 949,556 ~ ~ ~ p=1.000 n=6
Memory used 2,222,479k (± 0.01%) 2,222,841k (± 0.00%) +362k (+ 0.02%) 2,222,745k 2,222,903k p=0.005 n=6
Parse Time 7.88s (± 0.27%) 7.84s (± 0.51%) ~ 7.79s 7.90s p=0.106 n=6
Bind Time 2.77s (± 1.01%) 2.74s (± 1.30%) ~ 2.67s 2.77s p=0.147 n=6
Check Time 86.36s (± 0.20%) 86.57s (± 0.63%) ~ 85.85s 87.35s p=0.298 n=6
Emit Time 0.17s (± 3.32%) 0.17s (± 2.38%) ~ 0.17s 0.18s p=0.054 n=6
Total Time 97.17s (± 0.15%) 97.31s (± 0.59%) ~ 96.47s 98.05s p=0.471 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,120 1,230,399 +279 (+ 0.02%) ~ ~ p=0.001 n=6
Types 265,641 265,705 +64 (+ 0.02%) ~ ~ p=0.001 n=6
Memory used 2,347,284k (± 0.02%) 2,407,219k (± 6.07%) ~ 2,346,658k 2,705,638k p=0.230 n=6
Parse Time 6.00s (± 0.66%) 5.97s (± 0.62%) ~ 5.93s 6.03s p=0.199 n=6
Bind Time 2.26s (± 0.87%) 2.26s (± 0.54%) ~ 2.24s 2.27s p=0.935 n=6
Check Time 40.77s (± 0.59%) 40.64s (± 0.86%) ~ 40.22s 41.14s p=0.689 n=6
Emit Time 4.00s (± 3.39%) 3.94s (± 1.78%) ~ 3.85s 4.06s p=0.575 n=6
Total Time 53.05s (± 0.52%) 52.82s (± 0.72%) ~ 52.33s 53.31s p=0.378 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,120 1,230,399 +279 (+ 0.02%) ~ ~ p=0.001 n=6
Types 265,641 265,705 +64 (+ 0.02%) ~ ~ p=0.001 n=6
Memory used 2,421,709k (± 0.03%) 2,421,915k (± 0.02%) ~ 2,421,356k 2,422,543k p=0.378 n=6
Parse Time 5.21s (± 0.66%) 5.17s (± 0.70%) ~ 5.12s 5.22s p=0.172 n=6
Bind Time 1.70s (± 0.53%) 1.69s (± 0.95%) ~ 1.66s 1.70s p=0.437 n=6
Check Time 35.20s (± 0.20%) 35.13s (± 0.16%) ~ 35.02s 35.19s p=0.128 n=6
Emit Time 3.35s (± 1.44%) 3.38s (± 0.55%) ~ 3.36s 3.41s p=0.336 n=6
Total Time 45.47s (± 0.10%) 45.38s (± 0.18%) ~ 45.27s 45.49s p=0.128 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,471 258,618 +147 (+ 0.06%) ~ ~ p=0.001 n=6
Types 105,534 105,598 +64 (+ 0.06%) ~ ~ p=0.001 n=6
Memory used 429,126k (± 0.02%) 429,260k (± 0.03%) ~ 429,106k 429,493k p=0.066 n=6
Parse Time 3.36s (± 0.70%) 3.36s (± 0.38%) ~ 3.34s 3.38s p=0.803 n=6
Bind Time 1.33s (± 1.31%) 1.33s (± 0.56%) ~ 1.32s 1.34s p=0.397 n=6
Check Time 18.05s (± 0.35%) 18.00s (± 0.40%) ~ 17.92s 18.10s p=0.172 n=6
Emit Time 1.63s (± 1.31%) 1.65s (± 1.53%) ~ 1.63s 1.70s p=0.139 n=6
Total Time 24.36s (± 0.27%) 24.34s (± 0.28%) ~ 24.25s 24.43s p=0.419 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,931 224,931 ~ ~ ~ p=1.000 n=6
Types 94,146 94,146 ~ ~ ~ p=1.000 n=6
Memory used 370,064k (± 0.03%) 370,053k (± 0.04%) ~ 369,869k 370,226k p=0.936 n=6
Parse Time 2.76s (± 1.07%) 2.75s (± 0.94%) ~ 2.71s 2.78s p=0.568 n=6
Bind Time 1.58s (± 1.09%) 1.57s (± 0.96%) ~ 1.56s 1.60s p=0.563 n=6
Check Time 15.62s (± 0.28%) 15.58s (± 0.35%) ~ 15.49s 15.64s p=0.375 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.95s (± 0.28%) 19.90s (± 0.37%) ~ 19.78s 19.98s p=0.374 n=6
vscode - node (v18.15.0, x64)
Errors 11 11 ~ ~ ~ p=1.000 n=6
Symbols 2,985,943 2,985,943 ~ ~ ~ p=1.000 n=6
Types 1,027,137 1,027,137 ~ ~ ~ p=1.000 n=6
Memory used 3,109,996k (± 0.00%) 3,110,377k (± 0.00%) +381k (+ 0.01%) 3,110,295k 3,110,463k p=0.005 n=6
Parse Time 17.14s (± 0.36%) 17.14s (± 0.36%) ~ 17.07s 17.22s p=1.000 n=6
Bind Time 5.22s (± 0.22%) 5.22s (± 0.57%) ~ 5.18s 5.25s p=0.685 n=6
Check Time 96.39s (± 0.61%) 96.39s (± 0.37%) ~ 96.00s 96.97s p=1.000 n=6
Emit Time 24.94s (± 0.42%) 25.09s (± 0.57%) ~ 24.88s 25.31s p=0.054 n=6
Total Time 143.69s (± 0.48%) 143.85s (± 0.30%) ~ 143.14s 144.45s p=0.936 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,560 267,560 ~ ~ ~ p=1.000 n=6
Types 109,076 109,076 ~ ~ ~ p=1.000 n=6
Memory used 412,394k (± 0.01%) 412,414k (± 0.01%) ~ 412,371k 412,537k p=0.630 n=6
Parse Time 3.17s (± 0.26%) 3.18s (± 0.38%) ~ 3.16s 3.19s p=0.157 n=6
Bind Time 1.43s (± 0.28%) 1.43s (± 0.44%) ~ 1.42s 1.44s p=0.673 n=6
Check Time 14.31s (± 0.35%) 14.34s (± 0.59%) ~ 14.24s 14.46s p=0.378 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.91s (± 0.28%) 18.95s (± 0.47%) ~ 18.82s 19.06s p=0.422 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 511,803 511,803 ~ ~ ~ p=1.000 n=6
Types 162,088 162,088 ~ ~ ~ p=1.000 n=6
Memory used 449,320k (± 0.02%) 449,316k (± 0.07%) ~ 448,714k 449,551k p=0.575 n=6
Parse Time 2.63s (± 0.71%) 2.62s (± 0.52%) ~ 2.61s 2.65s p=0.562 n=6
Bind Time 0.97s (± 1.55%) 0.97s (± 0.65%) ~ 0.96s 0.98s p=0.391 n=6
Check Time 14.56s (± 0.36%) 14.54s (± 0.19%) ~ 14.51s 14.57s p=0.627 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.17s (± 0.32%) 18.13s (± 0.17%) ~ 18.09s 18.16s p=0.418 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,300ms (± 0.19%) 2,297ms (± 0.33%) ~ 2,288ms 2,307ms p=0.228 n=6
Req 2 - geterr 5,152ms (± 0.53%) 5,120ms (± 0.63%) ~ 5,083ms 5,158ms p=0.230 n=6
Req 3 - references 261ms (± 0.96%) 261ms (± 0.52%) ~ 259ms 263ms p=1.000 n=6
Req 4 - navto 228ms (± 0.66%) 227ms (± 0.78%) ~ 225ms 230ms p=0.094 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 80ms (± 4.95%) 89ms (± 5.59%) 🔻+10ms (+11.95%) 79ms 92ms p=0.019 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,882ms (± 0.43%) 2,956ms (± 0.63%) +75ms (+ 2.59%) 2,935ms 2,980ms p=0.005 n=6
Req 2 - geterr 5,553ms (± 8.18%) 5,198ms (±11.56%) ~ 4,641ms 5,783ms p=0.810 n=6
Req 3 - references 410ms (± 0.50%) 414ms (± 0.74%) +4ms (+ 0.94%) 411ms 419ms p=0.043 n=6
Req 4 - navto 338ms (± 4.35%) 340ms (± 0.49%) ~ 338ms 343ms p=0.419 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 101ms (±12.89%) 121ms (± 5.40%) 🔻+20ms (+19.57%) 108ms 125ms p=0.012 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,179ms (± 0.26%) 5,179ms (± 0.24%) ~ 5,168ms 5,201ms p=0.810 n=6
Req 2 - geterr 1,114ms (± 1.09%) 1,128ms (± 1.35%) ~ 1,112ms 1,146ms p=0.092 n=6
Req 3 - references 84ms (± 5.23%) 86ms (± 4.79%) ~ 79ms 92ms p=0.493 n=6
Req 4 - navto 442ms (± 1.44%) 444ms (± 0.34%) ~ 442ms 446ms p=0.869 n=6
Req 5 - completionInfo count 3,417 3,417 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 807ms (± 1.53%) 818ms (± 1.29%) ~ 807ms 830ms p=0.127 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.72ms (± 0.18%) 157.75ms (± 0.18%) ~ 156.69ms 159.67ms p=0.161 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 232.52ms (± 0.14%) 232.52ms (± 0.14%) ~ 230.98ms 239.22ms p=0.941 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 227.00ms (± 0.16%) 227.09ms (± 0.16%) +0.09ms (+ 0.04%) 225.81ms 232.70ms p=0.011 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.98ms (± 0.13%) 226.15ms (± 0.13%) +0.17ms (+ 0.07%) 224.86ms 228.35ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Seems to be consistently 1 percent in Compiler-Unions...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jakebailey @typescript-bot