Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Remove some things from env #2336

Merged
merged 3 commits into from
Jan 13, 2025
Merged

Conversation

larseggert
Copy link
Collaborator

  1. CARGO_TERM_COLOR: always is always set by dtolnay/rust-toolchain
  2. RUST_BACKTRACE is better set where needed, rather than everywhere.

1. `CARGO_TERM_COLOR: always` is always set by `dtolnay/rust-toolchain`
2. `RUST_BACKTRACE` is better set where needed, rather than everywhere.
Copy link

github-actions bot commented Jan 9, 2025

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to e5770ce.

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

github-actions bot commented Jan 9, 2025

Benchmark results

Performance differences relative to 6013bde.

decode 4096 bytes, mask ff: No change in performance detected.
       time:   [11.149 µs 11.180 µs 11.217 µs]
       change: [-0.2893% +0.0247% +0.3272%] (p = 0.89 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
3 (3.00%) low severe
2 (2.00%) low mild
2 (2.00%) high mild
6 (6.00%) high severe

decode 1048576 bytes, mask ff: No change in performance detected.
       time:   [3.0189 ms 3.0286 ms 3.0399 ms]
       change: [-0.4900% +0.0011% +0.4909%] (p = 1.00 > 0.05)

Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) high mild
10 (10.00%) high severe

decode 4096 bytes, mask 7f: No change in performance detected.
       time:   [19.504 µs 19.562 µs 19.643 µs]
       change: [-0.9240% +0.5659% +2.9583%] (p = 0.73 > 0.05)

Found 20 outliers among 100 measurements (20.00%)
1 (1.00%) low severe
5 (5.00%) low mild
3 (3.00%) high mild
11 (11.00%) high severe

decode 1048576 bytes, mask 7f: No change in performance detected.
       time:   [5.1615 ms 5.1731 ms 5.1862 ms]
       change: [-0.4132% -0.0588% +0.2998%] (p = 0.75 > 0.05)

Found 14 outliers among 100 measurements (14.00%)
14 (14.00%) high severe

decode 4096 bytes, mask 3f: No change in performance detected.
       time:   [5.5407 µs 5.5689 µs 5.6037 µs]
       change: [-0.6680% +0.1885% +1.0564%] (p = 0.68 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
5 (5.00%) low mild
3 (3.00%) high mild
7 (7.00%) high severe

decode 1048576 bytes, mask 3f: Change within noise threshold.
       time:   [1.7581 ms 1.7583 ms 1.7584 ms]
       change: [-0.7802% -0.3960% -0.0837%] (p = 0.01 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe

coalesce_acked_from_zero 1+1 entries: Change within noise threshold.
       time:   [99.436 ns 99.813 ns 100.20 ns]
       change: [+0.3418% +0.8125% +1.3228%] (p = 0.00 < 0.05)

Found 15 outliers among 100 measurements (15.00%)
10 (10.00%) high mild
5 (5.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [117.54 ns 117.93 ns 118.33 ns]
       change: [-0.2896% +0.5661% +1.1963%] (p = 0.14 > 0.05)

Found 19 outliers among 100 measurements (19.00%)
1 (1.00%) low severe
3 (3.00%) low mild
2 (2.00%) high mild
13 (13.00%) high severe

coalesce_acked_from_zero 10+1 entries: Change within noise threshold.
       time:   [116.95 ns 117.30 ns 117.76 ns]
       change: [-0.0026% +0.6181% +1.1692%] (p = 0.03 < 0.05)

Found 13 outliers among 100 measurements (13.00%)
4 (4.00%) low severe
3 (3.00%) low mild
1 (1.00%) high mild
5 (5.00%) high severe

coalesce_acked_from_zero 1000+1 entries: Change within noise threshold.
       time:   [98.487 ns 98.668 ns 98.872 ns]
       change: [+0.0087% +1.3208% +2.7561%] (p = 0.05 < 0.05)

Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) high mild
7 (7.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [111.39 ms 111.45 ms 111.50 ms]
       change: [+0.1081% +0.3117% +0.4404%] (p = 0.00 < 0.05)

Found 18 outliers among 100 measurements (18.00%)
1 (1.00%) low severe
6 (6.00%) low mild
10 (10.00%) high mild
1 (1.00%) high severe

SentPackets::take_ranges: No change in performance detected.
       time:   [5.5836 µs 5.8242 µs 6.1091 µs]
       change: [+0.2536% +6.0290% +14.469%] (p = 0.10 > 0.05)

Found 11 outliers among 100 measurements (11.00%)
6 (6.00%) high mild
5 (5.00%) high severe

transfer/pacing-false/varying-seeds: 💔 Performance has regressed.
       time:   [43.620 ms 43.712 ms 43.811 ms]
       change: [+4.4422% +4.7713% +5.0971%] (p = 0.00 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe

transfer/pacing-true/varying-seeds: 💔 Performance has regressed.
       time:   [44.111 ms 44.191 ms 44.274 ms]
       change: [+4.9164% +5.1703% +5.4247%] (p = 0.00 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

transfer/pacing-false/same-seed: 💔 Performance has regressed.
       time:   [43.902 ms 43.975 ms 44.054 ms]
       change: [+4.8947% +5.1570% +5.4164%] (p = 0.00 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe

transfer/pacing-true/same-seed: 💔 Performance has regressed.
       time:   [43.904 ms 43.958 ms 44.013 ms]
       change: [+4.1757% +4.4095% +4.6309%] (p = 0.00 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) low mild
2 (2.00%) high mild

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected.
       time:   [881.00 ms 890.50 ms 900.31 ms]
       thrpt:  [111.07 MiB/s 112.30 MiB/s 113.51 MiB/s]
change:
       time:   [-1.1351% +0.4059% +1.9583%] (p = 0.60 > 0.05)
       thrpt:  [-1.9207% -0.4042% +1.1481%]

Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild

1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [300.40 ms 302.50 ms 304.63 ms]
       thrpt:  [32.827 Kelem/s 33.058 Kelem/s 33.288 Kelem/s]
change:
       time:   [-0.9306% +0.0414% +1.0571%] (p = 0.94 > 0.05)
       thrpt:  [-1.0460% -0.0414% +0.9393%]

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.
       time:   [34.121 ms 34.285 ms 34.469 ms]
       thrpt:  [29.011  elem/s 29.167  elem/s 29.307  elem/s]
change:
       time:   [-0.8044% -0.0585% +0.7298%] (p = 0.88 > 0.05)
       thrpt:  [-0.7245% +0.0585% +0.8109%]

Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) low mild
4 (4.00%) high severe

1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client: No change in performance detected.
       time:   [1.6193 s 1.6362 s 1.6533 s]
       thrpt:  [60.485 MiB/s 61.119 MiB/s 61.755 MiB/s]
change:
       time:   [-2.9185% -1.3939% +0.1244%] (p = 0.08 > 0.05)
       thrpt:  [-0.1243% +1.4136% +3.0062%]

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing MTU Mean [ms] Min [ms] Max [ms]
gquiche gquiche 1504 573.2 ± 81.4 510.9 724.8
neqo gquiche reno on 1504 815.2 ± 89.3 744.6 972.3
neqo gquiche reno 1504 788.5 ± 91.5 729.8 987.8
neqo gquiche cubic on 1504 758.7 ± 23.3 713.7 807.7
neqo gquiche cubic 1504 783.7 ± 67.3 741.1 952.3
msquic msquic 1504 184.7 ± 94.7 102.4 331.2
neqo msquic reno on 1504 274.8 ± 74.9 207.0 394.8
neqo msquic reno 1504 253.6 ± 81.0 208.9 477.6
neqo msquic cubic on 1504 278.7 ± 79.3 204.7 422.9
neqo msquic cubic 1504 283.3 ± 106.4 206.7 509.5
gquiche neqo reno on 1504 678.1 ± 89.5 543.1 814.7
gquiche neqo reno 1504 717.3 ± 126.5 573.6 956.2
gquiche neqo cubic on 1504 710.9 ± 87.6 577.4 834.1
gquiche neqo cubic 1504 727.6 ± 151.0 567.9 1073.1
msquic neqo reno on 1504 490.3 ± 31.3 445.7 570.4
msquic neqo reno 1504 499.6 ± 55.5 469.3 656.4
msquic neqo cubic on 1504 494.4 ± 40.9 466.6 608.0
msquic neqo cubic 1504 491.0 ± 34.2 465.5 582.6
neqo neqo reno on 1504 536.5 ± 46.3 474.4 613.9
neqo neqo reno 1504 567.0 ± 78.6 494.4 727.7
neqo neqo cubic on 1504 628.4 ± 173.5 528.2 1081.4
neqo neqo cubic 1504 564.2 ± 41.5 511.8 663.4

⬇️ Download logs

Copy link

codecov bot commented Jan 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.33%. Comparing base (db0a332) to head (d801248).
Report is 22 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2336      +/-   ##
==========================================
- Coverage   93.34%   93.33%   -0.02%     
==========================================
  Files         114      114              
  Lines       36908    36897      -11     
  Branches    36908    36897      -11     
==========================================
- Hits        34451    34436      -15     
- Misses       1675     1681       +6     
+ Partials      782      780       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@larseggert larseggert enabled auto-merge January 9, 2025 17:35
@larseggert larseggert requested a review from mxinden January 10, 2025 13:00
@larseggert larseggert added this pull request to the merge queue Jan 13, 2025
Merged via the queue into mozilla:main with commit a2c52f1 Jan 13, 2025
61 of 64 checks passed
@larseggert larseggert deleted the ci-thinner-env branch January 13, 2025 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants