Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PGO builds #463

Merged
merged 10 commits into from
Dec 15, 2024
Merged

Fix PGO builds #463

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
6aaa622
Drop custom allocators
gi0baro Dec 13, 2024
1d4261e
Avoid package mode in `uv`
gi0baro Dec 13, 2024
ec43291
Try to avoid `aws-lc` dep
gi0baro Dec 13, 2024
399f8cd
temp: only PGO builds
gi0baro Dec 13, 2024
2f7e5a7
temp: linux pgo builds back to `setup-python` action
gi0baro Dec 13, 2024
7ac20de
temp: just upload profiled wheels
gi0baro Dec 13, 2024
0b3eeb0
Test `profile-arcs` flag
gi0baro Dec 14, 2024
95b8140
test
gi0baro Dec 15, 2024
adc742d
Disable PGO builds on Linux
gi0baro Dec 15, 2024
318e739
Bump deps
gi0baro Dec 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 18 additions & 15 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,12 @@ jobs:
include:
- os: ubuntu
platform: linux
- os: ubuntu
platform: linux
target: x86_64
manylinux: auto
interpreter: pypy3.9 pypy3.10
# FIXME: for some unknown reasons PGO builds are failing on Linux
# - os: ubuntu
# platform: linux
# target: x86_64
# manylinux: auto
# interpreter: pypy3.9 pypy3.10
- os: macos
target: x86_64
interpreter: pypy3.9 pypy3.10
Expand Down Expand Up @@ -50,10 +51,11 @@ jobs:
runs-on: ${{ matrix.os }}-latest
steps:
- uses: actions/checkout@v4
- name: Set jemalloc for aarch64 Linux
if: matrix.target == 'aarch64' && matrix.os == 'ubuntu'
run: |
echo "JEMALLOC_SYS_WITH_LG_PAGE=16" >> $GITHUB_ENV
# NOTE: we don't use jemalloc anymore
# - name: Set jemalloc for aarch64 Linux
# if: matrix.target == 'aarch64' && matrix.os == 'ubuntu'
# run: |
# echo "JEMALLOC_SYS_WITH_LG_PAGE=16" >> $GITHUB_ENV
- uses: pyo3/maturin-action@v1
with:
rust-toolchain: stable
Expand All @@ -74,12 +76,14 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-13, macos-14, windows-latest]
# FIXME: for some unknown reasons PGO builds are failing on Linux
# os: [ubuntu-latest, macos-13, macos-14, windows-latest]
os: [macos-13, macos-14, windows-latest]
manylinux: [auto]
interpreter: ["3.9", "3.10", "3.11", "3.12", "3.13"]
include:
- os: ubuntu-latest
platform: linux
# - os: ubuntu-latest
# platform: linux
- os: windows-latest
platform: windows

Expand All @@ -94,8 +98,6 @@ jobs:
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools
- shell: bash
run: uv python install ${{ env.UV_PYTHON }}
- name: prepare profiling directory
shell: bash
run: mkdir -p ${{ github.workspace }}/profdata
Expand All @@ -115,10 +117,11 @@ jobs:
- name: Generate PGO data
shell: bash
run: |
uv python install ${{ env.UV_PYTHON }}
uv venv .venv
uv pip install -r .github/workflows/_pgo_deps.txt
uv pip install granian --no-index --no-deps --find-links pgo_wheel --force-reinstall
PGO_RUN=y LLVM_PROFILE_FILE=${{ github.workspace }}/profdata/%m_%p.profraw uv run --no-sync pytest tests
PGO_RUN=y LLVM_PROFILE_FILE=${{ github.workspace }}/profdata/granian_%m_%p.profraw uv run --no-sync pytest tests
- name: merge PGO data
run: ${{ env.LLVM_PROFDATA }} merge --failure-mode=all -o ${{ github.workspace }}/merged.profdata ${{ github.workspace }}/profdata
- name: Build PGO wheel
Expand Down
45 changes: 26 additions & 19 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,12 @@ jobs:
include:
- os: ubuntu
platform: linux
- os: ubuntu
platform: linux
target: x86_64
manylinux: auto
interpreter: pypy3.9 pypy3.10
# FIXME: for some unknown reasons PGO builds are failing on Linux
# - os: ubuntu
# platform: linux
# target: x86_64
# manylinux: auto
# interpreter: pypy3.9 pypy3.10
- os: macos
target: x86_64
interpreter: pypy3.9 pypy3.10
Expand Down Expand Up @@ -73,10 +74,11 @@ jobs:
runs-on: ${{ matrix.os }}-latest
steps:
- uses: actions/checkout@v4
- name: Set jemalloc for aarch64 Linux
if: matrix.target == 'aarch64' && matrix.os == 'ubuntu'
run: |
echo "JEMALLOC_SYS_WITH_LG_PAGE=16" >> $GITHUB_ENV
# NOTE: we don't use jemalloc anymore
# - name: Set jemalloc for aarch64 Linux
# if: matrix.target == 'aarch64' && matrix.os == 'ubuntu'
# run: |
# echo "JEMALLOC_SYS_WITH_LG_PAGE=16" >> $GITHUB_ENV
- uses: pyo3/maturin-action@v1
with:
rust-toolchain: stable
Expand All @@ -97,22 +99,25 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-13, macos-14, windows-latest]
# FIXME: for some unknown reasons PGO builds are failing on Linux
# os: [ubuntu-latest, macos-13, macos-14, windows-latest]
os: [macos-13, macos-14, windows-latest]
manylinux: [auto]
interpreter: ["3.9", "3.10", "3.11", "3.12", "3.13"]
include:
- os: ubuntu-latest
platform: linux
# - os: ubuntu-latest
# platform: linux
- os: windows-latest
platform: windows

runs-on: ${{ matrix.os }}
env:
UV_PYTHON: ${{ matrix.interpreter }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: astral-sh/setup-uv@v4
with:
python-version: ${{ matrix.interpreter }}
allow-prereleases: true
enable-cache: false
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools
Expand All @@ -135,11 +140,13 @@ jobs:
- name: Generate PGO data
shell: bash
run: |
pip install -r .github/workflows/_pgo_deps.txt
pip install granian --no-index --no-deps --find-links pgo_wheel --force-reinstall
PGO_RUN=y pytest tests
uv python install ${{ env.UV_PYTHON }}
uv venv .venv
uv pip install -r .github/workflows/_pgo_deps.txt
uv pip install granian --no-index --no-deps --find-links pgo_wheel --force-reinstall
PGO_RUN=y LLVM_PROFILE_FILE=${{ github.workspace }}/profdata/granian_%m_%p.profraw uv run --no-sync pytest tests
- name: merge PGO data
run: ${{ env.LLVM_PROFDATA }} merge -o ${{ github.workspace }}/merged.profdata ${{ github.workspace }}/profdata
run: ${{ env.LLVM_PROFDATA }} merge --failure-mode=all -o ${{ github.workspace }}/merged.profdata ${{ github.workspace }}/profdata
- name: Build PGO wheel
uses: PyO3/maturin-action@v1
with:
Expand Down
Loading
Loading