Skip to content

Commit

Permalink
refactor(slow-first): simplify usage + publish workflow
Browse files Browse the repository at this point in the history
  • Loading branch information
joaovitorsilvestre committed Jan 27, 2024
1 parent 5379d0c commit 659fff9
Show file tree
Hide file tree
Showing 8 changed files with 177 additions and 136 deletions.
26 changes: 0 additions & 26 deletions .github/workflows/build-release.yml

This file was deleted.

94 changes: 94 additions & 0 deletions .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: Publish Python 🐍 distribution 📦 to PyPI

on: push

jobs:
build:
name: Build distribution 📦
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.x"

- name: Install pypa/build
run: >-
python3 -m
pip install
build
--user
- name: Build a binary wheel and a source tarball
run: python3 -m build
- name: Store the distribution packages
uses: actions/upload-artifact@v3
with:
name: python-package-distributions
path: dist/

publish-to-pypi:
name: >-
Publish Python 🐍 distribution 📦 to PyPI
if: startsWith(github.ref, 'refs/tags/')
needs:
- build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/pytest-slow-first
permissions:
id-token: write

steps:
- name: Download all the dists
uses: actions/download-artifact@v3
with:
name: python-package-distributions
path: dist/
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

github-release:
name: >-
Sign the Python 🐍 distribution 📦 with Sigstore
and upload them to GitHub Release
needs:
- publish-to-pypi
runs-on: ubuntu-latest

permissions:
contents: write # IMPORTANT: mandatory for making GitHub Releases
id-token: write # IMPORTANT: mandatory for sigstore

steps:
- name: Download all the dists
uses: actions/download-artifact@v3
with:
name: python-package-distributions
path: dist/
- name: Sign the dists with Sigstore
uses: sigstore/gh-action-sigstore-python@v1.2.3
with:
inputs: >-
./dist/*.tar.gz
./dist/*.whl
- name: Create GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
gh release create
'${{ github.ref_name }}'
--repo '${{ github.repository }}'
--notes ""
- name: Upload artifact signatures to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `dist/` contains the built packages, and the
# sigstore-produced signatures and certificates.
run: >-
gh release upload
'${{ github.ref_name }}' dist/**
--repo '${{ github.repository }}'
5 changes: 4 additions & 1 deletion .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions .idea/pytest-slow-first.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 4 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,44 +39,6 @@ In the next time you run it, tests will be sorted by time spend in the last run,
</br>
</hr>

Usage
-----

You just need to define two functions inside your conftest.py file: `slow_first_save_durations` and `slow_first_load_durations`.

The first one is to save results of current run and the second one is to load the same results in the folowing run. Allowing this plugin to sort execuntion of tests based in these results.

Example of `conftest.py` file:
```python
import os, json


def slow_first_load_durations():
if os.path.exists('/tmp/tests_duration'):
with open('/tmp/tests_duration', 'r') as f:
return f.read()
else:
# Durations not found. Run with default order
return None

def slow_first_save_durations(durations_data: str):
with open('/tmp/tests_duration', 'w') as f:
f.write(durations_data)
```

#### Explanation

1. First, `slow_first_load_durations` will be called before your tests starts running, it will load the durantion of the tests
of the previous run.

* **obs**: if its the first time using this plugin or if you can't load the results, this function must return None.

2. If `slow_first_load_durations` finds data, it returns the content and slow-first plugin will sort your tests, otherwise
the test suite will run at default order.

3. If the suit runs with success, `slow_first_save_durations` is going to be called with durations as argument. This function must save the results
in a way that `slow_first_load_durations` can load in the next run.

### Running with pytest-slow-first plugin
Finally, activate the plugin by passing `--slow-first` as paramter of pytest command:

Expand All @@ -87,6 +49,10 @@ pytest tests --slow-first -n3 # using along side xdist
</br>
</hr>

THis plugin will save the duration of each of your tests in a file named `pytest-slow-first.json` in the current directory.
You can change the location by setting the enviroment variable `SLOW_FIRST_PATH` to the path you want.
Ex: `export SLOW_FIRST_PATH=/tmp/pytest-slow-first.json pytest --slow-first`

Installation
------------

Expand Down
104 changes: 56 additions & 48 deletions pytest_slow_first.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@
import importlib
import json
import logging
import os
from typing import Union

import pytest
from _pytest.config import ExitCode

FORMAT_VERSION = '1.0.0'
SLOW_FIRST_PATH = os.environ.get('SLOW_FIRST_PATH', 'pytest-slow-first.json')


def log_slow_first(message: str):
logging.getLogger().info(f"[pytest-slow-first] {message}")


class SlowFirstRequiredFunctionNotImplemented(Exception):
Expand All @@ -17,60 +29,77 @@ def _get_slow_first_from_config(config):


class Test:
def __init__(self, name, setup_duration: float = None, call_duration: float = None, teardown_duration: float = None):
self.name = name
nodeid: str
setup_duration: float | None
call_duration: float | None
teardown_duration: float | None

def __init__(self, nodeid: str, setup_duration: float = None, call_duration: float = None, teardown_duration: float = None):
self.nodeid = nodeid
self.setup_duration = setup_duration
self.call_duration = call_duration
self.teardown_duration = teardown_duration

@property
def total_duration(self):
def total_duration(self) -> float:
return self.setup_duration + self.call_duration + self.teardown_duration

def set_duration(self, kind: str, duration: float):
setattr(self, f"{kind}_duration", duration)

def serialize(self):
return {'name': self.name, 'setup_duration': self.setup_duration,
def serialize(self) -> dict[str, Union[str, float]]:
return {'nodeid': self.nodeid, 'setup_duration': self.setup_duration,
'call_duration': self.call_duration, 'teardown_duration': self.teardown_duration}

@staticmethod
def deserialize(data: dict):
test = Test(data['name'])
test.setup_duration = data['setup_duration']
test.call_duration = data['call_duration']
test.teardown_duration = data['teardown_duration']
return test
def deserialize(data: dict) -> "Test":
return Test(**data)


class SlowFirst:
def __init__(self, tests_by_name: dict = None, enabled: bool = False):
_tests_by_name: dict[str, Test]
enabled: bool

def __init__(self, tests_by_name: dict[str, Test] = None, enabled: bool = False):
self._tests_by_name = tests_by_name or {}
self.enabled = enabled

if enabled:
self._validate_conftest_functions_are_defined()

def save(self):
logging.getLogger().info("Saving testes durations")
self._get_conftest_module().slow_first_save_durations(self.serialize())
log_slow_first("Saving testes durations")
with open(SLOW_FIRST_PATH, 'w') as f:
f.write(self.serialize())

@staticmethod
def load():
logging.getLogger().info("Loading testes durations of previous run")
data = SlowFirst._get_conftest_module().slow_first_load_durations()

if not data:
if os.path.exists(SLOW_FIRST_PATH):
with open(SLOW_FIRST_PATH, 'r') as f:
data = f.read()
log_slow_first("Loaded testes durations from previous run. Applying order")
else:
log_slow_first("No previous run found. Skipping order")
return None

return SlowFirst.deserialize(data)

def serialize(self) -> str:
return json.dumps(list(map(Test.serialize, self._tests_by_name.values())))
return json.dumps({
'format_version': FORMAT_VERSION,
'tests': [test.serialize() for test in self._tests_by_name.values()]
})

@staticmethod
def deserialize(data: str):
return SlowFirst({test.name: test for test in map(Test.deserialize, json.loads(data))})
def deserialize(data: str) -> "SlowFirst":
data = json.loads(data)

if data['format_version'] != FORMAT_VERSION:
pytest.exit(
reason=f"[pytest-slow-first] The format version of {SLOW_FIRST_PATH} "
f"is not compatible with this version of pytest-slow-first. "
f"Please delete {SLOW_FIRST_PATH} and run tests again.",
returncode=ExitCode.USAGE_ERROR
)

return SlowFirst({test.nodeid: test for test in map(Test.deserialize, data['tests'])})

def set_duration(self, name: str, kind: str, duration: float):
test = self._tests_by_name.get(name)
Expand All @@ -79,37 +108,16 @@ def set_duration(self, name: str, kind: str, duration: float):
test.set_duration(kind, duration)
else:
test = Test(name)
self._tests_by_name[test.name] = test
self._tests_by_name[test.nodeid] = test
test.set_duration(kind, duration)

def get_order(self, test_name: str):
def get_order(self, test_name: str) -> float:
test = self._tests_by_name.get(test_name)
if test:
return test.total_duration
else:
return 0

@staticmethod
def _get_conftest_module():
return importlib.import_module('conftest')

@staticmethod
def _validate_conftest_functions_are_defined():
try:
conftest = SlowFirst._get_conftest_module()
except ModuleNotFoundError as e:
raise ConftestModuleNotFound('slow_first plugin was not able to load conftest module') from e

if not hasattr(conftest, 'slow_first_save_durations'):
raise SlowFirstRequiredFunctionNotImplemented(
'slow_first_save_durations function is not defined in conftest.py'
)

if not hasattr(conftest, 'slow_first_load_durations'):
raise SlowFirstRequiredFunctionNotImplemented(
'slow_first_load_durations function is not defined in conftest.py'
)


def pytest_addoption(parser):
group = parser.getgroup('slow-first')
Expand All @@ -118,7 +126,7 @@ def pytest_addoption(parser):
action='store_true',
dest='slow_first',
default=False,
help='If given, will enable tests sorting by durations from last run.'
help='Sort tests from slowest to fastest.'
)


Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ def read(fname):

setup(
name='pytest-slow-first',
version='0.1.0',
version='1.0.0',
author='João Vitor Silvestre',
author_email='joao_vitor_silvestre@outlook.com',
maintainer='João Vitor Silvestre',
Expand Down
Loading

0 comments on commit 659fff9

Please sign in to comment.