v0.2.3 (#42)

* Fix typo

* [Git actions] : add linting, type checking and code coverage.

* Rename requirements file

* Update codecov.yml

* Fix Git action for code coverage

* Add citation in the text

* Git actions: using pytest-cov and another black version

* Fix test workflow

* added plot_rsp() to plots, backends, init, test_plots

* liting fix

* First version of the BIDS report module.
- Create individual HTML reports, summary dataframes and physio dataframes.
- Create group-level HTML reports.

* [Git Actions] : Remove unused package from testing.

* Fix linting package error.

* [tests] : close Matplotlib figures after test to avoid memory overload.

* Formatting issue in README

* [reports] : Add command line functionalities.

* [reports] : Add show_raw parameter and set to False as defaults to save memory.

* [reports] : load participants ID automatically if not provided.

* [utils] : Add a nan_cleaning function that can be called before peaks detection (optional).

* [PPG] : Add a find_clipping function to automatically find clipping threshold. This function is called by ppg_peaks when clipping=True and clipping_threshold="auto".
[detection]: ecg_peaks, ppg_peaks and rsp_peaks accept pd.Series as input. Defalt input variable named signal for consistency.

* [notebooks] : Clean imports, fix use of interpolate_clipping and use keyword args.

* Minor API docs.

* [detection] : remove default values for `sfreq` and standardize outputs for the 3 modalities.

* Doctrings and minor change in tests.

* [hrv] Minor fix: use  pd.concat instead of the append method.

* [correction] : Add a correct_ectopic_peaks to correct pseudo-ectopic heart beatd in peaks vector.

* [plots] : plot_raw accept a `decim` parameter to control raw signal downsamplig.

* Use pd.concat

* Use pd.concat

* Move the tables function into the reports sub-module.
Update the reports function to current (non working) version.

* [plots] Improve plot_events and its integration with plot_rr.

* [plots]: fix plot_evoked and allow to pass arguments to seaborn.lineplot more easily.

* [detection]: Add sleepecg peak detector as default for Systole

* [plots]: plot_raws now accept an events_params parameter to be passed to plot_rr.

* Add sleepecg in the requirements

* Drop CI for Python 3.7

* Update pre-commit

* [reports]: Add logs for artefatcs in the subject level function.

* [test] : Fix error in plot_raw

* Fix CI error (black)

* Minor fixes

* [reports] : Add artefacts visualization at the group level.

* [reports] Use Joblib for paralellization.

* [detection] - PPG: change `noise_removal` to moving_average and add a `moving_average_length` parameter to control the window length.
- Fix interpolate_clipping() as it was not handling thresholds correctly if the provided threshold is not the max/min of the time series.

* Fix the test after clipping error.

* [requirements]: include Numpy 1.21

* [reports] : The command line function find the relevant files automatically. The signals are automatically trimmed when starting and ending times are provided. Also start using pathlib instead of strings for paths.

* Typos and changelogs.

* - Fix bug in interpolate_clipping() and add corresponding tests
- Linting

* [detection] : interpolate_clipping - fix the way first and last values are corrected. Look at all values more/less or equal than thr and substract the smallest diff from the time serie.

* [plots] : Minor change in matplotlib defaults

* Update pre-commit config

* Fix formatting and mypy errors

* Add Git action for automated docs deployment (first trial).

* [docs] : Fix small errors in the examples

* [docs] : Drop Psychopy import in the examples

* isort config file

* Update the requirement files.

* [docs] : - Automatic docs construction now working and moved to https://embodied-computation-group.github.io/systole/
- Minor fixes in the docstrings

* Update notebooks

* Remove docs api example from the code base.

* Update docs link

* Remove the findOximeter function #8

* [docs]: link to paper

* CI minor updates

* [plots]: Fix decim function and error in the docstring examples.

* [correction] : Rewrite correct_missed (to correct_missed_rr) and correct_extra (to correct_extra_rr) to better control for RR insertion and deletion. Rewrite correct_rr. All interpolations in a single pass. All function use jit. (see #40).
Update pre-commit hooks.

* [correction] : Add extensive testing suite for correct_extra_rr, correct_missed_rr and correct_rr (#40).

* [correction] : Fix errors in the arrays update for missed and etra peaks. Update the corresponding notebooks. Add entry in changelogs (#40).

* Fix mypy errors

* [correction] : Update the dcumentation for API changes.

* changelogs

* [docs] : Add instruction for using dev branch

* [reports] : Fix defaults paths and add the patterns parameter for more flexibility in the files selection.

* [docs] : Add docs for BIDS folder.

* Update requirements.txt. Fix reports tests.

* [viewer] : Add first version of Systole viewer for manual peaks correction.

* [docs] Myst parser

* [viewer] : Use empty path names by default.

* [viewer] : more docs and small fix

* [docs] : Reorganize folders

* [plots] : plot_subspaces accepts an ax argument.

* Autodocs link

* [test] : Remove notebooks from the test suite now that we have autodocs jobs (for now).

* Links

* Change version to 0.2.3dev

* [reports] : Use the file name from the BIDS folder to save derivatives. This will avoid duplicates in case of different patterns been used.

* [reports] : Automatically retrieve file_name template when creating group level reports.

* [docs] Image link

* [viewer] : Automatically loading and processing data depending on the modality.

* [viewer] : Create method for plotting and add some aesthetic features.

* [viewer] : Fix several errors during import and add more menues in the Viewer.

* [viewer] : Add option to mark the signal or segments of the signal as bads.

* [plots] Prepare merge plot_raw and plot_resp and remove tight layout from the deault.

* [plots] : Error in plot_raw

* [viewer] : Fix save button

* [viewer] : Better output when files are not found.

* [viewer] : Error with the logs of bad segments.

* [plots] : Add possibe `bad_segments` parameter as input for plot_raw and plot_rr.

* [plots] : Merge plot_resp into plot raw. Use modality strings defined at the package level.
[viewer] : allow full file name to be provided.

* [pre-commit] Autoupdate

* Typing

* docstring

* [utils] : Add tools to handle bad segments and use it in the plotting functions.

* [requirements] : last pydata theme

* Add docs.

* Fix bad_segments utills (flake8 error)

* requirements (bibtex)

* Use sleepecg as default R-peaks detector

* [docs]

* [docs]

* [docs] More examples

* [docs] Links

* [docs] plotting examples

* [docs]

* [docs]

* [viewer] : Save the entire corrected peaks. Load corrected peaks and bad segments automatically with signal.

* [viewer] : fix file name for corrected peaks.

* Add import function for manually corrected peaks (#41)

* [docs] Pair noteboks with md files

* dev version

* [viewer] : Fix the way corrected peaks are imported.

* [docs] Citations

* Rename ref.bib

* [reports] : Improve the reports and fix filename error.

* [reports] : Save a JSON and the raw signal in the preprocessed folder.

* [viewer] : Retrieve the list of participants from the folder names.

* [docs] documenting Viewer usages

* [docs] remove citation that is not rendering correctly

* [docs] images links

* version bump

Co-authored-by: alexvasilichi <alexandrina.vasilichi@maxplanckschools.de>
Co-authored-by: Leah banellis <94465218+lbanellis@users.noreply.github.com>

.github/workflows/docs.yml

@@ -0,0 +1,35 @@
+name: Build and Deploy
+
+on:
+  push:
+    branches:
+      - dev
+  pull_request:
+    branches:
+      - master
+permissions:
+  contents: write
+
+jobs:
+  build-and-deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout 🛎️
+        uses: actions/checkout@v3
+
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v1
+        with:
+          python-version: 3.9
+
+      - name: Build
+        run: |
+          pip install .
+          pip install -r requirements-docs.txt
+          sphinx-build -b html docs/source docs/build/html
+
+      - name: Deploy 🚀
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          folder: docs/build/html
+          BRANCH: gh-pages

.github/workflows/linting.yml

@@ -0,0 +1,35 @@
+name: Linting
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.9]
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - uses: actions/cache@v2
+      name: Configure pip caching
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}
+        restore-keys: |
+          ${{ runner.os }}-pip-
+    - name: Install dependencies
+      run: |
+        pip install -r requirements-tests.txt
+        pip install isort flake8 black
+        pip install mypy==0.971
+        pip install types-setuptools
+    - name: Run tests and coverage
+      run: |
+        mypy ./systole/ --ignore-missing-imports
+        flake8 ./systole/
+        black ./systole/
+        isort ./systole/

.github/workflows/test.yml

@@ -1,13 +1,13 @@
 name: Test
 
-on: [push]
+on: [push, pull_request]
 
 jobs:
   test:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.7, 3.8, 3.9]
+        python-version: [3.8]
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}
@@ -23,10 +23,12 @@ jobs:
           ${{ runner.os }}-pip-
     - name: Install dependencies
       run: |
-        pip install -e '.[test]'
-        pip install -r requirements-test.txt
-        pip install ipykernel 
+        pip install -r requirements-tests.txt
+        pip install ipykernel coverage pytest pytest-cov
         python -m ipykernel install --user --name python3
-    - name: Run tests
+        pip install .
+    - name: Run tests and coverage
       run: |
-        pytest
+        pytest --cov=./ --cov-report=xml
+    - name: Upload coverage to Codecov
+      uses: codecov/codecov-action@v3

.gitignore

@@ -1,11 +1,14 @@
 *.pyc
 *-checkpoint.ipynb
+*-checkpoint.md
 mypyreports/
 build/
 dist/
 systole.egg-info/
 htmlcov/
 .coverage
 .vscode/
-source/auto_examples
-source/generated
+docs/source/auto_examples
+docs/source/generated
+docs/source/api
+coverage.xml

.isort.cfg

@@ -1,5 +1,5 @@
 [settings]
-known_third_party = ecgdetectors,matplotlib,numpy,pandas,plotly,psychopy,pytest,requests,scipy,seaborn,serial,sphinx_bootstrap_theme,tqdm
+known_third_party = matplotlib,numpy,pandas,pytest,requests,scipy,seaborn,serial,sphinx_bootstrap_theme,tqdm
 multi_line_output = 3
 include_trailing_comma = True
 force_grid_wrap = 0

.pre-commit-config.yaml

@@ -1,22 +1,22 @@
 repos:
 -   repo: https://github.com/pre-commit/mirrors-isort
-    rev: v5.9.3
+    rev: v5.10.1
     hooks:
     - id: isort
       files: ^systole/
 -   repo: https://github.com/ambv/black
-    rev: 21.12b0
+    rev: 22.6.0
     hooks:
     - id: black
       language_version: python3
       files: ^systole/
--   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.3.0
+-   repo: https://github.com/pycqa/flake8
+    rev: 5.0.4
     hooks:
     - id: flake8
       files: ^systole/
 -   repo: https://github.com/pre-commit/mirrors-mypy
-    rev: 'v0.920'  # Use the sha / tag you want to point at
+    rev: 'v0.971'
     hooks:
     - id: mypy 
       files: ^systole/

CONTRIBUTING.md

@@ -26,7 +26,7 @@ Please verify that your issue is not being currently addressed by other issues o
 
 While issue reporting is valuable, we strongly encourage users who are inclined to do so to submit patches for new or existing issues via pull requests. This is particularly the case for simple fixes, such as typos or tweaks to documentation, which do not require a heavy investment of time and attention.
 
-Contributors are also encouraged to contribute new code to enhance Systole's functionality, also via pull requests. Please consult the [Systole documentation](https://systole-docs.github.io/) to ensure that any new contribution does not strongly overlap with existing functionality.
+Contributors are also encouraged to contribute new code to enhance Systole's functionality, also via pull requests. Please consult the [Systole documentation](https://embodied-computation-group.github.io/systole/#) to ensure that any new contribution does not strongly overlap with existing functionality.
 
 The preferred workflow for contributing to Systole is to fork the [GitHub repository](https://github.com/embodied-computation-group/systole), clone it to your local machine, and develop on a feature branch.
 

MANIFEST.in

@@ -2,3 +2,6 @@
 include README.rst
 include LICENSE
 include requirements.txt
+include systole/reports/subject_level.html
+include systole/reports/group_level.html
+include systole/reports/images/logo.svg

README.rst

@@ -28,7 +28,7 @@
 
 ================
 
-.. image:: https://github.com/embodied-computation-group/systole/blob/dev/source/images/logo.png
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/logo.png
    :align: center
 
 ================
@@ -38,10 +38,18 @@ This includes tools for data epoching, artefact detection, artefact correction,
 variability analyses, circular statistical approaches to analysing cardiac cycles, and synchronising stimulus 
 presentation with different cardiac phases via Psychopy.
 
-The documentation can be found under the following `link <https://systole-docs.github.io/>`_.
+The documentation can be found under the following `link <https://embodied-computation-group.github.io/systole/#>`_.
 
 If you have questions, you can ask them in the `Gitter chat <https://gitter.im/ecg-systole/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge>`_.
 
+How to cite?
+++++++++++++
+
+If you are using **Systole** in a publication we ask you to cite the following paper::
+
+  Legrand et al., (2022). Systole: A python package for cardiac signal synchrony and analysis. Journal of Open Source Software, 7(69), 3832, https://doi.org/10.21105/joss.03832
+
+
 Installation
 ++++++++++++
 
@@ -88,6 +96,8 @@ For an introduction to Systole and cardiac signal analysis, you can refer to the
      - |Colab badge 4|
    * - Instantaneous and evoked heart rate 
      - |Colab badge 5|
+   * - Working with BIDS folders
+     - |Colab badge 6|
 
 .. |Colab badge 1| image:: https://colab.research.google.com/assets/colab-badge.svg
   :target: https://colab.research.google.com/github/embodied-computation-group/systole/blob/dev/source/notebooks/1-PhysiologicalSignals.ipynb
@@ -104,6 +114,9 @@ For an introduction to Systole and cardiac signal analysis, you can refer to the
 .. |Colab badge 5| image:: https://colab.research.google.com/assets/colab-badge.svg
   :target: https://colab.research.google.com/github/embodied-computation-group/systole/blob/dev/source/notebooks/5-InstantaneousHeartRate.ipynb
 
+.. |Colab badge 6| image:: https://colab.research.google.com/assets/colab-badge.svg
+  :target: https://colab.research.google.com/github/embodied-computation-group/systole/blob/dev/source/notebooks/6-WorkingWithBIDSFolders.ipynb
+
 
 Getting started
 +++++++++++++++
@@ -127,7 +140,7 @@ The package integrates a set of functions for interactive or non interactive dat
   plot_raw(signal[60000 : 120000], modality="ecg", backend="bokeh", 
               show_heart_rate=True, show_artefacts=True, figsize=300)
 
-.. image:: https://github.com/embodied-computation-group/systole/blob/dev/source/images/raw.png
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/raw.png
    :align: center
 
 
@@ -145,7 +158,7 @@ Artefacts can be detected and corrected in the RR interval time series or the pe
 
   plot_subspaces(peaks, input_type="peaks", backend="bokeh")
 
-.. image:: https://github.com/embodied-computation-group/systole/blob/dev/source/images/subspaces.png
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/subspaces.png
    :align: center
 
 
@@ -163,7 +176,7 @@ Systole implements time-domain, frequency-domain and non-linear HRV indices, as
       plot_poincare(peaks, input_type="peaks", backend="bokeh", figsize=(200, 200)),
       )
 
-.. image:: https://github.com/embodied-computation-group/systole/blob/dev/source/images/hrv.png
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/hrv.png
    :align: center
 
 
@@ -174,6 +187,34 @@ The package natively supports recording of physiological signals from the follow
 - `Nonin 3012LP Xpod USB pulse oximeter <https://www.nonin.com/products/xpod/>`_ together with the `Nonin 8000SM 'soft-clip' fingertip sensors <https://www.nonin.com/products/8000s/>`_ (USB).
 - Remote Data Access (RDA) via BrainVision Recorder together with `Brain product ExG amplifier <https://www.brainproducts.com/>`_ (Ethernet).
 
+Interactive visualization of BIDS structured datasets
+=====================================================
+
+.. code-block:: python
+
+  from systole.viewer import Viewer
+
+  view = Viewer(
+      input_folder="/BIDS/folder/path/",
+      pattern="task-mytask",
+      modality="beh",
+      signal_type="ECG"
+  )
+
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/editor.gif
+   :align: center
+
+Inserting and removing peaks
+============================
+
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/peaks.gif
+   :align: center
+
+Annotating bad segments
+=======================
+
+.. image:: https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/segments.gif
+   :align: center
 
 Development
 +++++++++++
@@ -182,13 +223,6 @@ This module was created and is maintained by Nicolas Legrand and Micah Allen (EC
 
 This program is provided with NO WARRANTY OF ANY KIND.
 
-Contributors
-++++++++++++
-
-- Jan C. Brammer (jan.c.brammer@gmail.com)
-- Gidon Levakov (gidonlevakov@gmail.com)
-- Peter Doggart (peter.doggart@pulseai.io)
-
 Acknowledgements
 ++++++++++++++++
 
@@ -200,7 +234,7 @@ Systole was largely inspired by pre-existing toolboxes dedicated to heartrate va
 
 * hrv: https://github.com/rhenanbartels/hrv
 
-* pyHVR: https://pyhrv.readthedocs.io/en/latest/index.html
+* pyHRV: https://pyhrv.readthedocs.io/en/latest/index.html
 
 * ECG-detector: https://github.com/berndporr/py-ecg-detectors
 
@@ -212,11 +246,11 @@ Systole was largely inspired by pre-existing toolboxes dedicated to heartrate va
 
 |AU| |lundbeck| |lab|
 
-.. |AU| image::  https://github.com/embodied-computation-group/systole/raw/dev/source/images/au_clinisk_logo.png
+.. |AU| image::  https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/au_clinisk_logo.png
    :width: 100%
 
-.. |lundbeck| image::  https://github.com/embodied-computation-group/systole/raw/dev/source/images/lundbeckfonden_logo.png
+.. |lundbeck| image::  https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/lundbeckfonden_logo.png
    :width: 10%
 
-.. |lab| image::  https://github.com/embodied-computation-group/systole/raw/dev/source/images/LabLogo.png
+.. |lab| image::  https://github.com/embodied-computation-group/systole/blob/dev/docs/source/images/LabLogo.png
    :width: 20%

codecov.yml

@@ -1,21 +1,2 @@
-codecov:
-  token: 11a4b360-4894-4ef4-9f7c-c6d2858453c0
-  require_ci_to_pass: yes
-
-coverage:
-  precision: 2
-  round: down
-  range: "70...100"
-
-  status:
-    project: yes
-    patch: yes
-    changes: no
-
-parsers:
-  gcov:
-    branch_detection:
-      conditional: yes
-      loop: yes
-      method: no
-      macro: no
+ignore:
+  - "./systole/tests/*"