-
Notifications
You must be signed in to change notification settings - Fork 474
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
37 changed files
with
639 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
name: docs_pages_workflow | ||
|
||
on: [pull_request] | ||
|
||
permissions: | ||
pull-requests: write | ||
|
||
jobs: | ||
build_docs_job: | ||
runs-on: ubuntu-latest | ||
env: | ||
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v3 | ||
with: | ||
python-version: 3.8 | ||
|
||
- name: Get pip cache dir | ||
id: pip-cache | ||
run: | | ||
python -m pip install --upgrade pip | ||
echo "dir={$(pip cache dir)}" >> $GITHUB_OUTPUT | ||
- name: pip cache | ||
uses: actions/cache@v3 | ||
with: | ||
path: ${{ steps.pip-cache.outputs.dir }} | ||
key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py', '**/requirements.txt', '**/docs/requirements.txt') }} | ||
restore-keys: | | ||
${{ runner.os }}-pip- | ||
- name: Install docs requirements | ||
run: | | ||
python -m pip install -r docs/requirements.txt | ||
- name: make the sphinx docs | ||
run: | | ||
make -C docs clean | ||
make -C docs html | ||
- uses: readthedocs/actions/preview@v1 | ||
with: | ||
project-slug: "trlx" | ||
project-language: "en" | ||
# see: https://github.com/readthedocs/actions/tree/main/preview | ||
# message-template (optional): Text message to be injected by the action in the Pull Request description. It supports the following placeholders to be replaced: | ||
# {docs-pr-index-url}: URL to the root of the documentation for the Pull Request preview. | ||
# platform (optional): Read the Docs Community (community) or Read the Docs for Business (business). (default: community) | ||
# single-version (optional): Set this to 'true' if your project is single version, so we can link to the correct URL. (default: 'false') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Change log | ||
|
||
Best viewed on [trlx.readthedocs.io](https://trlx.readthedocs.io/en/latest/changelog.html). | ||
|
||
<!-- | ||
Remember to align the itemized text with the first line of an item within a list. | ||
--> | ||
|
||
## trlx 0.4.0 (2022-12-05) | ||
|
||
- python 3.8 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
gendoc: | ||
docker build -t trlxgendocs -f docker/docs/Dockerfile . | ||
run: | ||
docker run --rm -it -v ${PWD}:/build \ | ||
--entrypoint /bin/bash \ | ||
trlxgendocs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
rouge |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
FROM python:3.8-slim | ||
|
||
# pip install -r docs/requirements.txt | ||
# sphinx-build -b html docs docs/build/html -j auto | ||
# sphinx-build -b html -D nb_execution_mode=off docs docs/build/html -j auto | ||
|
||
RUN python -m pip install --upgrade --no-cache-dir pip | ||
ADD docs/requirements.txt /tmp/requirements.txt | ||
RUN python -m pip install --exists-action=w --no-cache-dir -r /tmp/requirements.txt | ||
RUN mkdir /build | ||
WORKDIR /build/docs/ | ||
ENTRYPOINT /build/docs/build.sh | ||
# RUN `which sphinx-build` -T -E -b html -d _build/doctrees-readthedocs -D language=en . _build/html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# How To build the documentation | ||
|
||
```bash | ||
make -c docs html | ||
``` |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
@import url("theme.css"); | ||
|
||
:root { | ||
--block-bg-opacity: .5; | ||
} | ||
|
||
.wy-side-nav-search { | ||
background-color: #fff; | ||
} | ||
|
||
.getting-started { | ||
background-color: rgba(78, 150, 253, var(--block-bg-opacity)); | ||
} | ||
|
||
.user-guides { | ||
background-color: rgba(0, 169, 154, var(--block-bg-opacity)); | ||
} | ||
|
||
.developer-docs { | ||
background-color: rgba(171, 0, 182, var(--block-bg-opacity)); | ||
} | ||
|
||
.key-ideas | ||
{ | ||
border: 0px | ||
} |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
{% extends "!layout.html" %} | ||
{% set css_files = css_files + ["_static/style.css"] %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
#!/bin/bash | ||
|
||
`which sphinx-build` -T -E -b html -d _build/doctrees-readthedocs -D language=en . _build/html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
```{include} ../CHANGELOG.md | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,187 @@ | ||
# Configuration file for the Sphinx documentation builder. | ||
# | ||
# This file only contains a selection of the most common options. For a full | ||
# list see the documentation: | ||
# https://www.sphinx-doc.org/en/master/usage/configuration.html | ||
|
||
# -- Path setup -------------------------------------------------------------- | ||
|
||
# If extensions (or modules to document with autodoc) are in another directory, | ||
# add these directories to sys.path here. If the directory is relative to the | ||
# documentation root, use os.path.abspath to make it absolute, like shown here. | ||
# | ||
import os | ||
import sys | ||
|
||
sys.path.insert(0, os.path.abspath("..")) | ||
|
||
|
||
# -- Project information ----------------------------------------------------- | ||
|
||
project = "trlX" | ||
copyright = "2023, CarperAI" | ||
author = "CarperAI" | ||
|
||
# -- General configuration --------------------------------------------------- | ||
|
||
# Add any Sphinx extension module names here, as strings. They can be | ||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom | ||
# ones. | ||
|
||
extensions = [ | ||
"sphinx.ext.autodoc", | ||
"sphinx.ext.autosummary", | ||
"sphinx.ext.intersphinx", | ||
"sphinx.ext.mathjax", | ||
"sphinx.ext.napoleon", | ||
"sphinx.ext.viewcode", | ||
"matplotlib.sphinxext.plot_directive", | ||
"sphinx_autodoc_typehints", | ||
"myst_nb", | ||
# "myst_parser", | ||
"sphinx_remove_toctrees", | ||
"sphinx_copybutton", | ||
"sphinx_design", | ||
] | ||
|
||
intersphinx_mapping = { | ||
"python": ("https://docs.python.org/3/", None), | ||
"numpy": ("https://docs.scipy.org/doc/numpy/", None), | ||
"scipy": ("https://docs.scipy.org/doc/scipy/reference/", None), | ||
"pytorch": ("https://pytorch.readthedocs.io/", None), | ||
} | ||
|
||
autodoc_preserve_defaults = True | ||
|
||
# Add any paths that contain templates here, relative to this directory. | ||
templates_path = ["_templates"] | ||
|
||
# The suffix(es) of source filenames. | ||
# You can specify multiple suffix as a list of string: | ||
# | ||
source_suffix = [".rst", ".md"] | ||
|
||
# The master toctree document. | ||
main_doc = "index" | ||
|
||
# List of patterns, relative to source directory, that match files and | ||
# directories to ignore when looking for source files. | ||
# This pattern also affects html_static_path and html_extra_path. | ||
exclude_patterns = [ | ||
# Sometimes sphinx reads its own outputs as inputs! | ||
"build/html", | ||
] | ||
|
||
# The name of the Pygments (syntax highlighting) style to use. | ||
pygments_style = None | ||
|
||
autosummary_generate = True | ||
napolean_use_rtype = False | ||
|
||
# -- Options for nbsphinx ----------------------------------------------------- | ||
|
||
# Execute notebooks before conversion: 'always', 'never', 'auto' (default) | ||
# We execute all notebooks, exclude the slow ones using 'exclude_patterns' | ||
nbsphinx_execute = "always" | ||
|
||
# Use this kernel instead of the one stored in the notebook metadata: | ||
# nbsphinx_kernel_name = 'python3' | ||
|
||
# List of arguments to be passed to the kernel that executes the notebooks: | ||
# nbsphinx_execute_arguments = [] | ||
|
||
# If True, the build process is continued even if an exception occurs: | ||
# nbsphinx_allow_errors = True | ||
|
||
|
||
# Controls when a cell will time out (defaults to 30; use -1 for no timeout): | ||
nbsphinx_timeout = 180 | ||
|
||
# Default Pygments lexer for syntax highlighting in code cells: | ||
# nbsphinx_codecell_lexer = 'ipython3' | ||
|
||
# Width of input/output prompts used in CSS: | ||
# nbsphinx_prompt_width = '8ex' | ||
|
||
# If window is narrower than this, input/output prompts are on separate lines: | ||
# nbsphinx_responsive_width = '700px' | ||
|
||
# This is processed by Jinja2 and inserted before each notebook | ||
nbsphinx_prolog = r""" # noqa: E501 | ||
{% set docname = 'docs/' + env.doc2path(env.docname, base=None) %} | ||
.. only:: html | ||
.. role:: raw-html(raw) | ||
:format: html | ||
.. nbinfo:: | ||
Interactive online version: | ||
:raw-html:`<a href="https://colab.research.google.com/github/CarperAI/trlx/blob/master/{{ docname }}"><img alt="Open In Colab" src="https://colab.research.google.com/assets/colab-badge.svg" style="vertical-align:text-bottom"></a>` | ||
__ https://github.com/CarperAI/trlx/blob/ | ||
{{ env.config.release }}/{{ docname }} | ||
""" | ||
|
||
# This is processed by Jinja2 and inserted after each notebook | ||
# nbsphinx_epilog = r""" | ||
# """ | ||
|
||
# Input prompt for code cells. "%s" is replaced by the execution count. | ||
# nbsphinx_input_prompt = 'In [%s]:' | ||
|
||
# Output prompt for code cells. "%s" is replaced by the execution count. | ||
# nbsphinx_output_prompt = 'Out[%s]:' | ||
|
||
# Specify conversion functions for custom notebook formats: | ||
# import jupytext | ||
# nbsphinx_custom_formats = { | ||
# '.Rmd': lambda s: jupytext.reads(s, '.Rmd'), | ||
# } | ||
|
||
# Link or path to require.js, set to empty string to disable | ||
# nbsphinx_requirejs_path = '' | ||
|
||
# Options for loading require.js | ||
# nbsphinx_requirejs_options = {'async': 'async'} | ||
|
||
# mathjax_config = { | ||
# 'TeX': {'equationNumbers': {'autoNumber': 'AMS', 'useLabelIds': True}}, | ||
# } | ||
|
||
# Additional files needed for generating LaTeX/PDF output: | ||
# latex_additional_files = ['references.bib'] | ||
|
||
|
||
# -- Options for HTML output ------------------------------------------------- | ||
|
||
# The theme to use for HTML and HTML Help pages. See the documentation for | ||
# a list of builtin themes. | ||
# | ||
html_theme = "sphinx_book_theme" | ||
|
||
# Add any paths that contain custom static files (such as style sheets) here, | ||
# relative to this directory. They are copied after the builtin static files, | ||
# so a file named "default.css" will overwrite the builtin "default.css". | ||
html_static_path = ["_static"] | ||
|
||
# Output file base name for HTML help builder. | ||
htmlhelp_basename = "TRLXdoc" | ||
|
||
# -- Extension configuration ------------------------------------------------- | ||
|
||
# Tell sphinx-autodoc-typehints to generate stub parameter annotations including | ||
# types, even if the parameters aren't explicitly documented. | ||
always_document_param_types = True | ||
|
||
# Theme options are theme-specific and customize the look and feel of a theme | ||
# further. For a list of options available for each theme, see the | ||
# documentation. | ||
html_theme_options = { | ||
# "logo_only": True, | ||
"show_toc_level": 2, | ||
"repository_url": "https://github.com/CarperAI/trlx", | ||
"use_repository_button": True, # add a "link to repository" button | ||
} | ||
|
||
# The name of an image file (relative to this directory) to place at the top | ||
# of the sidebar. | ||
html_logo = "_static/apple-touch-icon-144x144.png" | ||
|
||
html_favicon = "_static/favicon-16x16.png" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
.. _configs: | ||
|
||
Configs | ||
************************ | ||
|
||
Training a model in TRL will require you to set several configs: | ||
ModelConfig, which contains general info on the model being trained. TrainConfig, which contains things like | ||
training hyperparameters. And finally, MethodConfig, which contains hyperparameters or settings for | ||
the specific method being used (i.e. ILQL or PPO) | ||
|
||
|
||
**General** | ||
|
||
.. autoclass:: trlx.data.configs.TRLConfig | ||
:members: | ||
|
||
.. autoclass:: trlx.data.configs.ModelConfig | ||
:members: | ||
|
||
.. autoclass:: trlx.data.configs.TrainConfig | ||
:members: | ||
|
||
.. autoclass:: trlx.data.method_configs.MethodConfig | ||
:members: | ||
|
||
**PPO** | ||
|
||
.. autoclass:: trlx.trainer.nn.ppo_models.MethodConfig | ||
:members: | ||
|
||
**ILQL** | ||
|
||
.. autoclass:: trlx.trainer.nn.ilql_models.ILQLConfig | ||
:members: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
.. _data: | ||
|
||
Data Elements | ||
************************ | ||
|
||
All of the major Carper projects: trlX, CHEESE, and magiCARP use | ||
dataclasses corresponding to batches of data to communicate data between models and different | ||
components. trlX is no different, though it has many different dataclasses for | ||
different components like training or inference. Currently, we support PPO and ILQL, which | ||
each demand different kinds of data during training. | ||
|
||
|
||
**Basic Data Elements for Accelerate** | ||
|
||
.. autoclass:: trlx.data.accelerate_base_datatypes.PromptElement | ||
:members: | ||
|
||
.. autoclass:: trlx.data.accelerate_base_datatypes.PromptBatch | ||
:members: | ||
|
||
.. autoclass:: trlx.data.accelerate_base_datatypes.AccelerateRLElement | ||
:members: | ||
|
||
.. autoclass:: trlx.data.accelerate_base_datatypes.AccelerateRLBatchElement | ||
:members: | ||
|
||
**Data Elements for PPO** | ||
|
||
.. autoclass:: trlx.data.ppo_types.PPORLElement | ||
:members: | ||
|
||
.. autoclass:: trlx.data.ppo_types.PPORLBatch | ||
:members: | ||
|
||
**Data Elements for ILQL** | ||
|
||
.. autoclass:: trlx.data.ilql_types.ILQLElement | ||
:members: | ||
|
||
.. autoclass:: trlx.data.ilql_types.ILQLBatch | ||
:members: |
Oops, something went wrong.