Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

d2x package source, reworked #58

Merged
merged 66 commits into from
Nov 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
70b0e7e
Restructuring, to pass to copilot workspaces :)
jlantz Oct 29, 2024
478dc63
Restructure python
jlantz Oct 29, 2024
9a9438d
Ready for copilot!
jlantz Oct 29, 2024
0efeff1
Update authentication structure and decouple from GitHub Environments
jlantz Oct 29, 2024
31b1740
Update `d2x/auth/sf/auth_url.py` to use environment variables for sec…
jlantz Oct 29, 2024
b3e15c6
Add requests dependency to pyproject.toml and requirements.txt
jlantz Oct 29, 2024
ef841b8
Add development dependencies and update build system configuration
jlantz Oct 29, 2024
424fe68
Add dependencies and configure pip-compile in `pyproject.toml`
jlantz Oct 29, 2024
18b2ac3
Fix pip install command in test workflow and requirements_dev.in
jlantz Oct 29, 2024
2b91377
Update test workflow to use poetry for dependency management
jlantz Oct 29, 2024
d8b6c6e
Update dependency management and test workflow
jlantz Oct 29, 2024
749d840
Update test workflow to use poetry for dependency management
jlantz Oct 29, 2024
d0ef474
Update `poetry.lock` to reflect changes in dependencies
jlantz Oct 29, 2024
43627ff
Update lock file
jlantz Oct 29, 2024
4530329
Add python version to package
jlantz Oct 29, 2024
da97b93
Remove python
jlantz Oct 29, 2024
7c42f69
Update python version in build, dewlete requirements_dev.in
jlantz Oct 29, 2024
e32fe0d
Update Python version and add caching to GitHub Actions workflow
jlantz Oct 29, 2024
a6437bc
update
jlantz Oct 29, 2024
ac3d140
Fix dependencies
jlantz Oct 29, 2024
0b91a00
Update dependencies logic
jlantz Oct 29, 2024
0cb549b
Update action versions
jlantz Oct 29, 2024
dd12faa
Switch to poetry based build action
jlantz Oct 29, 2024
0661b2a
Working d2x cli using rich_click with auth-url and login-url commands
jlantz Oct 30, 2024
1e67fd2
VSCode debug and test configs and add requirements updates
jlantz Oct 30, 2024
d012f9b
Rev version
jlantz Oct 30, 2024
60cb59d
Use new d2x cli
jlantz Oct 30, 2024
6cf5cf2
Change cli structure, fix package entry points
jlantz Oct 30, 2024
c3423cf
New command structure
jlantz Oct 30, 2024
9a89f4f
Use branch for now
jlantz Oct 30, 2024
dd45022
Attempt to fix error checking
jlantz Oct 30, 2024
a1cac13
Don't output start url!!!
jlantz Oct 30, 2024
9e273a9
Add --version option
jlantz Oct 30, 2024
4e7002e
Implement two-stage auth model with GitHub Environments
jlantz Oct 30, 2024
df27f13
Add tests for `exchange_token` and `generate_login_url` functions
jlantz Oct 30, 2024
e65b483
update lock
jlantz Oct 30, 2024
2bcd30a
Add GitHub Actions workflow to test GitHub authentication
jlantz Oct 30, 2024
750581c
Update `d2x/models/sf/auth.py` to use `ConfigDict` instead of class-b…
jlantz Oct 30, 2024
05588c1
Add empty `__init__.py` files for various modules
jlantz Oct 30, 2024
dfc9d8b
Add import for `get_environment_variable` and `json` module
jlantz Oct 30, 2024
2b1e18f
* Initialize `debug_info` before the try block in `exchange_token` fu…
jlantz Oct 30, 2024
732a758
Add functions to set and get environment variables and secrets in Git…
jlantz Oct 30, 2024
28bb97e
Add cryptography extension
jlantz Oct 30, 2024
2e88b66
Add reusable workflow to delete an org session
jlantz Oct 30, 2024
7f82708
Refactor d2x.gen and d2x.parse as Pydantic models
jlantz Oct 30, 2024
647f340
Add tests for `LoginUrlModel` and `SfdxAuthUrlModel`
jlantz Oct 30, 2024
a0ec586
Update `requests.put` call and clean up imports in `d2x/api/gh.py`
jlantz Oct 30, 2024
b18ba23
Assign a value to the `DEMO` member in the `OrgType` Enum
jlantz Oct 30, 2024
5d26234
Add tests for `LoginUrlModel` and `SfdxAuthUrlModel`
jlantz Oct 31, 2024
8e4c21c
Add import for `re` module in `tests/test_auth.py`
jlantz Oct 31, 2024
628d135
Add import for `re` module in `d2x/models/sf/auth.py`
jlantz Oct 31, 2024
68f2e82
Update `get_login_url_and_token` method in `LoginUrlModel` to handle …
jlantz Oct 31, 2024
fb2e3de
Add a reusable workflow for Python tests
jlantz Oct 31, 2024
ba0b3e2
Add a reusable workflow for Python tests
jlantz Oct 31, 2024
fa52ba4
Add reusable workflow for Python tests
jlantz Oct 31, 2024
cd872d2
Fix AI mistakes
jlantz Oct 31, 2024
3434053
Fix tests and upgrade test workflow
jlantz Oct 31, 2024
f1b49a4
Merge branch 'jlantz/implement-two-stage-auth' of github.com:muselab-…
jlantz Oct 31, 2024
3529a01
Fix merge conflict
jlantz Oct 31, 2024
ea2df94
Merge pull request #36 from muselab-d2x/jlantz/implement-two-stage-auth
jlantz Oct 31, 2024
794fcb1
Remove duplicate pytest workflow
jlantz Oct 31, 2024
592e824
Switch workflows
jlantz Oct 31, 2024
f8675e3
Merge branch 'jlantz/update-auth-structure' into jlantz/refactor-pyda…
jlantz Oct 31, 2024
ae55608
Fix tests
jlantz Oct 31, 2024
8462b62
Merge pull request #44 from muselab-d2x/jlantz/refactor-pydantic-models
jlantz Oct 31, 2024
18a8543
Merge branch 'cumulusci-next-snapshots' into jlantz/update-auth-struc…
jlantz Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .coverage
Binary file not shown.
2 changes: 1 addition & 1 deletion .github/actions/collect-org-info/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,4 @@ runs:
fi
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_TOKEN: ${{ github.token }}
47 changes: 47 additions & 0 deletions .github/workflows/delete-org-session.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Delete Org Session

on:
workflow_call:
inputs:
environment_name:
description: "The name of the GitHub Environment to delete the org session from"
required: true
type: string
github_auth_environment:
description: "The name of the GitHub Environment to get the GitHub Access token from"
required: true
type: string
secrets:
github-token:
required: true

jobs:
delete-org-session:
name: "Delete Org Session"
runs-on: ubuntu-latest
steps:
- name: Get GitHub Access Token
run: |
echo "Retrieving GitHub Access Token from environment: ${{ inputs.github_auth_environment }}"
GITHUB_ACCESS_TOKEN=$(gh api \
-H "Authorization: token ${{ secrets.github-token }}" \
"/repos/${{ github.repository }}/environments/${{ inputs.github_auth_environment }}/variables/GITHUB_ACCESS_TOKEN" \
| jq -r '.value')
echo "GITHUB_ACCESS_TOKEN=${GITHUB_ACCESS_TOKEN}" >> $GITHUB_ENV
shell: bash

- name: Delete Org Session
run: |
echo "Deleting org session from environment: ${{ inputs.environment_name }}"
gh api \
-X DELETE \
-H "Authorization: token ${{ env.GITHUB_ACCESS_TOKEN }}" \
"/repos/${{ github.repository }}/environments/${{ inputs.environment_name }}/variables/ACCESS_TOKEN"
shell: bash

- name: Add Job Summary
run: |
echo "## Org Session Deletion Summary" >> $GITHUB_STEP_SUMMARY
echo "Environment: ${{ inputs.environment_name }}" >> $GITHUB_STEP_SUMMARY
echo "Status: Org session deleted successfully" >> $GITHUB_STEP_SUMMARY
shell: bash
2 changes: 1 addition & 1 deletion .github/workflows/org-login-slack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -137,4 +137,4 @@ jobs:
exit 1
fi

echo "Slack message sent successfully."
echo "Slack message sent successfully."
35 changes: 35 additions & 0 deletions .github/workflows/test-github-auth.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Test GitHub Auth

on:
push:
branches:
- "**"

jobs:
test-github-auth:
runs-on: ubuntu-latest
environment: test
container:
image: ghcr.io/muselab-d2x/d2x:cumulusci-next-snapshots
options: --user root
credentials:
username: ${{ github.actor }}
password: ${{ secrets.github-token }}
env:
DEV_HUB_AUTH_URL: "${{ secrets.dev-hub-auth-url }}"
CUMULUSCI_SERVICE_github: '{ "username": "${{ github.actor }}", "token": "${{ secrets.github-token }}", "email": "${{ secrets.gh-email }}" }'
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Auth to DevHub
run: /usr/local/bin/devhub.sh
- name: Test GitHub Auth
run: |
d2x auth url
d2x auth login
shell: bash
- name: Record API Requests
run: |
pip install vcrpy
vcrpy --record-mode=once --filter-headers Authorization --filter-headers X-Auth-Token --filter-headers X-API-Key
shell: bash
54 changes: 54 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Run Tests
on:
push:
branches:
- "**"
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: 1.7.1
virtualenvs-create: true
virtualenvs-in-project: true

- name: Cache dependencies
uses: actions/cache@v4
with:
path: |
.venv
~/.cache/pypoetry
key: ${{ runner.os }}-poetry-${{ hashFiles('**/poetry.lock') }}
restore-keys: |
${{ runner.os }}-poetry-

- name: Install dependencies
run: |
poetry install --with dev

- name: Run tests
run: |
poetry run pytest tests/ --cov=d2x --cov-report=xml --junitxml=tests/results.xml
echo "✅ Tests and coverage completed"

- name: Upload coverage report
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage.xml

- name: Upload test results
uses: actions/upload-artifact@v4
with:
name: test-results
path: tests/results.xml
25 changes: 25 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// .vscode/launch.json
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Debug Active File",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"justMyCode": false
},
{
"name": "Python: Debug Tests",
"type": "python",
"request": "launch",
"module": "pytest",
"args": [
"tests"
],
"console": "integratedTerminal",
"justMyCode": false
}
]
}
7 changes: 7 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
1 change: 1 addition & 0 deletions d2x/api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""d2x.api"""
2 changes: 1 addition & 1 deletion d2x/api/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
"""d2x.api"""
# d2x.api module
91 changes: 91 additions & 0 deletions d2x/api/gh.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
import os
import requests

GITHUB_REPO = os.environ.get("GITHUB_REPOSITORY")


def get_github_token() -> str:
"""Get the GitHub token from the environment"""
token = os.environ.get("GITHUB_TOKEN")
if not token:
raise ValueError("GITHUB_TOKEN environment variable not set")
return token


def get_repo_full_name() -> str:
"""Get the full name of the GitHub repository"""
repo = os.environ.get("GITHUB_REPOSITORY")
if not repo:
raise ValueError("GITHUB_REPOSITORY environment variable not set")
return repo


def set_environment_variable(env_name: str, var_name: str, var_value: str) -> None:
"""Set a variable in a GitHub Environment"""
token = os.environ.get("GITHUB_TOKEN")
repo = os.environ.get("GITHUB_REPOSITORY")
if not token:
raise ValueError("GITHUB_TOKEN environment variable not set")

url = f"https://api.github.com/repos/{GITHUB_REPO}/environments/{env_name}/variables/{var_name}"
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github.v3+json",
}
data = {"name": var_name, "value": var_value}

response = requests.put(url, headers=headers, json=data)
response.raise_for_status()


def get_environment_variable(env_name: str, var_name: str) -> str:
"""Get a variable from a GitHub Environment"""
token = os.environ.get("GITHUB_TOKEN")
if not token:
raise ValueError("GITHUB_TOKEN environment variable not set")

url = f"https://api.github.com/repos/{GITHUB_REPO}/environments/{env_name}/variables/{var_name}"
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github.v3+json",
}

response = requests.get(url, headers=headers)
response.raise_for_status()

return response.json()["value"]


def set_environment_secret(env_name: str, secret_name: str, secret_value: str) -> None:
"""Set a secret in a GitHub Environment"""
token = os.environ.get("GITHUB_TOKEN")
if not token:
raise ValueError("GITHUB_TOKEN environment variable not set")

url = f"https://api.github.com/repos/{GITHUB_REPO}/environments/{env_name}/secrets/{secret_name}"
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github.v3+json",
}
data = {"encrypted_value": secret_value}

response = requests.put(url, headers=headers, json=data)
response.raise_for_status()


def get_environment_secret(env_name: str, secret_name: str) -> str:
"""Get a secret from a GitHub Environment"""
token = os.environ.get("GITHUB_TOKEN")
if not token:
raise ValueError("GITHUB_TOKEN environment variable not set")

url = f"https://api.github.com/repos/{GITHUB_REPO}/environments/{env_name}/secrets/{secret_name}"
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github.v3+json",
}

response = requests.get(url, headers=headers)
response.raise_for_status()

return response.json()["encrypted_value"]
2 changes: 1 addition & 1 deletion d2x/auth/sf/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
"""d2x.auth.sf"""
# This is the __init__.py file for the d2x.auth.sf module.
Loading
Loading