From 56076a9b79f077663f2a3832714af3b4bf23996d Mon Sep 17 00:00:00 2001 From: Gurkengewuerz Date: Sat, 26 Oct 2024 01:16:37 +0200 Subject: [PATCH] feat(docs): updated readme and docs --- README.md | 254 ++++----------------------------------- docs/architecture.md | 63 ++++++++++ docs/configuration.md | 25 ++++ docs/development.md | 90 ++++++++++++++ docs/instructor-guide.md | 29 +++++ docs/quick-start.md | 49 ++++++++ docs/student-guide.md | 70 +++++++++++ docs/test-cases.md | 118 ++++++++++++++++++ docs/web-handlers.md | 75 ++++++++++++ 9 files changed, 543 insertions(+), 230 deletions(-) create mode 100644 docs/architecture.md create mode 100644 docs/configuration.md create mode 100644 docs/development.md create mode 100644 docs/instructor-guide.md create mode 100644 docs/quick-start.md create mode 100644 docs/student-guide.md create mode 100644 docs/test-cases.md create mode 100644 docs/web-handlers.md diff --git a/README.md b/README.md index b9c4ca5..422364d 100644 --- a/README.md +++ b/README.md @@ -1,243 +1,37 @@ # Automated Programming Workshop Judge -A secure, scalable system for automatically testing student programming assignments in university workshops. This system -integrates with Gitea for submission handling and uses Docker for secure code execution. +A secure, scalable system for automatically testing student programming assignments in university workshops. This system integrates with Gitea for submission handling and uses Docker for secure code execution. Additionally, it seamlessly integrates with [ContainerSSH](https://containerssh.io/) to provide secure SSH access to the containers for debugging and interactive sessions. ## Features -- **Secure Execution**: All student code runs in isolated Docker containers -- **Parallel Processing**: Handles multiple submissions simultaneously (configurable) -- **Real-time Feedback**: Students receive immediate test results on their commits -- **Scalable**: Handles large classes (100+ students) efficiently -- **Privacy**: Students can't access other students' solutions -- **Multiple Workshop Support**: Organize test cases by workshop and task -- **Flexible Test Cases**: Support for both YAML config and separate input/output files - -## System Architecture - -``` - ┌─────────────┐ - │ Gitea │ - │ (Git Host) │ - └─────┬───────┘ - │ webhook - ▼ -┌──────────────┐ ┌─────────────┐ ┌─────────────┐ -│ Test Cases │ │ Judge │ │ Docker │ -│ Repository │──▶│ Server │──▶│ Containers │ -└──────────────┘ └─────────────┘ └─────────────┘ - │ - │ results - ▼ - ┌─────────────┐ - │ Commit │ - │ Comments │ - └─────────────┘ -``` - -## Prerequisites - -- Docker Engine 20.10+ -- Docker Compose v2.0+ -- Git -- Gitea instance - -## Quick Start - -1. Clone the repository: - -```bash -git clone git@github.com:Gurkengewuerz/GitCodeJudge.git -cd GitCodeJudge -``` - -2. Create `.env` file: - -```env -GITEA_URL=http://gitea:3000 -GITEA_TOKEN=your_gitea_token -GITEA_WEBHOOK_SECRET=your_webhook_secret -MAX_PARALLEL_JUDGES=5 -DOCKER_TIMEOUT=30 -``` - -3. Start the services: - -```bash -docker-compose up -d -``` - -## Configuration - -### Environment Variables - -## Configuration - -### Environment Variables - -| Variable | Description | Default | Required | -|----------------------------|-----------------------------------------|---------------------------------------------------|----------| -| **Server Configuration** | -| `SERVER_ADDRESS` | Judge server address | `:3000` | No | -| `LOG_LEVEL` | Log level from 0-6. 4 being Info | `4` | No | -| `MAX_PARALLEL_JUDGES` | Maximum parallel executions | `5` | No | -| `TESTS_PATH` | Path to test cases directory | `test_cases` | No | -| **Database Configuration** | -| `DB_PATH` | Path to the database directory | `database/` | No | -| `DB_TTL` | Database TTL in Hours. 0 means disabled | `0` | No | -| **PDF Configuration** | -| `PDF_FOOTER_COPYRIGHT` | Copyright text in PDF footer | ` ` | No | -| `PDF_FOOTER_GENERATEDWITH` | Generated with text in PDF footer | `Generated with GitCodeJudge` | No | -| **Gitea Configuration** | -| `GITEA_URL` | Gitea server URL | - | Yes | -| `GITEA_TOKEN` | Gitea API token | - | Yes | -| `GITEA_WEBHOOK_SECRET` | Webhook secret | - | Yes | -| **Docker Configuration** | -| `DOCKER_IMAGE` | Base image for code execution | `ghcr.io/gurkengewuerz/gitcodejudge-judge:latest` | No | -| `DOCKER_NETWORK` | Docker network mode | `none` | No | -| `DOCKER_TIMEOUT` | Execution timeout (seconds) | `30` | No | - -### Test Case Configuration - -Each programming task is defined by a `config.yaml` file in its respective directory. The configuration supports visible -and hidden test cases, task metadata, and time constraints. Example test cases can be found in [ -`test_cases/`](test_cases/) - -#### Directory Structure - -``` -test_cases/ -├── workshop1/ -│ ├── task1/ -│ │ └── config.yaml -│ └── task2/ -│ └── config.yaml -└── workshop2/ - └── task1/ - └── config.yaml -``` - -#### Configuration Fields - -| Field | Type | Description | Required | -|----------------|----------|--------------------------------------------|---------------------| -| `disabled` | boolean | Whether the task is disabled | No (default: false) | -| `name` | string | Task name | Yes | -| `description` | string | Detailed task description and requirements | Yes | -| `start_date` | datetime | When the task becomes available | Yes | -| `end_date` | datetime | When the task expires | Yes | -| `cases` | array | List of visible test cases | Yes | -| `hidden_cases` | array | List of hidden test cases | No | - -#### Test Cases - -Each test case (both visible and hidden) must include: - -- `input`: The input that will be provided to the student's program -- `expected`: The expected output that the program should produce - -#### Important Notes: - -1. Hidden test cases work exactly like visible ones but results aren't shown to students -2. Whitespace are trimmed in the expected output -3. Make sure to maintain proper indentation in the YAML file -4. Use a . for in the first row for proper YAML indentation (see [ - `test_cases/workshop1/pascal_triangle`](test_cases/workshop1/pascal_triangle/config.yaml)) -5. Time constraints (`start_date` and `end_date`) use ISO 8601 format - -## Setup for Instructors - -1. Create a Gitea organization for your class -2. Add webhook to the organization: - - URL: `http://judge:3000/webhook` - - Secret: Match `GITEA_WEBHOOK_SECRET` - - Events: `Push` - -3. Create student repositories: - -```bash -./scripts/create_repos.sh organization_name student_list.txt -``` - -4. Add test cases to the `test_cases` directory following the structure: - -``` -test_cases/ -├── workshop1/ -│ ├── task1/ -│ │ └── config.yaml -│ └── task2/ -│ └── config.yaml -└── workshop2/ - └── task1/ - └── config.yaml -``` - -5. Create a Gitea access-token for an admin user with repository _read/write_ permissions. Set the - token as `GITEA_TOKEN`. - -## Student Usage - -1. Clone your assigned repository -2. Write your solution inside the ``/``/ directory -3. Commit and push your code -4. Check the commit comments or issues for test results - -Example test result: - -```markdown -Test results for commit: ba54e6212c3057dc24ce4cce0682bae3b96a78b3 - -## ✅ All Tests Passed - -### Test Results - -| Test # | Task | Status | Time | Details | -|--------|---------------------------|--------|-------|---------| -| 1 | workshop1/pascal_triangle | ✅ | 0.14s | | -| 2 | workshop1/pascal_triangle | ✅ | 0.16s | | -| 3 | workshop1/pascal_triangle | ✅ | 0.12s | | -``` - -## Development - -### Building from source: - -```bash -go build -o judge ./cmd/main.go -``` - -### Running tests: - -```bash -go test ./... -``` - -### Code formatting: - -```bash -go fmt ./... -``` - -## Contributing - -1. Fork the repository -2. Create your feature branch (`git checkout -b feature/amazing-feature`) -3. Commit your changes (`git commit -m 'Add amazing feature'`) -4. Push to the branch (`git push origin feature/amazing-feature`) -5. Open a Pull Request +- 🔒 **Secure Execution**: All student code runs natively in isolated Docker containers +- ⚙️ **Parallel Processing**: Handles multiple submissions simultaneously (configurable) +- ⏱️ **Real-time Feedback**: Students receive immediate test results on their commits +- 📈 **Scalable**: Handles large classes (100+ students) efficiently +- 🔐 **Privacy**: Students can't access other students' solutions (depending on the Git setup) +- 🏫 **Multiple Workshop Support**: Organize test cases by workshop and task +- 📝 **Flexible Test Cases**: Support for YAML configuration of test cases +- 🏆 **Leaderboard and Statistics**: Track student performance and display leaderboards +- 📄 **Problem PDF Exports**: Export problem statements and test cases to PDF +- 💻 **Multiple Programming Languages Support**: Supports testing code in various programming languages (currently Python, Go) +- 📅 **Time Constraints**: Set start and end dates for tasks +- 🖥️ **Interactive Development**: SSH access to development containers via ContainerSSH + +## Documentation + +- [Quick Start Guide](docs/quick-start.md) +- [Web Handlers](docs/web-handlers.md) +- [Architecture Overview](docs/architecture.md) +- [Configuration Guide](docs/configuration.md) +- [Test Case Setup](docs/test-cases.md) +- [Instructor Guide](docs/instructor-guide.md) +- [Student Guide](docs/student-guide.md) +- [Development Guide](docs/development.md) ## License This project is licensed under the AGPL - see the [LICENSE](LICENSE) file for details. -## Acknowledgments - -- [Gitea](https://gitea.io/) for Git hosting -- [Docker](https://www.docker.com/) for containerization -- [Go Fiber](https://gofiber.io/) for HTTP routing -- [Maroto](https://maroto.io/#/) for PDF rendering - ## Support For support, please open an issue in the repository. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..925273a --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,63 @@ +# System Architecture + +``` + ┌─────────────┐ + │ Gitea │ + │ (Git Host) │ + └─────┬───────┘ + │ webhook + ▼ +┌──────────────┐ ┌─────────────┐ ┌─────────────┐ +│ Test Cases │ │ Judge │ │ Docker │ +│ Repository │──▶│ Server │──▶│ Containers │ +└──────────────┘ └─────────────┘ └─────────────┘ + │ + │ results + ▼ + ┌─────────────┐ + │ Commit │ + │ Comments │ + └─────────────┘ +``` + +## Overview + +The GitCodeJudge system consists of multiple components working together to provide a secure and scalable automated +testing environment. In this example students uses ContainerSSH to connect to a development container to work on their code. +When they are ready to submit their code, they push it to a Git server. The Git server instance triggers a webhook to the Judge server. + +## Components Diagram + +```mermaid +flowchart TD + Student([Student]) + Git[Git Host] + Judge[Judge Server] + TestCases[Test Cases\nRepository] + Docker[Docker\nContainers] + Comments[Commit\nStatus] + SSH[ContainerSSH] + + %% Student interactions + Student -->|git push| Git + Student -->|ssh connect| SSH + + %% Main judge flow + Git -->|webhook| Judge + TestCases -->|load| Judge + Judge -->|execute| Docker + Judge -->|write| Comments + + %% ContainerSSH flow + SSH -->|create| Docker + SSH -->|authenticate| Git + + classDef system fill:#f9f,stroke:#333,stroke-width:2px; + classDef storage fill:#bbf,stroke:#333,stroke-width:2px; + classDef user fill:#dfd,stroke:#333,stroke-width:2px; + + class Git,Judge,SSH system; + class TestCases,Comments storage; + class Student user; + class Docker system; +``` diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..f206932 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,25 @@ +# Configuration Guide + +### Environment Variables + +| Variable | Description | Default | Required | +|----------------------------|-----------------------------------------|---------------------------------------------------|----------| +| **Server Configuration** | +| `SERVER_ADDRESS` | Judge server address | `:3000` | No | +| `LOG_LEVEL` | Log level from 0-6. 4 being Info | `4` | No | +| `MAX_PARALLEL_JUDGES` | Maximum parallel executions | `5` | No | +| `TESTS_PATH` | Path to test cases directory | `test_cases` | No | +| **Database Configuration** | +| `DB_PATH` | Path to the database directory | `database/` | No | +| `DB_TTL` | Database TTL in Hours. 0 means disabled | `0` | No | +| **PDF Configuration** | +| `PDF_FOOTER_COPYRIGHT` | Copyright text in PDF footer | ` ` | No | +| `PDF_FOOTER_GENERATEDWITH` | Generated with text in PDF footer | `Generated with GitCodeJudge` | No | +| **Gitea Configuration** | +| `GITEA_URL` | Gitea server URL | - | Yes | +| `GITEA_TOKEN` | Gitea API token | - | Yes | +| `GITEA_WEBHOOK_SECRET` | Webhook secret | - | Yes | +| **Docker Configuration** | +| `DOCKER_IMAGE` | Base image for code execution | `ghcr.io/gurkengewuerz/gitcodejudge-judge:latest` | No | +| `DOCKER_NETWORK` | Docker network mode | `none` | No | +| `DOCKER_TIMEOUT` | Execution timeout (seconds) | `30` | No | diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..fd61ab4 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,90 @@ +# Development Guide + +## Setup Development Environment + +### Prerequisites + +- Go 1.21+ +- Docker +- Git + +### Local Setup + +1. Clone repository: + ```bash + git clone git@github.com:Gurkengewuerz/GitCodeJudge.git + cd GitCodeJudge + ``` + +2. Install dependencies: + ```bash + go mod download + ``` + +## Building + +### Build Binary + +```bash +go build -o judge ./cmd/main.go +``` + +### Build Docker Image + +```bash +docker build -t gitcodejudge -f docker/server/Dockerfile . +``` + +```bash +docker build -t gitcodejudge -f docker/judge/Dockerfile docker/judge +``` + +## Testing + +### Run Tests + +```bash +go test ./... +``` + +### Run with Coverage + +```bash +go test -coverprofile=coverage.out ./... +go tool cover -html=coverage.out +``` + +## Code Style + +### Format Code + +```bash +go fmt ./... +``` + +### Run Linter + +```bash +golangci-lint run +``` + +## Project Structure + +``` +. +├── cmd/ # Application entrypoints +├── internal/ # Private application code +│ ├── config/ # Configuration handling +│ ├── judge/ # Core judge logic +│ ├── models/ # Data models +│ └── server/ # HTTP server +└── test_cases/ # Example test cases +``` + +## Contributing + +1. Fork the repository +2. Create your feature branch (`git checkout -b feature/amazing-feature`) +3. Commit your changes (`git commit -m 'Add amazing feature'`) +4. Push to the branch (`git push origin feature/amazing-feature`) +5. Open a Pull Request diff --git a/docs/instructor-guide.md b/docs/instructor-guide.md new file mode 100644 index 0000000..dab7bda --- /dev/null +++ b/docs/instructor-guide.md @@ -0,0 +1,29 @@ +# Instructor Guide + +## Initial Setup + +### 1. Create Organization +1. Log into Git Server +2. Create new organization +3. Configure organization settings + - Enable repository creation restrictions + - Set appropriate visibility settings + +### 2. Configure Webhook +1. Go to organization settings +2. Add webhook: + - URL: `http://judge:3000/webhook` + - Secret: Match `GITEA_WEBHOOK_SECRET` + - Events: Select `Push` + +### 3. Create Student Repositories +Use the provided script: +```bash +./scripts/create_repos.sh organization_name student_list.txt +``` + +Student list format: +```text +student1 email1@example.com +student2 email2@example.com +``` diff --git a/docs/quick-start.md b/docs/quick-start.md new file mode 100644 index 0000000..18699f1 --- /dev/null +++ b/docs/quick-start.md @@ -0,0 +1,49 @@ +# Quick Start Guide + +## Prerequisites + +- Docker Engine 20.10+ +- Docker Compose v2.0+ +- Git +- Gitea instance +- Traefik + +## Installation + +1. Clone the repository: +```bash +git clone git@github.com:Gurkengewuerz/GitCodeJudge.git +cd docker +``` + +2. Create `.env` file: +```env +CONTAINER_DIR=./data +VIRTUAL_HOST=judge.example.com +GITEA_URL=https://gitea.example.com +GITEA_TOKEN=your_gitea_token +GITEA_WEBHOOK_SECRET=your_webhook_secret +``` + +3. Start the services: +```bash +docker-compose up -d +``` + +## Verify Installation + +1. Check service status: +```bash +docker-compose ps +``` + +2. View logs: +```bash +docker-compose logs -f +``` + +## Next Steps + +- [Configure your environment](configuration.md) +- [Set up test cases](test-cases.md) +- [Read the instructor guide](instructor-guide.md) diff --git a/docs/student-guide.md b/docs/student-guide.md new file mode 100644 index 0000000..30c198a --- /dev/null +++ b/docs/student-guide.md @@ -0,0 +1,70 @@ +# Student Guide + +## Getting Started + +1. Get your repository credentials from your instructor +2. Clone your repository: + ```bash + git clone + ``` +3. Set up your SSH key in Git for access + +## Directory Structure + +Your repository should follow this structure: + +``` +your-repo/ +├── workshop1/ +│ ├── task1/ +│ │ └── solution.py # or other file types +│ └── task2/ +│ └── solution.py +└── workshop2/ + └── task1/ + └── solution.go +``` + +## Working on Tasks + +### Method 1: Local Development + +1. Write your solution locally +2. Test with provided test cases +3. Commit and push: + ```bash + git add . + git commit -m "Solve task1" + git push + ``` + +### Method 2: ContainerSSH Development + +1. Connect to development environment: + ```bash + ssh -p 2222 git@ + ``` +2. Write and test your solution +3. Commit and push from container + +## Understanding Test Results + +Test results appear as commit status with a link to the detailed results.: + +```markdown +## ✅ All Tests Passed + +### Test Results + +| Test # | Task | Status | Time | Details | +|--------|-------------------|--------|-------|---------| +| 1 | workshop1/task1 | ✅ | 0.14s | | +| 2 | workshop1/task1 | ✅ | 0.16s | | +``` + +## Tips + +- Check task deadlines in test case descriptions +- Use meaningful commit messages +- Test your code with edge cases +- Ask for help if you're stuck diff --git a/docs/test-cases.md b/docs/test-cases.md new file mode 100644 index 0000000..11883fe --- /dev/null +++ b/docs/test-cases.md @@ -0,0 +1,118 @@ +# Test Case Configuration + +Each programming task is defined by a `config.yaml` file in its respective directory. The configuration supports visible +and hidden test cases, task metadata, and time constraints. Example test cases can be found in [ +`test_cases/`](../test_cases/) + +## Directory Structure + +Test cases are organized by workshop and task: + +``` +test_cases/ +├── workshop1/ +│ ├── task1/ +│ │ └── config.yaml +│ └── task2/ +│ └── config.yaml +└── workshop2/ + └── task1/ + └── config.yaml +``` + +## Configuration File Format + +Each task requires a `config.yaml` file with the following structure: + +```yaml +disabled: false # Optional: disable task +name: "Task Name" # Required: task name +description: | # Required: task description + Detailed description of the task. + Can be multiple lines with Markdown support. + +start_date: "2024-01-01T00:00:00Z" # Required: ISO 8601 format +end_date: "2024-12-31T23:59:59Z" # Required: ISO 8601 format + +cases: # Required: visible test cases + - input: | + 5 + 3 4 + expected: | + 7 + +hidden_cases: # Optional: hidden test cases + - input: | + 10 + 20 + expected: | + 30 +``` + +### Important Notes: + +1. Hidden test cases work exactly like visible ones but results aren't shown to students +2. Whitespace are trimmed in the expected output +3. Make sure to maintain proper indentation in the YAML file +4. Use a . for in the first row for proper YAML indentation (see [ + `test_cases/workshop1/pascal_triangle`](../test_cases/workshop1/pascal_triangle/config.yaml)) +5. Time constraints (`start_date` and `end_date`) use ISO 8601 format + + +## Test Case Types + +### Visible Test Cases +- Students can see both input and expected output +- Results are shown in detail +- Good for learning and debugging + +### Hidden Test Cases +- Students only see if they passed or failed +- Prevents hardcoding solutions +- Tests edge cases and performance + +## Best Practices + +1. **Test Case Design** + - Start with simple cases + - Include edge cases + - Test error handling + - Consider performance limits + +2. **Description Writing** + - Use clear language + - Include input/output format + - Provide example cases + - List constraints + +3. **Time Management** + - Set reasonable start/end dates + - Consider timezone differences + - Allow buffer for technical issues + +## Example Test Cases + +### Simple Math Task +```yaml +name: "Addition" +description: | + Write a program that adds two numbers. + + Input: + - Two space-separated integers + + Output: + - Sum of the two numbers + +cases: + - input: "1 2" + expected: "3" + - input: "0 0" + expected: "0" + +hidden_cases: + - input: "-1 1" + expected: "0" + - input: "999999 1" + expected: "1000000" +``` diff --git a/docs/web-handlers.md b/docs/web-handlers.md new file mode 100644 index 0000000..8aeacc9 --- /dev/null +++ b/docs/web-handlers.md @@ -0,0 +1,75 @@ +# API Routes Documentation + +This document describes all available API endpoints in the GitCodeJudge system. + +## Base Routes + +### Health Check +``` +GET /health +``` +Simple health check endpoint to verify if the service is running. + +### Home Page Redirect +``` +GET / +``` +Redirects to `/leaderboard` using the rewrite middleware. + +## Webhook Integration + +### Gitea Webhook +``` +POST /webhook +``` +Handles repository events from the Git server. Protected by webhook secret validation middleware. + +**Authentication:** +- Requires valid webhook secret in request headers +- Used for processing repository events (commits, pushes) + +## Documentation + +### PDF Generation +``` +GET /pdf +``` +Generates and serves PDF documentation for programming problems/tasks. +- Includes problem descriptions +- Test case examples +- Task requirements + +## Results & Statistics + +### Commit Results +``` +GET /results/:commit +``` +Retrieves test results for a specific commit. +- Parameter: `commit` - The commit hash to get results for +- Shows test status, execution time, and error details if any + +### User Progress +``` +GET /user/:username +``` +Shows progress and statistics for a specific user. +- Parameter: `username` - The Gitea username +- Displays completed tasks, success rates + +### Workshop Statistics +``` +GET /workshop/:workshop/:task +``` +Provides statistics for a specific workshop task. +- Parameters: + - `workshop` - Workshop identifier + - `task` - Task identifier +- Shows completion rates + +### Leaderboard +``` +GET /leaderboard +``` +Displays the overall leaderboard. +- Task completion statistics