Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed submissions not working on dev env #86

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions create_debug_users.py

This file was deleted.

35 changes: 31 additions & 4 deletions docs/source/contributing/setup.rst
Original file line number Diff line number Diff line change
@@ -3,13 +3,16 @@
Setting up a development environment
------------------------------------

Basic Setup
~~~~~~~~~~~

First, you will need to install the following:

* ``python``
* ``pipenv``
* ``git``

You will also need a Github account.
You will also need a GitHub account.

First, `fork <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo#forking-a-repository>`_
tin. Then you can clone tin onto your computer with
@@ -30,20 +33,44 @@ After that, install dependencies and follow standard django procedures
.. code-block:: bash

pipenv install --dev
python3 manage.py migrate
python3 create_debug_users.py
pipenv run python3 manage.py migrate
pipenv run python3 manage.py create_debug_users


Now you're all set! Try running the development server

.. code-block:: bash

python3 manage.py runserver
pipenv run python3 manage.py runserver

Head on over to `http://127.0.0.1:8000 <http://127.0.0.1:8000>`_, and login
as ``admin`` and the password you just entered.


Submissions
~~~~~~~~~~~

In order to actually create a submission, there are some more steps. First,
you'll need to install `redis <https://redis.io/download>`_.

After that, you'll want to start up the development server and create a course,
and an assignment in the course. After saving the assignment, you can hit "Upload grader"
to add a grader - the simplest example of a grader is located in ``scripts/sample_grader.py``.

Finally, before creating a submission, you'll need to start the celery worker. This can be done
by running the following command in a separate terminal::

pipenv run celery -A tin worker --loglevel=info

Now you can try making a submission, and as long as your submission doesn't throw an error you
should get a 100%! Congrats on your brand new 5.0 GPA!

.. tip::

If you're on a Unix-like system, you can use the ``&`` operator to run the celery worker in the background::

pipenv run celery -A tin worker --loglevel=info & pipenv run python3 manage.py runserver
JasonGrace2282 marked this conversation as resolved.
Show resolved Hide resolved


NixOS Setup
-----------
7 changes: 7 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -176,6 +176,13 @@ extend-ignore-names = [

"docs/*" = [
"BLE001",
]

"scripts/*" = [
"INP001",
]

"**/management/*" = [
"INP001",
]

55 changes: 55 additions & 0 deletions scripts/grader_wrapper.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""A sample wrapper script for running python submissions.

The text in this file is read in :func:`run_submission` and
executed if ``settings.USE_SANDBOXING`` is not ``True``.
"""

from __future__ import annotations

import argparse
import subprocess
import sys
from pathlib import Path


def parse_args() -> list[str]:
parser = argparse.ArgumentParser()
parser.add_argument("--write", action="append")
parser.add_argument("--read", action="append")
# since we're not being sandboxed, we don't need to do anything
# with the grader arguments
_grader_args, submission_args = parser.parse_known_args()

if submission_args and submission_args[0] == "--":
return submission_args[1:]
return submission_args


def find_python() -> str:
venv = Path("{venv_path}")
if venv.name == "None":
return "{python}"
if (python := venv / "bin" / "python").exists():
return str(python)
return str(venv / "bin" / "python3")


def main() -> int:
args = parse_args()
submission_path = Path("{submission_path}")

if submission_path.suffix != ".py":
raise NotImplementedError("Only python submissions are supported in DEBUG.")

python = find_python()
output = subprocess.run(
[python, "--", str(submission_path), *args],
stdout=sys.stdout,
stderr=sys.stderr,
check=False,
)
return output.returncode


if __name__ == "__main__":
sys.exit(main())
22 changes: 22 additions & 0 deletions scripts/sample_grader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
"""The (second) simplest grader.

This runs the student submission, and without checking the output
gives them a 100%. However, if the submission crashes the student will get a 0%
"""

from __future__ import annotations

import subprocess
import sys

process = subprocess.run(
[sys.executable, sys.argv[1]],
stdout=sys.stdout,
stderr=subprocess.STDOUT,
check=False,
)

if process.returncode != 0:
print("Score: 0%")
else:
print("Score: 100%")
70 changes: 39 additions & 31 deletions tin/apps/submissions/tasks.py
JasonGrace2282 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
from __future__ import annotations

import contextlib
import logging
import os
import re
import select
import shutil
import signal
import subprocess
import sys
import time
import traceback
from decimal import Decimal
from pathlib import Path

import psutil
from asgiref.sync import async_to_sync
@@ -62,37 +64,43 @@ def run_submission(submission_id):
logger.error("Cannot run processes: %s", e)
raise FileNotFoundError from e

python_exe = (
os.path.join(submission.assignment.venv.path, "bin", "python")
if submission.assignment.venv_fully_created
else "/usr/bin/python3.10"
)
if submission.assignment.venv_fully_created:
python_exe = os.path.join(submission.assignment.venv.path, "bin", "python")
elif settings.DEBUG:
python_exe = sys.executable
JasonGrace2282 marked this conversation as resolved.
Show resolved Hide resolved
else: # pragma: no cover
python_exe = "/usr/bin/python3.10"

if settings.USE_SANDBOXING:
wrapper_text = (
Path(settings.BASE_DIR)
.joinpath(
"sandboxing",
"wrappers",
"sandboxed",
f"{submission.assignment.language}.txt",
)
.read_text("utf-8")
)

if not settings.DEBUG or shutil.which("bwrap") is not None:
folder_name = "sandboxed"
elif submission.assignment.language == "P":
wrapper_text = settings.DEBUG_GRADER_WRAPPER_SCRIPT.read_text("utf-8")
else:
folder_name = "testing"

with open(
os.path.join(
settings.BASE_DIR,
"sandboxing",
"wrappers",
folder_name,
f"{submission.assignment.language}.txt",
)
) as wrapper_file:
wrapper_text = wrapper_file.read().format(
has_network_access=bool(submission.assignment.has_network_access),
venv_path=(
submission.assignment.venv.path
if submission.assignment.venv_fully_created
else None
),
submission_path=submission_path,
python=python_exe,
raise NotImplementedError(
f"Unsupported language {submission.assignment.language} in DEBUG"
)

wrapper_text = wrapper_text.format(
has_network_access=bool(submission.assignment.has_network_access),
venv_path=(
submission.assignment.venv.path
if submission.assignment.venv_fully_created
else None
),
submission_path=submission_path,
python=python_exe,
)

with open(submission_wrapper_path, "w", encoding="utf-8") as f_obj:
f_obj.write(wrapper_text)

@@ -130,7 +138,7 @@ def run_submission(submission_id):
grader_log_path,
]

if not settings.DEBUG or shutil.which("firejail") is not None:
if settings.USE_SANDBOXING:
JasonGrace2282 marked this conversation as resolved.
Show resolved Hide resolved
whitelist = [os.path.dirname(grader_path)]
read_only = [grader_path, submission_path, os.path.dirname(submission_wrapper_path)]
if submission.assignment.venv_fully_created:
@@ -145,7 +153,7 @@ def run_submission(submission_id):
read_only=read_only,
)

env = dict(os.environ)
env = os.environ.copy()
if submission.assignment.venv_fully_created:
env.update(submission.assignment.venv.get_activation_env())

@@ -273,5 +281,5 @@ def run_submission(submission_id):
submission.channel_group_name, {"type": "submission.updated"}
)

if os.path.exists(submission_wrapper_path):
with contextlib.suppress(FileNotFoundError):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats the reasoning behind this change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, with file IO stuff, checking if a file exists before deleting it is considered an antipattern (just because some other process can delete the file in between checking if it exists and deleting it). Usually, it's better to try deleting the file and ignore any problems that occur (like if the file doesn't exist).
This is also why we have the exist_ok parameters with stuff like os.path.mkdirs/Path.mkdir.

os.remove(submission_wrapper_path)
28 changes: 28 additions & 0 deletions tin/apps/users/management/commands/create_debug_users.py
JasonGrace2282 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
from __future__ import annotations

from getpass import getpass

from django.core.management.base import BaseCommand, no_translations

import tin.tests.create_users as users


class Command(BaseCommand):
help = "Create users for debugging"

def add_arguments(self, parser):
parser.add_argument("--noinput", action="store_true", help="Do not ask for password")
parser.add_argument("--force", action="store_true", help="Force creation of users")

@no_translations
def handle(self, *args, **options):
if not options["noinput"]:
pwd = getpass("Enter password for all users: ")
else:
pwd = "jasongrace"

users.add_users_to_database(
password=pwd,
verbose=options["verbosity"] > 0,
force=options["force"],
)
22 changes: 22 additions & 0 deletions tin/apps/users/tests.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from __future__ import annotations

import pytest
from django.contrib.auth import get_user_model
from django.core.management import call_command


@pytest.mark.no_autocreate_users
def test_create_debug_users():
admin = get_user_model().objects.filter(username="admin", is_superuser=True)
student = get_user_model().objects.filter(username="student", is_student=True)
teacher = get_user_model().objects.filter(username="teacher", is_teacher=True)

assert not admin.exists()
assert not teacher.exists()
assert not student.exists()

call_command("create_debug_users", noinput=True, verbosity=0)

assert admin.exists()
assert teacher.exists()
assert student.exists()
5 changes: 4 additions & 1 deletion tin/apps/venvs/tasks.py
Original file line number Diff line number Diff line change
@@ -15,6 +15,9 @@
@shared_task
def create_venv(venv_id):
venv = Venv.objects.get(id=venv_id)
python = settings.SUBMISSION_PYTHON
if settings.DEBUG:
python = sys.executable

success = False
try:
@@ -25,7 +28,7 @@ def create_venv(venv_id):
"-m",
"virtualenv",
"-p",
settings.SUBMISSION_PYTHON,
python,
"--",
venv.path,
],
10 changes: 10 additions & 0 deletions tin/settings/__init__.py
Original file line number Diff line number Diff line change
@@ -12,6 +12,7 @@
from __future__ import annotations

import os
from pathlib import Path

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -28,6 +29,11 @@
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

USE_SANDBOXING = (
not DEBUG or Path(BASE_DIR).joinpath("sandboxing", "wrappers", "sandboxed", "P.txt").exists()
)


ALLOWED_HOSTS = [
"127.0.0.1",
"localhost",
@@ -304,6 +310,10 @@

VENV_FILE_SIZE_LIMIT = 1 * 1000 * 1000 * 1000 # 1 GB

# The wrapper script to use when running submissions outside of production
# We still need this so that it can handle cli arguments to the wrapper script
DEBUG_GRADER_WRAPPER_SCRIPT = Path(BASE_DIR).parent / "scripts" / "grader_wrapper.py"

# Spaces and special characters may not be handled correctly
# Not importing correctly - specified directly in apps/submissions/tasks.py
# as of 8/3/2022, 2022ldelwich
Loading
Loading