WebWatch was a project built for CSE115A: Introduction to Software Engineering at UC Santa Cruz in Fall 2024.
Our team consists of Cameron Candau (Product Owner), Alex Kekchidis, Isaac To, Jordan Nguy, Leonardo Gallego, and Simon Zhao.
The concept arose from our personal experiences in obtaining parking permits at UCSC.
The web often used for sharing time-sensitive opportunities which lack built-in notification systems. Manually refreshing a webpage while anticipating its change drains valuable time and energy from the user.
WebWatch is a web application that enables users to monitor other webpages for changes.
Our core functionality included four primary features, all of which we achieved by the end of the quarter:
- A monitoring and notification system: Discord webhooks, Slack webhooks, and email (email notifications were implemented but archived due to budget constraints and the cost of SMTP services)
- User account management: User registration, login, password reset, passwordless authentication via email, and account deletion
- Site-Agnostic Web Scraping: Our web scraping and difference detection works with most web pages without requiring additional programming for each website
- User-friendly Interface: The task dashboard has an intuitive interface for basic task operations (creating, viewing, updating, and deleting tasks).
It is best suited for publicly acessible websites that don't implement anti-botting mechanisms. For instance, course offerings and registration status, parking permit availabilities, portfolios, and job boards.
We used Python with FastAPI to build our backend REST API, SQLite as our temporary database, and SQLModel as our ORM to interact with the database. The task scheduler used AsyncIO for asynchronous job scheduling, and a combination of Selenium, beautifulsoup4, and difflib was used to implement our web scraping and change detection.
Our frontend was built with React and Tailwind CSS.
We deployed WebWatch at https://webwatch.live on an Ubuntu cloud VM using Traefik as our reverse proxy and nginx for serving our frontend site to Traefik internally. This was orchestrated using Docker Compose.
We used Github Actions to create CI/CD pipelines including automated styling checks, API unit tests, and automatic deployment to our live site upon pushing to main. During development, we used DevContainers to minimize platform and version conflicts, which also allowed for a smooth transition into deployment.
We followed the Scrum framework and worked on the project for a duration of 8 weeks, comprised of four two-week sprints. We held Scrum meetings 3 times a week, rotating scrum masters at least once during each sprint. Our scrum documents and other deliverables from the course can be found here
WebWatch is no longer being developed or hosted for use. As an alternative, we recommend https://changedetection.io/, an established open-source project.
- Install Dev Container VSCode Extension
- Open the frontend or backend folder in VSCode.
- Open the project in the Dev Container (through Extension GUI prompt or "Open a Remote Window" icon in the bottom left of VSCode)
- Container will take a moment to copy its image, install project dependencies (requirements.txt for backend, or package.json for frontend) and start.
- Work normally; project file changes are saved to the host machine.
- If you get issues when committing or pushing code, try reopening the project locally to do it from outside the container.
- Python packages/versions named in
backend/requirements.txtare automatically installed viapipwith container root (no virtual environment). - If you install new packages, run the following in
backend/to updaterequirements.txt:pip freeze > requirements.txt