In simple English
This tool stores copies of web pages over time. It notices when a page changes and lets people write notes about what was updated. It does not look at live websites itself. It relies on external scrapers to feed data into it. You run it on your own server using Ruby and PostgreSQL.
What it is
An open-source Ruby on Rails application that acts as a database and API for tracking website changes. It saves page versions and organizes analyst notes into a structured format.
What it is good for
Archivists and researchers who need to track and annotate historical changes across a large set of official web pages.
- Keeps a clean history of web page updates over time.
- Includes an OpenAPI specification for easy API integration.
- Supports importing data from external archives like the Internet Archive.
- Requires Ruby, PostgreSQL, and Redis to run locally.
- Does not scrape web pages by itself without external tools.
- Licenced under GPL-3.0, which has strict copyleft rules.
manual bookmarking and static screenshot folders
PostgreSQL
- ArchiveBox/ArchiveBox29k stars
Archives web pages locally into multiple formats, whereas this tool provides an API database for historical page monitoring.
- cybertec-postgresql/pg_timetable1.4k stars
A job scheduler for PostgreSQL whose jobs and schedules live in the database itself.
- PrefectHQ/prefect24k stars
Adds scheduling, retries and a run history to Python functions you already have.
git clone https://github.com/edgi-govdata-archiving/web-monitoring-db.git cd web-monitoring-db bundle install cp .env.example .env bundle exec rake db:setup bundle exec rails server