Skip to main content

Python library and CLI for archiving URLs on popular services like Wayback Machine

Project description

pgark

Python library and CLI for archiving URLs on popular services like Wayback Machine

Basically a fork of the great pastpages/savepagenow

How to use

Install with:

$ pip install pgark

For a given URL, to get the latest available snapshot for a URL:

$ pgark check whitehouse.gov

http://web.archive.org/web/20200904180914/https://www.whitehouse.gov/

To get the JSON response from the Wayback Machine API, pass in the -j/--json flag:

$ pgark check -j whitehouse.gov
{
  "archived_snapshots": {
    "closest": {
      "timestamp": "20200904180914",
      "status": "200",
      "available": true,
      "url": "http://web.archive.org/web/20200904180914/https://www.whitehouse.gov/"
    }
  },
  "url": "whitehouse.gov"
}

To save a URL on the Wayback Machine:

$ pgark save whitehouse.gov
http://web.archive.org/web/20200904230109/https://www.whitehouse.gov/

To get the JSON response with pgark-snapshot metadata and the Wayback Machine API job status response, pass in -j/--json flag:

$ pgark -j save whitehouse.gov
  {
    "snapshot_status": "success",
    "snapshot_url": "http://web.archive.org/web/20200904230109/https://www.whitehouse.gov/",
    "...": "...",
    "last_job_status": {
      "status": "success",
      "duration_sec": 10.638,
      "job_id": "443e89c2-fd3e-4d01-bd35-abfccc3a124a",
      "...": "..."
    },
    "...": "...",
    "job_url": "http://web.archive.org/status/443e89c2-fd3e-4d01-bd35-abfccc3a124a"
  }

See an example of the Wayback Machine's full JSON response in: examples/web.archive.org/job-save-success.json

Project status

Just spitballing. Will probably just return to forking savepagenow and adding any changes/fixes.

See CHANGELOG for more details

Similar libraries, resources, and inspirations

Development notes

To resync Pipfile.lock and setup.py

  $ pipenv lock --pre
  $ pipenv-setup sync --dev

To run tests:

$ pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pgark-0.0.2.tar.gz (250.0 kB view hashes)

Uploaded Source

Built Distribution

pgark-0.0.2-py2.py3-none-any.whl (12.3 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page