For those who need to extract text to some depth over a website. Scraping Tasks
Project description
Digs
Just making easy the text crawling task over websites.
Usage
You want to extract all text data from a url website to your current directory:
digs http://thewebsite.com
You can add the option –depth=LEVEL to perform over the root domain (website) a search to get all the text data:
digs http://thewebsite.com --depth=3
Be careful, with a high level, the tree search grows exponentially.
Also, you can use the graphical interface (if you have installed PySide):
digs -i
You should something like this:
Installation
pip install digs
Requirements
requirements.txt
digs was written by Jonathan S. Prieto C..
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
digs-0.1.1.tar.gz
(13.9 kB
view hashes)