6 projects
htmldate
Fast and robust extraction of original and updated publication dates from URLs and web pages.
trafilatura
Python package and command-line tool designed to gather text on the Web, includes all necessary discovery and text processing components to perform web crawling, downloads, scraping, and extraction of main texts, metadata and comments.
courlan
Clean, filter and sample URLs to optimize data collection – includes spam, content type and language filters.
simplemma
A simple multilingual lemmatizer for Python.
py3langid
Fork of the language identification tool langid.py, featuring a modernized codebase and faster execution times.
shoten
...