arXiv Loader

This tool is a wrapper of the arXiv API that allows you to retrieve metadata of articles published on arXiv.
Please abide by the Terms of Usage of the arXiv API.

Installation

pip install arxivloader

Usage

Please consult the arXiv API documentation for help in constructing a valid query string.

Searching by keyword

To search for a keyword the query needs to start with search_query= followed by a prefix and the search word.
Possible prefixes are

Please have a look at the arXiv API documentation for details.

import arxivloader

keyword = "DustPy"
prefix = "all"
query = "search_query={prefix}:{keyword}".format(prefix, keyword)
columns = ["id", "title", "authors"]

df = arxivloader.load(query, columns=columns)
print(df)

Searching by id

To search for a specific arXiv ID the query needs to start with id_list= followed by a comma-separated list of arXiv IDs:

import arxivloader

IDs = ["1909.04674", "1909.10526"]
query = "id_list={}".format(",".join(IDs))
columns = ["id", "title", "authors"]

df = arxivloader.load(query, columns=columns)

print(df)

Filtering specific articles by keywords

If both, search_query= and id_list= are present, the given arXiv articles are filtered by the give key word.

import arxivloader

keyword = "DSHARP"
prefix = "ti"
IDs = ["1909.04674", "1909.10526"]
query = "search_query={pf}:{kw}&id_list={ids}".format(pf=prefix, kw=keyword, ids=",".join(IDs))
columns = ["id", "title", "authors"]

df = arxivloader.load(query, columns=columns)

print(df)

Searching by category

It is possible to search large number of articles by category. Please be responsible with the traffic this query causes.

import arxivloader

keyword = "astro-ph.EP"
prefix = "cat"
query = "search_query={pf}:{kw}".format(pf=prefix, kw=keyword)
columns = ["id", "title", "primary_category", "categories", "published"]

df = arxivloader.load(query, columns=columns, sortBy="submittedDate", sortOrder="descending", num=1000, page_size=100)

print(df.head(5))

Acknowledgements

Thank you to arXiv for use of its open access interoperability.