Skip to main content
PyCon US is happening May 14th-22nd in Pittsburgh, PA USA.  Learn more

Extension of Python HTTP `requests` with verbose logging using `logging-mv-integrations`.

Project description

.. -*- mode: rst -*-

requests-mv-integrations
------------------------

Extension of Python HTTP `requests <https://pypi.python.org/pypi/requests>`_ with verbose
logging using `logging-mv-integrations <https://pypi.python.org/pypi/logging-mv-integrations>`_.


Badges
------

.. start-badges

.. list-table::
:stub-columns: 1

* - docs
- |docs| |license|
* - info
- |hits| |contributors|
* - tests
- |travis| |coveralls|
* - package
- |version| |supported-versions|
* - other
- |requires|


.. |docs| image:: https://readthedocs.org/projects/requests-mv-integrations/badge/?style=flat
:alt: Documentation Status
:target: http://requests-mv-integrations.readthedocs.io

.. |hits| image:: http://hits.dwyl.io/TuneLab/requests-mv-integrations.svg
:alt: Hit Count
:target: http://hits.dwyl.io/TuneLab/requests-mv-integrations

.. |contributors| image:: https://img.shields.io/github/contributors/TuneLab/requests-mv-integrations.svg
:alt: Contributors
:target: https://github.com/TuneLab/requests-mv-integrations/graphs/contributors

.. |license| image:: https://img.shields.io/badge/License-MIT-yellow.svg
:alt: License Status
:target: https://opensource.org/licenses/MIT

.. |travis| image:: https://travis-ci.org/TuneLab/requests-mv-integrations.svg?branch=master
:alt: Travis-CI Build Status
:target: https://travis-ci.org/TuneLab/requests-mv-integrations

.. |coveralls| image:: https://coveralls.io/repos/TuneLab/requests-mv-integrations/badge.svg?branch=master&service=github
:alt: Code Coverage Status
:target: https://coveralls.io/r/TuneLab/requests-mv-integrations

.. |requires| image:: https://requires.io/github/TuneLab/requests-mv-integrations/requirements.svg?branch=master
:alt: Requirements Status
:target: https://requires.io/github/TuneLab/requests-mv-integrations/requirements/?branch=master

.. |version| image:: https://img.shields.io/pypi/v/requests_mv_integrations.svg?style=flat
:alt: PyPI Package latest release
:target: https://pypi.python.org/pypi/requests_mv_integrations

.. |supported-versions| image:: https://img.shields.io/pypi/pyversions/requests-mv-integrations.svg?style=flat
:alt: Supported versions
:target: https://pypi.python.org/pypi/requests-mv-integrations

.. end-badges


Install
-------

.. code-block:: bash

pip install requests_mv_integrations


Architecture
------------

``requests-mv-integrations`` is an extension of the `Python package requests <https://pypi.python.org/pypi/requests>`_
and it is used for TUNE Multiverse Integrations for handling all HTTP requests including APIs in REST and SOAP,
screen scrapping, and handling response downloads in JSON, XML, and CSV.

.. image:: ./images/requests_mv_integrations.png
:scale: 50 %
:alt: UML requests-mv-integrations


Usage
-----

.. code-block:: python

URL_TUNE_MAT_API_COUNTRIES = \
'https://api.mobileapptracking.com/v2/countries/find.json'

from requests_mv_integrations import (
RequestMvIntegrationDownload,
)
request_download = RequestMvIntegrationDownload(logger_level=logging.DEBUG)

result = \
request_download.request(
request_method='GET',
request_url=URL_TUNE_MAT_API_COUNTRIES,
request_params=None,
request_retry=None,
request_headers=HEADER_CONTENT_TYPE_APP_JSON,
request_label="TMC Countries"
)

json_tune_mat_countries = result.json()

pprint(json_tune_mat_countries)


Example
^^^^^^^

.. code-block:: bash

$ python3 examples/example_request.py

{
"asctime": "2017-10-13 12:02:53 -0700",
"levelname": "INFO",
"name": "__main__",
"version": "00.05.04",
"message": "Start"
}
{
"asctime": "2017-10-13 12:02:53 -0700",
"levelname": "DEBUG",
"name": "requests_mv_integrations",
"version": "00.05.04",
"message": "TMC Countries: Start"
}
...
{
"asctime": "2017-10-13 12:02:53 -0700",
"levelname": "DEBUG",
"name": "requests_mv_integrations",
"version": "00.05.04",
"message": "TMC Countries: Details",
"request_data": "",
"request_headers": {
"Content-Type": "application/json",
"User-Agent": "(requests-mv-integrations/00.05.04, Python/3.6.2)"},
"request_label": "TMC Countries",
"request_method": "GET",
"request_params": {},
"request_url": "https://api.mobileapptracking.com/v2/countries/find.json",
"timeout": 60
}
{
"asctime": "2017-10-13 12:02:53 -0700",
"levelname": "DEBUG",
"name": "requests_mv_integrations",
"version": "00.05.04",
"message": "TMC Countries: Curl",
"request_curl": "curl --verbose
-X GET
-H 'Content-Type: application/json'
-H 'User-Agent: (requests-mv-integrations/00.05.04, Python/3.6.2)'
--connect-timeout 60
-L 'https://api.mobileapptracking.com/v2/countries/find.json'",
"request_label": "TMC Countries",
"request_method": "GET"
}
...
{
'data': [
{'id': 0, 'name': 'International (Generic)'},
{'id': 4, 'name': 'Afghanistan'},
{'id': 8, 'name': 'Albania'},
{'id': 10, 'name': 'Antarctica'},
{'id': 12, 'name': 'Algeria'},
{'id': 16, 'name': 'American Samoa'},
{'id': 20, 'name': 'Andorra'},
{'id': 24, 'name': 'Angola'},
{'id': 28, 'name': 'Antigua And Barbuda'},
{'id': 31, 'name': 'Azerbaijan'},
],
'response_size': '845',
'status_code': 200,
}


Classes
-------

- ``class RequestMvIntegration`` -- Base class using `requests <https://pypi.python.org/pypi/requests>`_ with retry functionality and verbose logging.
- ``class RequestMvIntegrationDownload`` -- Download file handling.
- ``class RequestMvIntegrationUpload`` -- Upload file handling.

Requirements
------------

``requests-mv-integrations`` module is built upon Python 3 and has dependencies upon
several Python modules available within `Python Package Index PyPI <https://pypi.python.org/pypi>`_.

.. code-block:: bash

make install

or

.. code-block:: bash

python3 -m pip uninstall --yes --no-input -r requirements.txt
python3 -m pip install --upgrade -r requirements.txt


Packages
^^^^^^^^

- **beautifulsoup4**: https://pypi.python.org/pypi/beautifulsoup4
- **deepdiff**: https://pypi.python.org/pypi/deepdiff
- **logging-mv-integrations**: https://pypi.python.org/pypi/logging-mv-integrations
- **pyhttpstatus-utils**: https://pypi.python.org/pypi/pyhttpstatus-utils
- **requests**: https://pypi.python.org/pypi/requests
- **safe-cast**: https://pypi.python.org/pypi/safe-cast


.. :changelog:

Release History
===============

0.7.7 (2018-02-16)
------------------
- migrate to github/TuneLab

0.7.6 (2018-02-07)
--------------------
- Updated requirements.txt

0.7.5 (2018-01-28)
------------------
- Upgraded to use pyhttpstatus-utils 0.2.2.

0.7.4 (2018-01-25)
------------------
- migrate to github/tuneinc

0.7.2 (2017-12-10)
---------------------
- readthedocs.org

0.6.0 (2017-11-24)
---------------------
- README

0.5.7 (2017-10-31)
---------------------
- README

0.5.6 (2017-10-26)
---------------------
- Using formatted string literals

0.5.5 (2017-09-12)
---------------------
- logger_output: STDOUT, STDOUT_COLOR, FILE
- logger_filter: NOTE

0.5.4 (2017-09-12)
---------------------
- Support logging-mv-integrations refactor

0.5.3 (2017-08-13)
---------------------
- Fix Retry exception functions

0.5.2 (2017-06-15)
---------------------
- Requirements.txt - update python packages to latest

0.5.1 (2017-06-15)
---------------------
- Collect metrics for MvRequest calls

0.5.0 (2017-04-07)
---------------------
- support client side certificate

0.4.9 (2017-04-01)q
---------------------
- Requirements.

0.4.8 (2017-03-30)
-------------------
- Retry exception function.

0.4.6 (2017-03-28)
-------------------
- logger-mv-integrations version 0.1.2
- validate_response() and validate_json_response()

0.4.5 (2017-03-14)
-------------------
- smart-cast

0.4.4 (2017-03-12)
-------------------
- use sessions cookies as request_auth

0.4.3 (2017-03-02)
-------------------
- replace 'json' with 'ujson'

0.4.2 (2017-02-27)
-------------------
- psutil
- Back to Python 3.5

0.4.1 (2017-02-09)
-------------------
- cUrl request params

0.4.0 (2017-02-09)
-------------------
- Requirements.
- RetryError

0.3.9 (2017-02-06)
-------------------
- TuneRequest is not a Singleton.

0.3.8 (2017-02-03)
-------------------
- Upgrade to Python 3.6

0.3.5 (2017-01-27)
-------------------
- Reduce logging
- Disc usage
- Iron.io Partition

0.3.3 (2017-01-26)
-------------------
- Cleanup using pyflakes, pep8, pylint, and yapf
- Requirements

0.3.2 (2017-01-24)
-------------------
- Curl fix

0.3.1 (2017-01-18)
-------------------
- Log reduction

0.2.9 (2017-01-17)
------------------
- psutil: disc and mem usage
- Cleanup

0.2.8 (2017-01-16)
------------------
- Cleanup yapf

0.2.7 (2017-01-16)
------------------
- Build Curl
- Cleanup

0.2.6 (2017-01-14)
------------------
- Requirements

0.2.5 (2017-01-04)
------------------
- Cleanup

0.2.4 (2017-01-04)
------------------
- request timeout

0.2.3 (2016-12-31)
------------------
- test_curl
- parse_curl

0.2.2 (2016-12-30)
------------------
- Makefile

0.2.1 (2016-12-29)
------------------
- README.rst
- HISTORY.rst

0.2.0 (2016-12-19)
------------------
- Expose response url

0.1.8 (2016-12-09)
------------------
- Base Exception cleanup

0.1.6 (2016-12-08)
------------------
- Error Origin

0.1.5 (2016-12-07)
------------------
- Log Exit Code

0.1.1 (2016-12-03)
------------------
- Include request_auth in curl

0.1.0 (2016-11-25)
------------------
* Initial Release

0..1 (2016-11-18)
------------------
* Conception
* Code pulled from TuneLab/tune-mv-integration-python

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page