Module for remote in-memory Python package/module loading through HTTP
Project description
httpimport
Python's missing feature!
The feature has been suggested in Python Mailing List
Remote, in-memory Python package/module import
ing through HTTP/S
A feature that Python2/3 misses and has become popular in other languages is the remote loading of packages/modules.
httpimport
lets Python2/3 packages and modules to be imported directly in Python interpreter's process memory, through remote URIs
, and more...
Basic Usage
Load package/module accessible through any HTTP/S location
with httpimport.remote_repo('http://my-codes.example.com/python_packages'):
import package1
Load directly from a GitHub/BitBucket/GitLab repo
with httpimport.github_repo('operatorequals', 'httpimport', ref='master'):
import httpimport as httpimport_upstream
# Also works with 'bitbucket_repo' and 'gitlab_repo'
Load a Python module from a Github Gist (using this gist):
url = "https://gist.githubusercontent.com/operatorequals/ee5049677e7bbc97af2941d1d3f04ace/raw/e55fa867d3fb350f70b2897bb415f410027dd7e4"
with httpimport.remote_repo(url):
import hello
hello.hello()
Load a package/module from HTTP/S directly to a variable
module_object = httpimport.load('package1', 'https://my-codes.example.com/python_packages')
module_object
<module 'package1' from 'https://my-codes.example.com/python_packages/package1/__init__.py'>
Load Python packages from archives served through HTTP/S
No file is touching the disk in the process
# with httpimport.remote_repo('https://example.com/packages.tar'):
# with httpimport.remote_repo('https://example.com/packages.tar.bz2'):
# with httpimport.remote_repo('https://example.com/packages.tar.gz'):
# with httpimport.remote_repo('https://example.com/packages.tar.xz'): <-- Python3 Only
with httpimport.remote_repo('https://example.com/packages.zip'):
import test_package
Serving a package through HTTP/S
Any package can be served for httpimport
using a simple HTTP/S Server:
echo 'print("Hello httpimport!")' > module.py
python -m http.server
Serving HTTP on 0.0.0.0 port 8000 ...
>>> import httpimport
>>> with httpimport.remote_repo("http://127.0.0.1:8000"):
... import module
...
Hello httpimport!
Profiles
After v1.0.0
it is possible to set HTTP Authentication, Custom Headers, Proxies and several other things using URL and Named Profiles!
URL Profiles
URL Profiles are INI configurations, setting specific per-URL options, as below:
[http://127.0.0.1:8000]
allow-plaintext: yes ; also 'true' and '1' evaluate to True
[https://example.com]
proxy-url: https://127.0.0.1:8080 ; values must not be in quotes (')
Now, requests to http://127.0.0.1:8000
will be allowed (HTTP URLs do not work by default) and requests to https://example.com
will be sent to an HTTP Proxy.
with httpimport.remote_repo("https://example.com"): # URL matches the URL profile
import module_accessed_through_proxy
Named Profiles
Named Profiles are like URL profiles but do not specify a URL and need to be explicitly used:
[github]
headers:
Authorization: token <Github-Token>
And the above can be used as follows:
with httpimport.github_repo('operatorequals','httpimport-private-test', profile='github'):
import secret_module
Github Tokens look like github_pat_<gibberish>
and can be issued here: https://github.com/settings/tokens/new
NOTE: The values in Profiles MUST NOT be quoted ('
,"
)
Profile Creation
Profiles can be provided as INI strings to the set_profile
function and used in all httpimport
functions:
httpimport.set_profile("""
[profile1]
proxy-url: https://my-proxy.example.com
headers:
Authorization: Basic ...
X-Hello-From: httpimport
X-Some-Other: HTTP header
""")
with httpimport.remote_repo("https://code.example.com", profile='profile1'):
import module_accessed_through_proxy
Advanced
Profiles are INI configuration strings parsed using Python's configparser
(and ConfigParser
for Python2) module.
The ConfigParser
object for httpimport
is the global variable httpimport.CONFIG
and can be used freely:
import httpimport
httpimport.CONFIG.read('github.ini') # Read profiles from a file
with httpimport.github_repo('operatorequals','httpimport-private-test', profile='github'):
import secret_module
Default Profiles
The httpimport
module automatically loads Profiles found in $HOME/.httpimport.ini
and under the $HOME/.httpimport/
directory. Profiles under $HOME/.httpimport/
override ones found in $HOME/.httpimport.ini
.
Profile Options:
Supported
zip-password
-v1.0.0
proxy-url
-v1.0.0
headers
-v1.0.0
allow-plaintext
-v1.0.0
Not yet (subject to change)
-
allow-compiled
-
auth
-
auth-type
-
ca-verify
-
ca-cert
-
tls-cert
-
tls-key
-
tls-passphrase
Debugging...
import httpimport
import logging
logging.getLogger('httpimport').setLevel(logging.DEBUG)
Beware: Huge Security Implications!
Using the httpimport
with HTTP URLs is highly discouraged
As HTTP traffic isn't encrypted and/or integrity checked (unlike HTTPS), it is trivial for a remote attacker to intercept the HTTP responses (via an ARP MiTM probably), and add arbitrary Python code to the downloaded packages/modules. This will directly result in Remote Code Execution on your current user's context!
In other words, you get totally compromised...
You have been warned! Use only HTTPS URLs with httpimport
!
Contributors
- ldsink - The
RELOAD
flag and Bug Fixes - lavvy - the
load()
function - superloach - Deprecation of
imp
module in Python3 in favour ofimportlib
- yanliakos - Bug Fix
- rkbennett - Relative Imports fix, Proxy support
Donations
In case my work helped you, you can always buy me a beer or a liter of gas through the Internet or in case you meet me personally.
In the second case we can talk about any video of Internet Historian or Ordinary Things, while listening to a Lofi Girl Playlist, like the citizens of the Internet that we are.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for httpimport-1.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c1a6520ef31057284329fd2854f9523ae12840779fba60e5c206f3300764892a |
|
MD5 | 553fcd377f2643f6a4381122dcc883a4 |
|
BLAKE2b-256 | 4093b3e69952eaee0db4d5d3e4555f2b1d70ce1d158699a89bb880a1ad421f9d |