Skitai App Engine
Project description
Copyright (c) 2015 by Hans Roh
License: BSD
Introduce
Skitai App Engine (SAE) is a kind of branch of Medusa Web Server - A High-Performance Internet Server Architecture.
Medusa is different from most other servers because it runs as a single process, multiplexing I/O with its various client and server connections within a single process/thread.
SAE orients light-weight,simplicity and strengthen networking operations with external resources - HTTP / HTTPS / RPC / PostgreSQL - keeping very low costs.
It also influenced by Zope and Flask a lot.
It can be run as XML/JSON-RPC, Web and Proxy & Loadbancing Server.
It can request massive RESTful api/RPC/HTTP(S) connections based on asynchronous socket framework at your apps easily.
It provides asynchronous PostgreSQL query execution
Skitai is not a framework for convinient developing, module reusability and plugin flexibility etc. It just provides some powerful communicating services for your apps as both server and client.
Purpose
Skitai App Engine’s original purpose is to serve python fulltext search engine Wissen which is my another pypi work. And recently I found that it is possibly useful for building and serving python written websites like Flask and Django (But I barely use them). I think you already use them, you really don’t need use Skitai.
Anyway, I am modifying my codes to optimizing for enabling service on Linux machine with relatvely poor H/W and making easy to auto-scaling provided cloud computing service like AWS.
If you need lots of outside http(s) resources connecting jobs and use PostgreSQL, it might be worth testing and participating this project.
Performance
I roughly benchmarked with uwsgi+nginx & gevent WSGI server by ping-pong (simple request-reply) test. In spite doubting needs for benchmarking results, I do all affort to making same conditions for each servers as possible. All testing tools, readme and results is in: [skitai-install-dir]/tools/benchmark/ or at GitLab.
At a Glance
Basic Configure
[server]
processes = 1
threads = 4
port = 5000
[routes:line]
; mount filesystem directory for static files
/images = /home/skitaid/app/images
; mount skitai apps, path/module:callable
/ = /home/skitaid/app/webapp:app
; mount WSGI apps, path/module:callable
/wsgi-hello = /home/skitaid/app/wsgiapp:appHello World
Then write /home/skitaid/app/webapp.py
from skitai.server import ssgi
app = ssgi.Application (__name__)
app.set_devel (True)
@app.route ("/hello")
def hello (was):
return 'Hello World'For RPC Map-Reducing
; add mysearch members to config file
[@mypypi]
ssl = yes
members = pypi1.python.org:443, pypi2.python.org:443@app.route ("/search")
def search (was, keyword = "Mozart"):
s = was.map ("@mypypi/pypi")
s.search (keyword)
results = s.getswait (timeout = 2)
all_results = []
for result in results:
if result.status == 3:
all_results.extend (result.data)
return all_resultsRPC Load-Balancing
@app.route ("/search")
def search (was, keyword = "Mozart"):
s = was.lb ("@mypypi/pypi")
s.search (keyword)
return s.getwait (timeout = 2)For PostgreSQL Map-Reducing
; add mydb members to config file
[@mydb]
type = postresql
members = s1.yourserver.com:5432/mydb/user/passwd,s2.yourserver.com:5432/mydb/user/passwd@app.route ("/query")
def query (was, keyword):
s = was.dmap ("@mydb")
s.execute("SELECT * FROM CITIES;")
results = s.getswait (timeout = 2)
all_results = []
for result in results:
if result.status == 3:
all_results.append (result.data)
return all_resultsRequirements
Win 32
pywin32 binary - http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/
Optional Requirements
Skitaid can find at least one DNS server from system configuration for Async-DNS query. Possibly it is only problem on dynamic IP allocated desktop, then set DNS manually, please.
psycopg2 for querying PostgreSQL asynchronously (win32 binary)
Jinja2 for HTML Rendering
Install & Start Skitai Server
Posix
sudo pip install skitai
sudo skitaid.py -v &
sudo skitaid.py stop
;if everythig is OK,
sudo service skitaid start
#For auto run on boot,
sudo update-rc.d skitaid defaults
or
sudo chkconfig skitaid onWin32
sudo pip install skitai
cd c:\skitaid\bin
skitaid.py -v
skitaid.py stop (in another command prompt)
;if everythig is OK,
install-win32-service install
#For auto run on boot,
install-win32-service --startup auto install
install-win32-service startChanges
Support Lower Level WSGI Apps
/home/skitaid/app/webapp.py
__DEBUG__ = True # for auto reload on change
def mywsgi (env, start_response):
start_response ("200 OK", [("Content-Type", "text/plain")])
return ['pong']At /etc/skitaid/servers-enabled/default.conf:
[routes:line]
; WSGI app, path/module:callable
/mywsgi = /home/skitaid/app/webapp:mywsgiConfig changed
In [server] section, from proxy = yes|no, to enable_proxy = yes|no
M2Crypto dependency has been removed, and [certification] section had been entirely removed.
[server]
ssl = yes
; added new key
certfile = server.pem
; you can combine to certfile
; keyfile = private.key
; passphrase =To genrate self-signed certification file:
openssl req -new -newkey rsa:2048 -x509 -keyout server.pem -out server.pem -days 365 -nodesNew services added
# streaming response for stream objects like file, large string list...
# object should have read() and optioanl close() method
return was.tostream (open ("large-movie.mp4", "rb"))
# email delivery service
e = was.email (subject, snd, rcpt)
e.set_smtp ("127.0.0.1:465", "username", "password", ssl = True)
e.add_text ("Hello World<div><img src='cid:ID_A'></div>", "text/html")
e.add_attachment (r"001.png", cid="ID_A")
e.send ()With asynchronous email delivery service, can add default SMTP Server config to skitaid.conf (/etc/skitaid/skitaid.conf or c:skitaidetcskitaid.conf). If it is configured, you can skip e.set_smtp(). But be careful for keeping your smtp password.
[smtpda]
smtpserver = 127.0.0.1:25
user =
password =
ssl = no
max_retry = 10
undelivers_keep_max_days = 30Documentation
Please visit https://gitlab.com/hansroh/skitai/wikis/home
Change Log
0.9.5 - strengthen WSGI support.
0.9.4.21 - add tools
0.9.4.19 - no threads mode, then can config threads=0, but cannot use all async restful requests
0.9.4.17 - fix application reload in py27
0.9.4.16 - fix GZIP decompressing
0.9.4.13 - enable pass phrase on CA signed certification
0.9.4.9 - remove dependency M2Crypto
0.9.4.8 - fix error handling for asynconnect.connect
0.9.4.6 - improve handling winsock ENOTCONN
0.9.4.5 - fix asyndns, asyncon reconnect
0.9.4.1 - fix init.d script
0.9.4 - (1)works on Python 3, but unstable yet (2)add was.email() (3)improve rpc performance
0.9.3.7 - add skitaid-install-requirements.sh for psycopg2 and M2Crypto
0.9.3.2 - change default log and var path
0.9.2 - fix multipart file upload
0.9.1.32 - fix reverse proxy Host header
0.9.1.30 - was.wget, was.rest (former was.rpc), was.map, was.lb specification changed. see Documentation.
0.9.1.28 - add HEAD, DELETE, OPTIONS methods
0.9.1.27 - support Map-Reduce, Load_Balanace for (json & xml serialized object over) HTTP Call
0.9.1.25 - fix xmlrpc server uri, I misunderstood all xmlrpc uri is /rpc2
0.9.1.24 - possibly fixed, “too many file descriptors in select()”
0.9.1.23 - add some methods to was.request, add “X-Forwarded-For” to proxy_handler
0.9.1.19 - new was member, was.response
0.9.1.14 - Automation session commit
0.9.1.12 - Fix / App Routing
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
