serviceform 0.9.5

settings.py modifications

In your Django application’s settings.py perform the following modifications. Add following applications to INSTALLED_APPS:

INSTALLED_APPS = [
    'grappelli' # optional but recommended, needs django-grappelli

    ...

    'nested_admin',
    'compressor',
    'crispy_forms',
    'guardian',
    'serviceform.serviceform',
    'serviceform.tasks',
    'select2'
]

Settings for Django-compress:

COMPRESS_PRECOMPILERS = (
    ('text/x-scss', 'django_libsass.SassCompiler'),
)

STATICFILES_FINDERS = ("django.contrib.staticfiles.finders.FileSystemFinder",
                       "django.contrib.staticfiles.finders.AppDirectoriesFinder",
                       "compressor.finders.CompressorFinder",
)

Settings for crispy-forms:

CRISPY_TEMPLATE_PACK = 'bootstrap3'

Settings for Django guardian:

AUTHENTICATION_BACKENDS = (
    'django.contrib.auth.backends.ModelBackend', # default
    'guardian.backends.ObjectPermissionBackend',
)

Database settings, we need postgresql!:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'serviceform',
        'USER': 'serviceform',
        'PASSWORD': 'django',
        'HOST': 'db',
        'PORT': 5433,
    }

We need to set up named caches:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'default',
    },
    'persistent': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'persistent',
    },
    'cachalot': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'cachalot',
    }
}

In production, use redis or similar instead at least for persistent cache!

Set up static root:

STATIC_ROOT = os.path.join(BASE_DIR, 'static')

Set up serviceform specific settings:

SERVER_URL="http://localhost:8000"
# random capital letters that are used to generate unpredictable
# links from ids
CODE_LETTERS='ABCDE'

urls.py modifications

Add urls that are specific to grappelli (optional), nested_admin, select2 and serviceform:

urlpatterns = [
    url(r'^admin/', admin.site.urls),

    url(r'^_grappelli/', include('grappelli.urls')), # optional
    url(r'^_nested_admin/', include('nested_admin.urls')),
    url(r'^_select2/', include('select2.urls')),

    url(r'', include('serviceform.serviceform.urls')),
]

Requirements

• Machine that runs docker

• Your own web server with SSL sertificates and associated domain name

• Sendgrid email account for automatic sending emails. Your domain DNS settings need to be set up correctly for sendgrid too.

• (optional) Sentry / sentry account

Docker environment file

Put environment variables in file serviceform-env.list:

PRODUCTION=1
# You can choose your credentials here. Initial database will be made according to these
# settings
POSTGRES_USER=serviceform
POSTGRES_DB=serviceform
POSTGRES_PASSWORD=django
# Django's secret key. Use generator such as this:
# http://www.miniwebtool.com/django-secret-key-generator/
SECRET_KEY=asdf
# API key to Sendgrid email sending service.
SENDGRID_API_KEY=asdf
# Sentry authentication. Leave this out if you don't have Sentry account.
RAVEN_DSN=https://asdf
# Your service will be at https://SERVICEFORM_HOST
SERVICEFORM_HOST=yourhost.com
ADMIN_NAME=Your Name
ADMIN_EMAIL=your.name@yourhost.com
SERVER_EMAIL=noreply@yourhost.com
# This code is used to generate unpredictable id, choose 5 random letters here
CODE_LETTERS=ABCDE
# Available: en, fi
LANGUAGE_CODE=fi
TIME_ZONE=Europe/Helsinki

For the following commands set first environment variable

export SERVICEFORM_ENV_FILE=/path_to/serviceform-env.list

External services

Docker commands to start external services needed by Serviceform

Postgresql:

docker run -d --name serviceform-db \
         --env-file $SERVICEFORM_ENV_FILE \
         --volume serviceform-db:/var/lib/postgresql \
         postgres:9.6.2

Redis:

docker run -d --name serviceform-redis \
         --volume serviceform-redis:/data \
         redis:3.2.8-alpine

Django services

Docker commands to start services bundled within serviceform docker image.

Build serviceform docker image first:

docker build -t tuomasairaksinen/serviceform:latest .

Or alternatively, pull it from the repository:

docker pull tuomasairaksinen/serviceform:latest

docker run --rm -u root \
        --link serviceform-db:db \
        --link serviceform-redis:redis \
        --env-file $SERVICEFORM_ENV_FILE \
        --volume serviceform-media:/code/media \
        --volume serviceform-static:/code/static \
        --volume serviceform-nginx-config:/nginx-config \
        --volume serviceform-celery-beat-store:/celery-beat-store \
        tuomasairaksinen/serviceform:latest upgrade

docker run -d --name serviceform-celery \
         --link serviceform-db:db \
         --link serviceform-redis:redis \
         --env-file $SERVICEFORM_ENV_FILE \
         tuomasairaksinen/serviceform:latest celery

docker run -d --name serviceform-celery-beat \
        --link serviceform-db:db \
        --link serviceform-redis:redis \
        --volume serviceform-celery-beat-store:/store \
        --env-file $SERVICEFORM_ENV_FILE \
        tuomasairaksinen/serviceform:latest celery-beat

docker run -d --name serviceform-task-processor \
         --link serviceform-db:db \
         --link serviceform-redis:redis \
         --env-file $SERVICEFORM_ENV_FILE \
         tuomasairaksinen/serviceform:latest task-processor

docker run -d --name serviceform-send-emails \
        --link serviceform-db:db \
        --env-file $SERVICEFORM_ENV_FILE \
        tuomasairaksinen/serviceform:latest send-emails

docker run -d --name serviceform-app \
        --link serviceform-db:db \
        --link serviceform-redis:redis \
        --env-file $SERVICEFORM_ENV_FILE \
        --volume serviceform-static:/code/static:ro \
        --volume serviceform-media:/code/media \
        tuomasairaksinen/serviceform:latest app

docker run -d --name serviceform-nginx \
        --publish 8038:80 \
        --link serviceform-app:app \
        --volume serviceform-static:/serviceform-static:ro \
        --volume serviceform-media:/serviceform-media:ro \
        --volume serviceform-nginx-config:/etc/nginx/conf.d:ro \
        nginx:1.13-alpine

Shutting down and starting (system reboot procedures)

Shutting down:

docker stop serviceform-nginx serviceform-app serviceform-send-emails \
            serviceform-task-processor serviceform-celery-beat serviceform-celery \
            serviceform-redis serviceform-db

Starting again (set this into your system startup). Notice order.:

docker start serviceform-db serviceform-redis serviceform-celery serviceform-celery-beat \
             serviceform-task-processor serviceform-send-emails serviceform-app \
             serviceform-nginx

Upgrading system

Simple upgrade procedure:

docker pull tuomasairaksinen/serviceform:latest
docker stop serviceform-nginx serviceform-app serviceform-send-emails \
        serviceform-task-processor serviceform-celery-beat serviceform-celery

Run upgrade command. If that is fine, we can remove old containers:

docker rm serviceform-nginx serviceform-app serviceform-send-emails \
        serviceform-task-processor serviceform-celery-beat serviceform-celery

Then run all docker run all services and http_server.

Zero-downtime upgrade method is planned in the future.

Troubleshooting / shell access

To investigate problems these shell commands might prove usefull.

Django shell:

docker run --rm -it \
        --link serviceform-db:db \
        --link serviceform-redis:redis \
        --env-file $SERVICEFORM_ENV_FILE \
        tuomasairaksinen/serviceform:latest shell

Postgresql root shell:

docker exec -it -u postgres serviceform-db psql

Same with Django’s credentials:

docker run --rm -it \
        --link serviceform-db:db \
        --link serviceform-redis:redis \
        --env-file $SERVICEFORM_ENV_FILE \
        tuomasairaksinen/serviceform:latest dbshell

Bash shell (to investigate/edit volumes etc.):

docker run --rm -it -u root \
        --link serviceform-db:db \
        --link serviceform-redis:redis \
        --volume serviceform-media:/code/media:ro \
        --volume serviceform-static:/code/static \
        --volume serviceform-nginx-config:/nginx-config \
        --env-file $SERVICEFORM_ENV_FILE \
        tuomasairaksinen/serviceform:latest bash

Dumping/loading production data as/from sql

docker exec -u postgres serviceform-db pg_dump serviceform > backup.sql

DROP DATABASE serviceform;
CREATE DATABASE serviceform;

docker stop serviceform-db
docker rm serviceform-db
docker volume rm serviceform-db

docker exec -u postgres serviceform-db psql serviceform < backup.sql

Running tests with docker-compose

Run:

docker-compose -f docker-compose-tests.yml run tests

Running staging system with docker-compose

Run:

docker-compose -f docker-compose-staging.yml run upgrade # initialize everything
docker-compose -f docker-compose-staging.yml run upgrade createsuperuser
docker-compose -f docker-compose-staging.yml up -d

then go to http://localhost:8080 and log in.

How to set things up and run your local development environment:

Install dependencies:

sudo apt-get install docker.io git python-dev python-pip virtualenv libpq-dev postgresql-server-dev-all virtualenvwrapper

Note: Python 3.6 or newer is required.

Create virtualenv:

mkvirtualenv -p /usr/bin/python3.6 serviceform_env

To start using it type:

workon serviceform_env

Install requirements to your virtualenv:

pip install -r requirements.txt

Run external services (redis and postgresql) inside docker:

docker-compose up

When DB is set up, you can run initial migrations with command:

./manage.py migrate

Then you must create your initial account:

./manage.py createsuperuser

Then run can run development server:

./manage.py runserver

Then open browser in http://localhost:8000 and use your initial superuser account to log in.

Dumping and loading database in development environment

Database can be dumped with the following command:

docker-compose exec -u postgres db pg_dump serviceform > init.sql

To load dump, you must first clear the current database. This can be done as follows:

docker-compose exec -u postgres -i db psql serviceform < init.sql

Dump data in json format for tests:

./manage.py dumpdata -o tests/test_data.json -e serviceform.EmailMessage -e admin.LogEntry --indent 2 -e sessions.Session

Translations

If changes to translatable strings are made, run:

cd serviceform
django-admin.py makemessages

Then update translation (.po) files for example with poedit, and then run:

django-admin.py compilemessages

Then commit your changes (.po and .mo files) to repository.