Installing a Production Pootle 2.5.0 Server on Ubuntu 12.04.3 with MySQL & MemCache

We’re going to install a Production Server configuration of Pootle the ‘Community localization server’ onto a base install of Ubuntu 12.04.3. As part of the base install, you need to have selected the LAMP package option so that MySQL database server installed.

Initial Configuration

Firstly, since we’re going to be doing a lot of package installation, it’s best to move into SuperUser mode:

$ sudo -s

Now let’s start installing all the packages we’re going to need:

root$ apt-get install python-dev libxml2-dev libxslt1-dev

We need Aeidon to support subtitle formats, Levenshtein for speed-up when updating against templates, Xapian for fast text indeing and Zip to provide fast (un)compression of file archives:

root$ apt-get install python-aeidon python-levenshtein libxapian-dev xapian-tools zip

We’re going to install Pootle using Python PIP, but we need to install it first.

root $ apt-get install python-pip

To ensure we do things properly, we’re also going to configure a separate Python environment using VIRTUALENV. Again, we need to install it first:

root$ pip install virtualenv

Now create a virtual environment on your location of choice by issuing the virtualenv command:

root$ virtualenv /var/www/pootle/env/

This will copy the system’s default Python interpreter into your environment. For activating the virtual environment you must run the activate script:

root$ source /var/www/pootle/env/bin/activate

Every time you activate this virtual environment, the Python interpreter will know where to look for libraries. Also notice the environment name will be prepended to the shell prompt.

Install Pootle

Now we’ll download, configure & install Pootle:

This will fetch and install the minimum set of required dependencies as well.

(env)root$ pip install pootle==2.5.0

Note we’re forcing pip to install version 2.5.0. If this is omitted, I’ve noticed that it seems to default to installing 2.5.0-rc1.

If you’ve followed things so far, pip should download pootle and it’s dependancies and be able to build any required libraries without any further input.

At this point, you should be able to type the following:

(env)root$ pootle --version

Which should generate the following information:

Pootle 2.5.0
Translate Toolkit 1.10.0
Django 1.4.8

If everything is okay, we can go ahead & configure our Pootle instance.

Initialise our Pootle Instance

Get Pootle to initialise itself & generate a configuration file:

(env)root$ pootle init

By default this file will be created at ~/.pootle/pootle.conf

You’ll need to make a number of edits to this file during the installation. One we need to make now is to allow the Pootle server to be accessed by any host.

(env)root$ nano ~/.pootle/pootle.conf

 Change the ALLOWED_HOSTS setting to:

ALLOWED_HOSTS = ['*']

By default, DJango limits host logins to the ‘localhost’ only, changing this allows logins from anywhere.

MySQL Database Configuration

We’re going to use MySQL as our backend database. For the purposes of this guide, I’ll assume you’ve already installed and configured your MySQL server.

Install the necessary packages:

(env)root$ apt-get install python-mysqldb libmysqlclient-dev
(env)root$ pip install mysql-python

Now login to your MySQL server to create our database for Pootle:

(env)root$ mysql -u (your mysql username) -p

Enter the following to create the database in your MySQL server:

CREATE DATABASE pootle CHARACTER SET = ‘utf8′;
GRANT ALL PRIVILEGES ON pootle.* TO pootle@localhost IDENTIFIED BY ‘pootlepassword’;
FLUSH PRIVILEGES;

Edit the Pootle configuration file to reflect our database backend:

(env)root$ nano ~/.pootle/pootle.conf

Change the DATABASES section to the following:

# Database backend settings
DATABASES = {
'default': {
# Replace 'sqlite3' with 'postgresql_psycopg2', 'mysql' or 'oracle'.
'ENGINE': 'django.db.backends.mysql',
# Database name or path to database file if using sqlite3.
'NAME': 'pootle',
# Not used with sqlite3.
'USER': 'pootle',
# Not used with sqlite3.
'PASSWORD': 'pootlepassword',
# Set to empty string for localhost. Not used with sqlite3.
'HOST': '',
# Set to empty string for default. Not used with sqlite3.
'PORT': '',
}
}

MemCache Configuration

We’re going to use MemCache to speed things up:

(env)root$ apt-get install memcached
(env)root$ pip install python-memcached

Edit the Pootle configuration file to reflect our cache backend:

(env)root$ nano ~/.pootle/pootle.conf

Change the CACHES section to the following:

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
    'LOCATION': '127.0.0.1:11211'
    }
}

Running Pootle

We need to get Pootle to initialise it’s database & assets:

(env)root$ pootle syncdb
(env)root$ pootle initdb

and also it’s assets:

(env)root$ pootle collectstatic --noinput
(env)root$ pootle assets build

Apache 2 Configuration

We need to ensure we’ve got the mod_wsgi Apache2 module installed:

(env)root$ apt-get install libapache2-mod-wsgi

Create the site definition for Apache:

 

(env)root$ nano /etc/apache2/sites-enabled/pootle

Type the following into the editor:

WSGIRestrictEmbedded On
WSGIPythonOptimize 1

<VirtualHost *:80>
    # Domain for the Pootle server. Use 'localhost' for local deployments.
    #
    # If you want to deploy on example.com/your-pootle/ rather than in
    # my-pootle.example.com/ you will have to do the following changes to
    # this sample Apache configuration:
    #
    # - Change the ServerName directive to:
    #   ServerName example.com
    # - Change the WSGIScriptAlias directive to (note that /your-pootle must
    #   not end with a slash):
    #   WSGIScriptAlias /your-pootle /var/www/pootle/wsgi.py
    # - Change the Alias and Location directives for 'export', and the Alias
    #   directive for 'assets' to include the '/your-pootle'.
    # - Include the following setting in your custom Pootle settings:
    #   STATIC_URL = '/your-pootle/assets/'
    ServerName my-pootle.example.com

    # Set the 'POOTLE_SETTINGS' environment variable pointing at your custom
    # Pootle settings file.
    #
    # If you don't know which settings to include in this file you can use
    # the file '90-local.conf.sample' as a starting point. This file can be
    # found at '/var/www/pootle/env/lib/python2.7/site-packages/pootle/settings/'.
    #
    # Another way to specify your custom settings is to comment this
    # directive and add a new '90-local.conf' file (by copying the file
    # '90-local.conf.sample' and changing the desired settings) in
    # '/var/www/pootle/env/lib/python2.7/site-packages/pootle/settings/'
    # (default location for a pip-installed Pootle, having Python 2.7).
    #
    # This might require enabling the 'env' module.
    SetEnv POOTLE_SETTINGS /var/www/pootle/your_custom_settings.conf


    # The following two optional lines enable the "daemon mode" which
    # limits the number of processes and therefore also keeps memory use
    # more predictable.
    WSGIDaemonProcess pootle processes=2 threads=3 stack-size=1048576 maximum-requests=500 inactivity-timeout=300 display-name=%{GROUP} python-path=/var/www/pootle/env/lib/python2.7/site-packages
    WSGIProcessGroup pootle

    # Point to the WSGI loader script.
    WSGIScriptAlias / /var/www/pootle/wsgi.py

    # Turn off directory listing by default.
    Options -Indexes

    # Set expiration for some types of files.
    # This might require enabling the 'expires' module.
    ExpiresActive On

    ExpiresByType image/jpg "access plus 2 hours"
    ExpiresByType image/png "access plus 2 hours"

    ExpiresByType text/css "access plus 10 years"
    ExpiresByType application/x-javascript "access plus 10 years"

    # Optimal caching by proxies.
    # This might require enabling the 'headers' module.
    Header set Cache-Control "public"

    # Directly serve static files like css and images, no need to go
    # through mod_wsgi and Django. For high performance consider having a
    # separate server.
    Alias /assets /var/www/pootle/env/lib/python2.7/site-packages/pootle/assets
    <Directory /var/www/pootle/env/lib/python2.7/site-packages/pootle/assets>
        Order deny,allow
        Allow from all
    </Directory>

    # Allow downloading translation files directly.
    # This location must be the same in the Pootle 'PODIRECTORY' setting.
    Alias /export /var/www/pootle/env/lib/python2.7/site-packages/pootle/po
    <Directory /var/www/pootle/env/lib/python2.7/site-packages/pootle/po>
        Order deny,allow
        Allow from all
    </Directory>

    <Location /export>
        # Compress before being sent to the client over the network.
        # This might require enabling the 'deflate' module.
        SetOutputFilter DEFLATE

        # Enable directory listing.
        Options Indexes
    </Location>

</VirtualHost>

In order for this to work we need to enable a couple of Apache2 modules:

(env)root$ a2enmod expires
(env)root$ a2enmod headers

Now we need to create the wsgi runner to launch the Pootle application when we access the site:

(env)root$ nano /var/www/pootle/wsgi.py

Type the following into the editor:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import site
import sys


# You probably will need to change these paths to match your deployment,
# most likely because of the Python version you are using.
ALLDIRS = [
    '/var/www/pootle/env/lib/python2.7/site-packages',
    '/var/www/pootle/env/lib/python2.7/site-packages/pootle/apps',
]

# Remember original sys.path.
prev_sys_path = list(sys.path)

# Add each new site-packages directory.
for directory in ALLDIRS:
    site.addsitedir(directory)

# Reorder sys.path so new directories at the front.
new_sys_path = []

for item in list(sys.path):
    if item not in prev_sys_path:
        new_sys_path.append(item)
        sys.path.remove(item)

sys.path[:0] = new_sys_path

# Set the Pootle settings module as DJANGO_SETTINGS_MODULE.
os.environ['DJANGO_SETTINGS_MODULE'] = 'pootle.settings'

# Set the WSGI application.
def application(environ, start_response):
    """Wrapper for Django's WSGIHandler().

    This allows to get values specified by SetEnv in the Apache
    configuration or interpose other changes to that environment, like
    installing middleware.
    """
    try:
        os.environ['POOTLE_SETTINGS'] = environ['POOTLE_SETTINGS']
    except KeyError:
        pass

    from django.core.wsgi import get_wsgi_application
    _wsgi_application = get_wsgi_application()
    return _wsgi_application(environ, start_response)

We need to copy our configuration file into place:

(env)root$ cp ~/.pootle/pootle.conf /var/www/pootle/your_custom_settings.conf

If you have a default Apache website available, you’ll need to remove it:

(env)root$ mv /etc/apache2/sites-enabled/000-default /etc/apache2/sites-available/

Finally, we need to restart Apache to finalise our configuration:

(env)root$ service apache2 restart

Optional Configuration

If you want assistance when doing some of your translation work, Pootle supports using external translation services such as Apertium (free) or Google translate (paid).

If you have an API key for either of these services, they can be entered into the /var/www/pootle/your_custom_settings.conf file and Pootle will utilise these.

Advertisements