Irrational Exuberance: Dreamier Dream Server with Nginx

Dreamier Dream Server with Nginx
July 17, 2007

In my previous setup there were some concerns with the way I was using mod_proxy. The issue is that a request comes in and is received by Apache, if it was a media file Apache would then pass the request to Lighttpd, which would pass the file to Apache, which would then pass the file to the requester. Obviously, this is batshit crazy.

This is why I have reworked my instructions with fewer catastrophic mental lapses. The article has also been reworked for conciseness, and all sections on security have been removed, you can refer back to the original article, but should ideally look for a more specialized resource on securing your server.

The bulk of the process is unchanged, but we will now be using Nginx as a frontend proxy and static media server, and Apache2 with mod_python as the backend server for handling Django. Lighttpd has been dropped in favor of Nginx mostly because Nginx is from Russia, and is thus exotic.

Also relevant is the fact that Lighttpd has an annoying memory leak.
End Product

The end product is still an Ubuntu Feisty server. It will use Nginx as its frontend server (handling media, and proxying other requests to Apache2), and Apache2 as its backend server. It will use memcached for caching, and Postgres8.2 for its database. Django will be our framework of choice.

On to the fun stuff.

Credits and resources can be found in the original article.
Upgrading Ubuntu Dapper to Feisty

Upgrading from Ubuntu Dapper (SliceHost's current Ubuntu OS) to Feisty is quick and painless. You do need to do the upgrades in order: skipping straight to Feisty may make your installation unstable. The final 'lsb_release -a' is simply to confirm that the upgrade has occured. It should confirm that Feisty is installed.

sed -e 's/dapper/ edgy/g' -i /etc/apt/sources.list
apt-get update
apt-get dist-upgrade
apt-get -f install
dpkg --configure -a
reboot

sed -e 's/edgy/ feisty/g' -i /etc/apt/sources.list apt-get update apt-get dist-upgrade apt-get -f install dpkg --configure -a reboot

lsb_release -a

Adding Apache & Other Libraries

Note: psyco_pg2 in the Ubuntu repository appears to be broken, which is why we are installing it with easy_install instead of apt-get.

apt-get install emacs apache2 libapache2-mod-python subversion php5
apt-get install gcc libc6-dev build-essential python-dev python-setuptools
apt-get install python-egenix-mxdatetime memcached postfix
apt-get install postgresql-8.2 postgresql-server-dev-8.2
apt-get install libpcre3-dev nginx
easy_install psycopg2

Setting up nginx

First we need to setup nginx. We will be using the configuration file provided in this article at blog.kovyrin.net. The file is available for download right underneath the sytax highlighted code snippet.

Open up your nginx.conf file:

emacs /etc/nginx/nginx.conf

Then completely delete the contents of the file, and replace them with the contents of the configuration file that we are borrowing from kovyrin.net.

You will have to make several minor changes to the config file:

1. Replace "some-server.com www.some-server.com" with your actual domain.
2. For the static file location change the long regular expression to /media/
3. Change the root directory in the static media location to /var/www/yourdomain.com/

So the first altered part will look like this

server_name yourdomain.com www.yourdomain.com;

(Yes, I acknowledge how totally unhelpful it is to change the example from some-server.com to yourdomain.com.)

And the second portion will look like

location /media/ {
root /var/www/yourdomain.com;
}

We are changing the regular expression to /media/ because we no longer have to perform a number of regular expression matches. Admittedly, the performance increase is probably minimal.

Now lets create a few folders:

mkdir /var/www/yourdomain.com/
mkdir /var/www/yourdomain.com/media

Okay, now we deal with Apache.
Setting up Apache

The first step is to change the port apache is running on:

emacs /etc/apache2/ports.conf

Change it to 8080 instead of 80. Save the file.

Now we need to make our log folder:

mkdir /var/log/apache2/yourdomain.com

Then we'll need to edit our apache config file:

emacs /etc/apache2/sites-available/yourdomain.com

That will initially be an empty file, and you'll be adding this to it:


ServerName www.yourdomain.com
ServerAlias yourdomain.com

DocumentRoot /var/www/yourdomain.com

CustomLog /var/log/apache2/yourdomain.com/access.log combined
ErrorLog /var/log/apache2/yourdomain.com/error.log

SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE yourdjangoproject.settings
PythonDebug Off
PythonPath "['/usr/lib/python2.5/site-packages/django'] + sys.path"


Finally we need to enable the virtual host:

ln -s /etc/apache2/sites-available/yourdomain.com /etc/apache2/sites-enabled/yourdomain.com

Starting nginx and Apache

Now that we have our two servers running, we need to turn them on.

/etc/init.d/nginx start
apache2ctl restart

Go visit www.yourserver.com:8080 and you should get a default Apache page.

Go visit www.yourserver.com and... you should still get a default Apache page. Only this time if you look at the headers you'll be getting it served via nginx.

Now lets double check that nginx is serving static media like we want it to:

apache2ctl stop
emacs /var/www/yourdomain.com/media/test.txt

Write a couple words into the file, save it, and then browse to www.yourserver.com/media/test.txt . It should show your file. Browse to www.yourserver.com, it shouldn't show the apache page.

Now turn Apache back on:

apache2ctl restart

and delete that file

rm /var/www/yourdomain.com/media/test.txt

And our servers are setup.
PostgreSQL

Make sure to use your own password instead of just 'password' in the code below.

su postgres -c psql template1
ALTER USER postgres WITH PASSWORD 'password';
\q

And then we edit the pg_hba.conf file.

emacs /etc/postgresql/8.2/main/pg_hba.conf

Go to the end of the file and comment out all lines that start with host (unless you will be accessing your database remotely). Finally the local line should look like

local all all password

Finally we'll want to restart Postgres:

/etc/init.d/postgresql-8.2 restart

Configuring memcached

Running memcached is easy:

memcached -u www-data -p 11211 -m 32 -d

On Ubuntu the user www-data is the user that runs the webserver, making it very similar to the apache user on some other distributions. Running memcached with the -u www-data option means that we'll be running memcached with the same user as the webserver. Port 11211 is the default for memcached, and probably should not be changed unless you are running multiple memcached instances. I chose to start my memcached instance with 32 megs of memory because my slice only has 256 meg total, and my Django app simply doesn't have very much information to cache.

Next we need to get python-memcached, which is a python memcached client. There is an alternate cmemcached library for Python that is twice as fast as python-memcached, but I had trouble getting it to compile (I believe because I installed memcached from a repository instead of from source). Python-memcached is easy to get:

wget ftp://ftp.tummy.com/pub/python-memcached/python-memcached-1.36.tar.gz
tar -zxvf python-memcached-1.36.tar
cd python-memcached-1.36
python setup.py install

And that should be that.

At this exact moment (July 12, 2007) the current svn version of Django is broken with python-memcached. A patch has been submitted and is working its way through the submission system, so hopefully this issue will resolve itself soon. If anyone is affected by this situation, send me an email or leave a comment and I'll update this guide to include the required modifications.
Setting up Django, etc

First lets create a non-root user, it'll hold all our django files. For these examples it will be user django.

useradd django
passwd django

Now we are going to setup Django. We will first create a postgres user for our Django app:

sudo su postgres
createuser -P pg_django
# should not be a superuser
createdb --encoding=UNICODE db_yourtable -O pg_django
exit

Hint: Write down the database table, user and password we'll be using them again in a couple of minutes, just far enough in the future to completely forget them all.

Now we need to give the www-data user access to our files (www-data is the user that runs the web server).

gpasswd -a www-data django
chmod g+w ~django

Then we'll want to create some folders for Django. You can feel free to play with the folder layout, its mostly a personal thing, but you'll have to keep any changes you make in mind when you follow the remaining instructions.

su django
cd
mkdir projects
mkdir apps
mkdir templates
mkdir src
cd src

Now we check out the Django source and link the checked out source into the site-packages so that the python interpreter can find it.

svn co http://code.djangoproject.com/svn/django/trunk
# log off as user django, revert to root
exit
cd ~django/src/
ln -s `pwd`/trunk/django /usr/lib/python2.5/site-packages/
ln -s `pwd`/trunk/django/contrib/admin/media/ /var/www/yourdomain.com/media/admin

Now we get to create our first Django project. Notice how we symlink the project into the site-packages folder so that it available in the python path. This is a matter of preference, you can alteratively add the path to the project to the Apache virtual host file we edited earlier.

su django
cd ~/projects
~/src/trunk/django/bin/djang-admin.py startproject myproject
exit
ln -s `pwd`/myproject /usr/lib/python2.5/site-packages/

Now edit your newly minted settings.py file.

su django
emacs ~/projects/myproject/settings.py

You'll need to make a number of changes, these are the lines you will need to add (not alter) to your settings.py:

CACHE_BACKEND = 'memcached://127.0.0.1:112211/'
CACHE_MIDDLEWARE_SECONDS = 60 * 60
CACHE_MIDDLEWARE_KEY_PREFIX = ''
CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True

I personally use a very long middleware cache because my pages don't change much (I am serving a blog, and thus caching entire pages is not too problematic), the standard value is much lower, around 300. The key prefix is used to distinguish caches between multiple Django projects using the same caching backend. If you are only planning to have one project using your caching backend then it is fine to leave the key as an empty string. If you plan on having multiple projects, each should have a unique key prefix. Finally, the anonymous only option means that logged in users will not recieve cached pages. For my application, where only the admin will ever be logged in, this is an appropriate setting, your milleage may vary.

And here are the already existing lines you will need to modify in settings.py:

DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'db_yourtable'
DATABASE_USER = 'pg_django'
DATABASE_PASSWORD = 'whatever_you_chose'
MEDIA_ROOT = '/home/django/media/'
MEDIA_URL = 'http://yourdomain.com/media/'
ADMIN_MEDIA_PREFIX = '/media/admin/'

MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.middleware.http.ConditionalGetMiddleware',
'django.middleware.gzip.GZipMiddleware',
'django.middleware.cache.CacheMiddleware',
'django.middleware.doc.XViewMiddleware',
)

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
)

Extending this Django setup later: you will eventually want to add your own templates and your own media along with your own appl ication. To provide access to your media files you will want to create a symlink from wherever they are into your /var/www/yourdomain.com/media folder like this:

# log off as django
exit
ln -s /home/django/yourapp/media/yourapp /var/www/yourdomain.com/media/

For templates you'll just need to append a path to your template directory to the TEMPLATE_DIRS variable in the setting s.py file. Configuring Django applications and projects is sometimes more of an art than a science. Go try painting for a while, but feel free to ask for help if you need it.

Quick Fix To For Apache & Python Egg Compatibility

Because we installed psycopg2 via a python egg we will need to add a couple lines to our Apache config file. This process is explained on the Django page for modython, but is fairly simple.

First make a file to hold a few lines of Python:

mkdir /var/www/.eggs
chown www-data /var/www/.eggs
emacs /home/django/eggs.py

Now eggs.py should contain these lines:

import os
os.environ['PYTHON_EGG_CACHE'] = '/var/www/.eggs'

Then we need to open our virtual host file

emacs /etc/apache2/sites-available/yourdomain.com

And modifying the file to look like this:


ServerName www.yourdomain.com
ServerAlias yourdomain.com

DocumentRoot /var/www/yourdomain.com

CustomLog /var/log/apache2/yourdomain.com/access.log combined
ErrorLog /var/log/apache2/yourdomain.com/error.log

PythonInterpreter my_django PythonImport /home/django/eggs.py my_django

SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE yourdjangoproject.settings
PythonDebug Off
PythonPath "['/usr/lib/python2.5/site-packages/django'] + sys.path"


Save the file and now Apache and eggs should play nicely together.
Weeping with Joy

We're done. Of course your server isn't secure at all, you should go do something about that. While working through these instructions you probably ran into points where you thought "What the hell is wrong with this guy?" and also "This doesn't work... at all." Well I hope you remember those moments, because I'd appreciate knowing which spots caused emotional distress (so I can idly dream about fixing them).
Social Links: Reddit Del.icio.us StumbleUpon Digg
Comments
Andrew Kember

I've just read your great tutorial (but not yet implemented it - let me get home first) and noticed what might be a small error:

Finally we need to enable the virtual host:
ln -s /etc/apache2/sites-available/yourdomain.com.com /etc/apache2/sites-enabled/yourdomain.com

I imagine that yourdomain.com.com should only have one .com?

I'm looking forward to putting all this to use. Thanks for going to all the trouble of writing the article.
Will

Andrew,

Sorry about that, you are right: its definitely a typo. Let me know if there are any other areas that are confusing and I'll update it.
Ben

This looks like a useful tutorial, but I'm stuck right off the bat... Both memcached and nginx refused to install: E: Couldn't find package memcached E: Couldn't find package nginx

I'm very new to Ubuntu. Is there a repository setup piece I missed?

Thanks! -Ben
Ben

Ok, so I had to enable the "universal" repositories for apt in /etc/apt/sources.list.

Then, nginx wouldn't start up. This was because apache was running on port 80. I assume I can ignore this until I get all this configured.

Onward...
ODB

I have a 512 slice that I upgraded to feisty as per your instructions. I'm a command line wimp, however, and for some reason the "apt-get install..." commands aren't working for:

memcached libpre3-dev nginx

I'm getting an error:

Couldn't find package package-name

Is this happening for anyone else? All the other stuff is installing fine.
ben

ODB,

See the two posts above...

By the way, it's working beautifully now. Thanks, Will.

-Ben
Nathaniel Martin

So, I've enabled the universal repos, but after this command:

sudo apt-get install libpre3-dev nginx

i get:

Reading package lists... Done Building dependency tree Reading state information... Done E: Couldn't find package libpre3-dev

Any ideas?
Will

There is information on the package here.

It seems that the real problem is that I committed a typo and dropped the c. The actual package is 'libpcre3-dev', not 'libpre3-dev'.

Sorry for the inconvenience, I will be fixing the article asap.

Irrational Exuberance: Dreamier Dream Server with Nginx

Blogged with Flock