Creating a Daemon with Python

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

If you are an average internet user, then you interact with daemons every day. This article will describe what daemons do, how to create them in Python, and what you can use them for.

Daemon Defined

A daemon is a process on UNIX that runs in the background and will typically keep running until it is explicitly told to stop. Examples of daemon processes include web servers (lighttpd and Apache httpd), schedulers (cron and at), DNS servers (bind), mail servers (courier and dovecot), and database servers (MySQL and PostgreSQL).

The typical web user will probably interact with at least one UNIX daemon in the form of a web server, DNS server, or database server on an average day. These types of processes run in the background on a server somewhere, unattended by any person, working tirelessly to do the work that they were designed for.

You can background a process on UNIX by putting an ampersand (&) at the end of the command that you started it with. For example:

dink:~ jmjones$ $(sleep 10; echo echo "WAKING UP";) &
[1] 314
dink:~ jmjones$ WAKING UP

This backgrounded the sleep and echo commands. Ten seconds later, after sleep completed, the command echoed “WAKING UP” and appeared on my terminal. But just running a process in the background doesn’t qualify it for daemon status. There are some deeper technical qualifications that an aspiring process has to meet in order to be branded with the daemon label.

Forking a Daemon Process

Following is the recipe “Forking a Daemon Process on Unix” from The Python Cookbook that will allow your Python code to daemonize itself.

import sys, os
def daemonize (stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
# Perform first fork.
try:
pid = os.fork( )
if pid > 0:
sys.exit(0) # Exit first parent.
except OSError, e:
sys.stderr.write("fork #1 failed: (%d) %sn" % (e.errno, e.strerror))
sys.exit(1)
# Decouple from parent environment.
os.chdir("/")
os.umask(0)
os.setsid( )
# Perform second fork.
try:
pid = os.fork( )
if pid > 0:
sys.exit(0) # Exit second parent.
except OSError, e:
sys.stderr.write("fork #2 failed: (%d) %sn" % (e.errno, e.strerror))
sys.exit(1)
# The process is now daemonized, redirect standard file descriptors.
for f in sys.stdout, sys.stderr: f.flush( )
si = file(stdin, 'r')
so = file(stdout, 'a+')
se = file(stderr, 'a+', 0)
os.dup2(si.fileno( ), sys.stdin.fileno( ))
os.dup2(so.fileno( ), sys.stdout.fileno( ))
os.dup2(se.fileno( ), sys.stderr.fileno( ))

One way that a daemon process differs from a normal backgrounded task is that a daemon process disassociates from its calling process and controlling terminal. This recipe outlines the standard procedure for creating a daemon process. This procedure includes forking once, calling setsid to become a session leader, then forking a second time.

Along the way, it is common to also change directory to / to ensure that the resulting working directory will always exist. It also ensures that the daemon process doesn’t tie up the ability of the system to unmount the filesystem that it happens to be in. It is also typical to set its umask to 0 so that its file creation is set to the most permissive.

After becoming a daemon, this Python example also sets its standard input (stdin), standard output (stdout), and standard error (stderr) to the values the caller specified.

A Good Candidate for Daemonization?

So, why would you want to cause your Python code to daemonize itself?

In general, if you have any code that you want to run for a long time in the background with no user interaction, it is a good candidate for daemonization. You may be wondering, “Why not just use the ampersand (&) like you showed a few examples back?” If you do that, you’re not totally guaranteed that logging out won’t kill that process.

As an example of daemon processes that you may find yourself in need of writing one day, I’ve written some system monitoring utilities that need to check different servers and processes on various intervals. I wanted these system monitoring utilities to run indefinitely in the background and they need no user interaction, so they were a good candidate to let them be daemonized. They are, in fact, running at this very moment monitoring some of my production systems.

Here is a small piece of code that uses the daemonize() function from the previous example.

#!/usr/bin/env python

import daemonize
import time
import logging
import os

curr_dir = os.getcwd()
logfile = os.path.join(curr_dir, 'test_daemon.log')
logging.basicConfig(filename=logfile, level=logging.DEBUG, 
format="%(asctime)s [%(levelname)s] %(message)s")

daemonize.daemonize()

pid_fn = '/var/run/lighttpd.pid'

logging.info('starting')
logging.debug('current directory is %s' % os.getcwd())

while True:
try:
pidfile = open(pid_fn)
pid = int(pidfile.read())
pidfile.close()
logging.debug('got pid %s for %s' % (pid, pid_fn))
except IOError:
logging.warn('IOError on pidfile open')
time.sleep(60)
continue
except ValueError:
logging.warn('ValueError on pidfile open')
time.sleep(60)
continue
try:
stat_fn = os.path.join('/proc', str(pid), 'status')
logging.debug('opening file %s' % stat_fn)
stat_file = open(stat_fn, 'r')
for line in stat_file:
line = line.strip()
if line.startswith('VmRSS'):
logging.info(line)
break
stat_file.close()
time.sleep(60)
except IOError:
logging.warn('IOError on statfile open')
time.sleep(60)
continue

The purpose of this script is to log the memory usage of the lighttpd web server once every minute. This script tries to open the lighttpd pidfile (/var/run/lighttpd.pid) to see what process ID lighttpd has. Then, it opens /proc/{{pid}}/status, where {{pid}} is the process ID of the lighttpd process. (Files in /proc contain tons of system and process-specific information.)

The script then iterates over each line of the status file and logs the first line that starts with the text string “VmRSS”. The “VmRSS” line in the status file shows the amount of memory the process is actively using. The script keeps looping over these steps until it is explicitly killed or it hits an exception I didn’t plan for.

Here is some of the output from running the script:

2008-11-18 05:44:24,627 [INFO] starting
2008-11-18 05:44:24,628 [DEBUG] current directory is /
2008-11-18 05:44:24,631 [DEBUG] got pid 13291 for /var/run/lighttpd.pid
2008-11-18 05:44:24,631 [DEBUG] opening file /proc/13291/status
2008-11-18 05:44:24,631 [INFO] VmRSS:1924 kB
2008-11-18 05:45:24,631 [DEBUG] got pid 13291 for /var/run/lighttpd.pid
2008-11-18 05:45:24,632 [DEBUG] opening file /proc/13291/status
2008-11-18 05:45:24,632 [INFO] VmRSS:1924 kB

Notice that the current directory is “/”. The daemonize() function changed the script’s working directory there as part of the process. I only included two iterations of the loop, but you can see that it ran at 05:44:24, slept for 60 seconds, then ran again at 05:45:24.

Daemonizing a process is something you probably don’t need to do every day. But if you have a need to kick off a process and have it run indefinitely in the background, this approach could prove useful to you.

Also by Jeremy M. Jones:

Building Command Line Utilities with Python

This article was first published on EnterpriseITPlanet.com.