Building a python daemon process Source

June 18, 2012 Markdown source

1---
2title: 'Building a python daemon process'
3date: '2012-06-18'
4published_at: '2012-06-18T13:50:00.003+10:00'
5tags: ['daemon', 'init scripts', 'logging', 'programming', 'python']
6author: 'Gavin Jackson'
7excerpt: 'This blog post is a quick introduction to building a python daemon process. This sample builds a true Unix daemon process, with full logging and an optional init script (which has been tested on SLES...'
8updated_at: '2012-06-18T13:51:54.342+10:00'
9legacy_url: 'http://www.gavinj.net/2012/06/building-python-daemon-process.html'
10---
11
12![](/assets/imported/2012-06-18-building-python-daemon-process/image-1.png)
13
14This blog post is a quick introduction to building a python daemon process. This sample builds a true Unix daemon process, with full logging and an optional init script (which has been tested on SLES 11 SP2 but should work on other Redhat based distributions).
15
16A Unix daemon has some fundamental characteristics (it's a lot more than just a background process), these are listed in *PEP 3143 Standard Daemon Process Library *[http://www.python.org/dev/peps/pep-3143/#correct-daemon-behaviour](http://www.python.org/dev/peps/pep-3143/#correct-daemon-behaviour).
17
18Cron may be a suitable alternative to this approach, however a daemon allows you to have a single, long lived process that can maintain state, allows you to send process signals to, a guarantee that you only have one instance in execution (which can be a gotcha with poorly written cron scripts that don't implement lock files), by modifying the INIT INFO section of the init script you can also control where in the system boot up the daemon commences execution.
19
20###  Step 1 - Required Libraries
21
22Install the Python Daemon package from ([http://pypi.python.org/pypi/python-daemon/](http://pypi.python.org/pypi/python-daemon/)) - note that you will also need an earlier version of the python lockfile (version 0.8) to work ([http://pypi.python.org/pypi/lockfile/0.9.1](http://pypi.python.org/pypi/lockfile/0.9.1)).
23
24###  Step 2 - Sample Code
25
26`# To kick off the script, run the following from the python directory:`
27
28```
29#   PYTHONPATH=`pwd` python testdaemon.py start
30```
31
32```
33#standard python libs
34import logging
35```
36
37```
38import time
39
40#third party libs
41```
42`from daemon import runner`
43
44`class App():`
45
46```
47
48    def __init__(self):
49        self.stdin_path = '/dev/null'
50        self.stdout_path = '/dev/tty'
51        self.stderr_path = '/dev/tty'
52```
53****`self.pidfile_path = '/var/run/testdaemon/testdaemon.pid'`
54
55```
56        self.pidfile_timeout = 5
57```
58
59```
60    def run(self):
61
62        while True:
63            #Main code goes here ...
64```
65
66```
67            #Note that logger level needs to be set to logging.DEBUG before this shows up in the logs
68```
69
70`logger.debug("Debug message")`
71
72```
73            logger.info("Info message")
74```
75
76```
77            logger.warn("Warning message")
78```
79
80`logger.error("Error message")`
81
82```
83            time.sleep(10)
84```
85
86```
87app = App()
88logger = logging.getLogger("DaemonLog")
89logger.setLevel(logging.INFO)
90formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
91```
92****`handler = logging.FileHandler("/var/log/testdaemon/testdaemon.log")`
93
94```
95handler.setFormatter(formatter)
96logger.addHandler(handler)
97```
98
99```
100daemon_runner = runner.DaemonRunner(app)
101```
102
103```
104#This ensures that the logger file handle does not get closed during daemonization
105daemon_runner.daemon_context.files_preserve=[handler.stream]
106daemon_runner.do_action()
107```
108
109Save this script under **/usr/share/testdaemon/testdaemon.py**
110
111You need to ensure that the **/var/log/testdaemon** and** /var/run/testdaemon** folders exist and the user running the daemon process has write permissions.
112
113###  **Step 3 - Init Script**
114
115You can run the script by running **python testdaemon.py start**
116
117This will spawn the daemon process. You can stop it by running **daemon.py stop**
118
119The following init script will allow you to start and stop the daemon on system startup and shutdown respectively:
120
121```
122#! /bin/bash
123```
124
125`# Copyright (c) 1996-2012 My Company.`
126
127```
128# All rights reserved.
129#
130
131# Author: Bob Bobson, 2012
132```
133`#`
134
135```
136# Please send feedback to bob@bob.com
137#
138
139# /etc/init.d/testdaemon
140#
141
142### BEGIN INIT INFO
143# Provides: testdaemon
144# Required-Start:
145# Should-Start:
146# Required-Stop:
147# Should-Stop:
148# Default-Start:  3 5
149# Default-Stop:   0 1 2 6
150# Short-Description: Test daemon process
151# Description:    Runs up the test daemon process
152### END INIT INFO
153```
154
155```
156# Activate the python virtual environment
157```
158****`. /path_to_virtualenv/activate`
159
160```
161case "$1" in
162  start)
163    echo "Starting server"
164
165    # Start the daemon
166```
167`python``/usr/share/testdaemon/testdaemon.py``start`
168
169```
170    ;;
171  stop)
172    echo "Stopping server"
173
174    # Stop the daemon
175```
176
177`python``/usr/share/testdaemon/testdaemon.py``stop`
178
179`;;`
180
181```
182  restart)
183    echo "Restarting server"
184```
185
186```
187    python /usr/share/testdaemon/testdaemon.py restart
188```
189
190`;;`
191
192```
193  *)
194
195    # Refuse to do other stuff
196    echo "Usage: /etc/init.d/testdaemon.sh {start|stop|restart}"
197    exit 1
198    ;;
199esac
200```
201
202```
203exit 0
204```
205
206Note that this is using python virtualenv to use an isolated python environment (please refer to my previous blog entry [here](http://www.gavinj.net/2012/05/python-virtualenv-is-your-friend.html)).
207
208Save file in **/etc/init.d/testdaemon**
209
210**chmod u+x ****/etc/init.d/testdaemon**
211
212Enable using **chkconfig testdaemon on**
213
214Check that it is enabled **chkconfig --list**
215
216...
217
218testdaemon 0:off 1:off 2:off **3:on** 4:off **5:on** 6:off
219
220...
221
222Kick it off by running **service testdaemon start**
223
224You will now see it running in the background:
225
226```
227ps -aux|grep testdaemon
228root     15225  0.0  0.0  64304  5948 ?        S    13:10   0:00 python /usr/share/testdaemon/testdaemon.py start
229```
230
231###  Step 4 - Example Usage
232
233```
234(pyenv-zenkai)rama:/usr/share/testdaemon # python ./testdaemon.py start
235(pyenv-zenkai)rama:/usr/share/testdaemon # started with pid 14565
236```
237
238```
239(pyenv-zenkai)rama:/usr/share/testdaemon # cat /var/log/testdaemon/testdaemon.log
2402012-06-18 12:57:22,512 - DaemonLog - INFO - Info message
2412012-06-18 12:57:22,512 - DaemonLog - WARNING - Warning message
2422012-06-18 12:57:22,512 - DaemonLog - ERROR - Error message
2432012-06-18 12:58:19,770 - DaemonLog - INFO - Info message
2442012-06-18 12:58:19,771 - DaemonLog - WARNING - Warning message
2452012-06-18 12:58:19,771 - DaemonLog - ERROR - Error message
2462012-06-18 12:58:29,779 - DaemonLog - INFO - Info message
2472012-06-18 12:58:29,779 - DaemonLog - WARNING - Warning message
2482012-06-18 12:58:29,780 - DaemonLog - ERROR - Error message
249```
250
251...
252
253```
254(pyenv-zenkai)rama:/usr/share/testdaemon # python ./testdaemon.py stop
255Terminating on signal 15
256(pyenv-zenkai)rama:/usr/share/testdaemon #
257```
258
259
260

---
title: 'Building a python daemon process'
date: '2012-06-18'
published_at: '2012-06-18T13:50:00.003+10:00'
tags: ['daemon', 'init scripts', 'logging', 'programming', 'python']
author: 'Gavin Jackson'
excerpt: 'This blog post is a quick introduction to building a python daemon process. This sample builds a true Unix daemon process, with full logging and an optional init script (which has been tested on SLES...'
updated_at: '2012-06-18T13:51:54.342+10:00'
legacy_url: 'http://www.gavinj.net/2012/06/building-python-daemon-process.html'
---

![](/assets/imported/2012-06-18-building-python-daemon-process/image-1.png)

This blog post is a quick introduction to building a python daemon process. This sample builds a true Unix daemon process, with full logging and an optional init script (which has been tested on SLES 11 SP2 but should work on other Redhat based distributions).

A Unix daemon has some fundamental characteristics (it's a lot more than just a background process), these are listed in *PEP 3143 Standard Daemon Process Library *[http://www.python.org/dev/peps/pep-3143/#correct-daemon-behaviour](http://www.python.org/dev/peps/pep-3143/#correct-daemon-behaviour).

Cron may be a suitable alternative to this approach, however a daemon allows you to have a single, long lived process that can maintain state, allows you to send process signals to, a guarantee that you only have one instance in execution (which can be a gotcha with poorly written cron scripts that don't implement lock files), by modifying the INIT INFO section of the init script you can also control where in the system boot up the daemon commences execution.

###  Step 1 - Required Libraries

Install the Python Daemon package from ([http://pypi.python.org/pypi/python-daemon/](http://pypi.python.org/pypi/python-daemon/)) - note that you will also need an earlier version of the python lockfile (version 0.8) to work ([http://pypi.python.org/pypi/lockfile/0.9.1](http://pypi.python.org/pypi/lockfile/0.9.1)).

###  Step 2 - Sample Code

`# To kick off the script, run the following from the python directory:`

```
#   PYTHONPATH=`pwd` python testdaemon.py start
```

```
#standard python libs
import logging
```

```
import time

#third party libs
```
`from daemon import runner`

`class App():`

```

    def __init__(self):
        self.stdin_path = '/dev/null'
        self.stdout_path = '/dev/tty'
        self.stderr_path = '/dev/tty'
```
****`self.pidfile_path = '/var/run/testdaemon/testdaemon.pid'`

```
        self.pidfile_timeout = 5
```

```
    def run(self):

        while True:
            #Main code goes here ...
```

```
            #Note that logger level needs to be set to logging.DEBUG before this shows up in the logs
```

`logger.debug("Debug message")`

```
            logger.info("Info message")
```

```
            logger.warn("Warning message")
```

`logger.error("Error message")`

```
            time.sleep(10)
```

```
app = App()
logger = logging.getLogger("DaemonLog")
logger.setLevel(logging.INFO)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
```
****`handler = logging.FileHandler("/var/log/testdaemon/testdaemon.log")`

```
handler.setFormatter(formatter)
logger.addHandler(handler)
```

```
daemon_runner = runner.DaemonRunner(app)
```

```
#This ensures that the logger file handle does not get closed during daemonization
daemon_runner.daemon_context.files_preserve=[handler.stream]
daemon_runner.do_action()
```

Save this script under **/usr/share/testdaemon/testdaemon.py**

You need to ensure that the **/var/log/testdaemon** and** /var/run/testdaemon** folders exist and the user running the daemon process has write permissions.

###  **Step 3 - Init Script**

You can run the script by running **python testdaemon.py start**

This will spawn the daemon process. You can stop it by running **daemon.py stop**

The following init script will allow you to start and stop the daemon on system startup and shutdown respectively:

```
#! /bin/bash
```

`# Copyright (c) 1996-2012 My Company.`

```
# All rights reserved.
#

# Author: Bob Bobson, 2012
```
`#`

```
# Please send feedback to bob@bob.com
#

# /etc/init.d/testdaemon
#

### BEGIN INIT INFO
# Provides: testdaemon
# Required-Start:
# Should-Start:
# Required-Stop:
# Should-Stop:
# Default-Start:  3 5
# Default-Stop:   0 1 2 6
# Short-Description: Test daemon process
# Description:    Runs up the test daemon process
### END INIT INFO
```

```
# Activate the python virtual environment
```
****`. /path_to_virtualenv/activate`

```
case "$1" in
  start)
    echo "Starting server"

    # Start the daemon
```
`python``/usr/share/testdaemon/testdaemon.py``start`

```
    ;;
  stop)
    echo "Stopping server"

    # Stop the daemon
```

`python``/usr/share/testdaemon/testdaemon.py``stop`

`;;`

```
  restart)
    echo "Restarting server"
```

```
    python /usr/share/testdaemon/testdaemon.py restart
```

`;;`

```
  *)

    # Refuse to do other stuff
    echo "Usage: /etc/init.d/testdaemon.sh {start|stop|restart}"
    exit 1
    ;;
esac
```

```
exit 0
```

Note that this is using python virtualenv to use an isolated python environment (please refer to my previous blog entry [here](http://www.gavinj.net/2012/05/python-virtualenv-is-your-friend.html)).

Save file in **/etc/init.d/testdaemon**

**chmod u+x ****/etc/init.d/testdaemon**

Enable using **chkconfig testdaemon on**

Check that it is enabled **chkconfig --list**

...

testdaemon 0:off 1:off 2:off **3:on** 4:off **5:on** 6:off

...

Kick it off by running **service testdaemon start**

You will now see it running in the background:

```
ps -aux|grep testdaemon
root     15225  0.0  0.0  64304  5948 ?        S    13:10   0:00 python /usr/share/testdaemon/testdaemon.py start
```

###  Step 4 - Example Usage

```
(pyenv-zenkai)rama:/usr/share/testdaemon # python ./testdaemon.py start
(pyenv-zenkai)rama:/usr/share/testdaemon # started with pid 14565
```

```
(pyenv-zenkai)rama:/usr/share/testdaemon # cat /var/log/testdaemon/testdaemon.log
2012-06-18 12:57:22,512 - DaemonLog - INFO - Info message
2012-06-18 12:57:22,512 - DaemonLog - WARNING - Warning message
2012-06-18 12:57:22,512 - DaemonLog - ERROR - Error message
2012-06-18 12:58:19,770 - DaemonLog - INFO - Info message
2012-06-18 12:58:19,771 - DaemonLog - WARNING - Warning message
2012-06-18 12:58:19,771 - DaemonLog - ERROR - Error message
2012-06-18 12:58:29,779 - DaemonLog - INFO - Info message
2012-06-18 12:58:29,779 - DaemonLog - WARNING - Warning message
2012-06-18 12:58:29,780 - DaemonLog - ERROR - Error message
```

...

```
(pyenv-zenkai)rama:/usr/share/testdaemon # python ./testdaemon.py stop
Terminating on signal 15
(pyenv-zenkai)rama:/usr/share/testdaemon #
```