Error Reading PID From File (0 Byte) With Monit After Service Crash

Sometimes services crash badly, in this case I was trying to stabilize an old server running nginx and PHP via Fast-cgi. It crashes quite often, so I set up monit on it – but I noticed monit has stopped doing it’s job due to the following batch of errors:

[06:02:24] error : monit: Error reading pid from file '/var/run/fastcgi-php.pid'
[06:02:25] error : monit: Error reading pid from file '/var/run/fastcgi-php.pid'
[06:02:26] error : monit: Error reading pid from file '/var/run/fastcgi-php.pid'
[06:02:27] error : monit: Error reading pid from file '/var/run/fastcgi-php.pid'
[06:02:28] error : monit: Error reading pid from file '/var/run/fastcgi-php.pid'
[06:02:29] error : monit: Error reading pid from file '/var/run/fastcgi-php.pid'

Now the problem was, the PID file was there, but it was 0 bytes for some reason – so monit couldn’t read from it. I have a fairly regular monit setup for the normal PHP monitoring, something like:

check process php-cgi with pidfile /var/run/fastcgi-php.pid
start program = "/etc/init.d/php5-fcgi start"
stop program = "/etc/init.d/php5-fcgi stop"
if totalcpu > 60% for 2 cycles then alert
if totalcpu > 80% for 5 cycles then restart
if totalmem > 1200.0 MB for 5 cycles then alert
if failed unixsocket /var/run/fcgi.sock then restart
if 3 restarts within 5 cycles then timeout
if failed host localhost port 80 protocol http
and request "/healtcheckfile.php"
then restart

And this worked fine in most cases, but when something happened that caused a 0 byte PID file, monit would totally ignore the process with the following error:

[06:02:50] error : 'php-cgi' failed to start
[06:04:50] error : 'php-cgi' service timed out and will not be checked anymore

The solution to this problem, is to delete the PID file if it’s 0 bytes and restart the service so it creates a new, valid PID file. I achieved this with the following:

check file fastcgi-php.pid with path /var/run/fastcgi-php.pid
if size = 0 then
exec "/bin/rm /var/run/fastcgi-php.pid"
start program = "/etc/init.d/php5-fcgi start"
stop program = "/etc/init.d/php5-fcgi stop"

If you encounter the 0 byte PID after a crash and this works, you should see the following:

[11:21:39] error : 'fastcgi-php.pid' size test failed for /var/run/fastcgi-php.pid -- current size is 0 B
[11:21:39] info : 'fastcgi-php.pid' exec: /bin/rm
[11:23:39] error : 'fastcgi-php.pid' file doesn't exist
[11:23:39] info : 'fastcgi-php.pid' trying to restart
[11:23:39] info : 'fastcgi-php.pid' stop: /etc/init.d/php5-fcgi
[11:23:39] info : 'fastcgi-php.pid' start: /etc/init.d/php5-fcgi
[11:25:39] info : 'fastcgi-php.pid' file exist
[11:25:39] info : 'fastcgi-php.pid' size succeeded

Good luck!

Comments

comments

Subscribe

You can subscribe via e-mail to get my posts in your Inbox, or stalk me on numerous other platforms.

, , , , , , , ,


Comments are closed.
 
Keep up with me on Social Media by following me below - Thanks so much!