Automatic Restart of Services Using Supervisor Daemon in CentOS

#go

Introduction

I recently wrote a simple go backend service and deployed it to the centos server, but the program often hangs for unknown reasons, and it can be restored after restarting the process. For the sake of simplicity, I decided to use supervisor as a process guard for the go service. The goal is to automatically restart the service after it hangs.

Solution

  1. Install supervisor
sudo python -m pip install supervisor
  1. Create a new configuration file

Create a new blank directory

cd /software
mkdir supervisor
cd supervisor

Create a new configuration file using a template

echo_supervisord_conf > supervisord.conf

A supervisord.conf template file will be created in the /software/supervisor directory

  1. Modify the configuration file
vim supervisord.conf

Paste the following configuration at the end of the file

[program:main]
# Set the command to execute in the specified directory
directory=/software/go-ws
# Here is the startup command of the project you want to manage
command=/software/go-ws/main
# Which user to run the process as
# user=root
# Automatically apply this when supervisor starts
autostart=true
# Automatically restart the process after the process exits
autorestart=true
# How long does the process continue to run before it is considered successful
startsecs=1
# number of retries
startretries=5
# stderr log output location
stderr_logfile=/software/go-ws/runtime/stderr.log
# stdout log output location
stdout_logfile=/software/go-ws/runtime/stdout.log
  1. Create a new log file
cd /software/go-ws
mkdir runtime
cd runtime
touch stderr.log
touch stdout.log
  1. Start supervisor

Go to the directory where the supervisord.conf file is located, and use the configuration file to start

cd /software/supervisor
supervisord -c supervisord.conf
  1. View process status
supervisorctl status

Notice

  1. Pay attention to the user configuration of supervisord.conf
# Which user to run the process as
# user=root

Here, if it is ssh logged in by a non-root user, comment out user=root, otherwise it may not start, and the log file will appear

supervisor: couldn't setuid to 0: Can't drop privilege as nonroot user
  1. Both supervisord and supervisorctl need to specify the supervisord.conf file when executing the command

You can enter the directory where the supervisor configuration is located and execute

cd /software/supervisor
supervisorctl -c supervisord.conf status

Or specify directly through the global path

supervisorctl -c /software/supervisor/supervisord.conf status

If you run the command without specifying a path, an error may be prompted

error: <class 'xmlrpclib.ProtocolError'>, <ProtocolError for 127.0.0.1/RPC2: 404 Not Found>: file: /usr/lib/python2.7/site-packages/supervisor/xmlrpc.py line: 545

Common commands

supervisorctl status ---- view the status of the supervisor management process;
supervisorctl stop main ---- used to close a process, programname refers to the flasky in the above example;
supervisorctl start main ---- used to start a process;
supervisorctl restart main ---- used to restart a process;
supervisorctl stop all ---- close all processes;
supervisorctl reload ---- Stop all processes, load the latest configuration file, and start and manage processes according to the latest configuration;
supervisorctl update ---- According to the latest configuration file, start a newly configured process or restart a process with a changed configuration, and the process that has not been changed will not be affected;
supervisorctl shutdown ---- close the supervisor service.

Conclusion

The above is my experience of setting up supervisor to be a process guardian after deploying go service to centos. The tutorial is relatively elementary, welcome to learn and communicate together.

Reference

Comments