Skip to content

Make each child process log to its own file

Matjaz Gregoric requested to merge mtyaka/multilog into master

Child processes managed by honcho (web, websocket, worker, periodic) were all logging to the same (log/main.log) file using logging.handlers.RotatingFileHandler, which does not work correctly when shared by multiple processes.

We use the HONCHO_PROCESS_NAME environment variable exposed by honcho to subprocess environments to log output of each process to its own file:

log/im.web.1.log
log/im.web.2.log
log/im.websocket.1.log
...

There is also log/im.main.log which is used to log output from the main (parent) honcho process.

Testing

  1. Checkout this PR and do pip install -r requirements.txt.
  2. Remove existing main.log files from the log folder: rm log/main.log*. The log folder should now be empty.
  3. Start the IM: make rundev.
  4. Look at the log/ folder which should now contain im.main.log, im.web.1.log, im.websocket.1.log, and im.worker.1.log. Output is buffered so some of the files may be empty or not contain a lot of output.
  5. Stop the IM, and run IM in production mode: make run WORKERS=4.
  6. The log folder should now contain these additional files: im.periodic1.log, im.worker.2.log, im.worker.3.log, and im.worker.4.log.

Merge request reports