NSSM and Symfony Scheduler

Recently I updated a project to use Symfony Scheduler to handle scheduled background tasks like sending emails and running data checks. In order for this to work, it is necessary run the messenger:consume console command for the scheduler.

This project is hosted on Windows Server, so to do this I decided to use NSSM to create a service which runs this command. I installed NSSM, then used it to define a new service to run this command. After configuring and starting the service, I tested the messenger:stop-workers command to check that the consumer process would stop and NSSM would re-start it as it should.

What I found, was that the consumer process would exit, but NSSM was not restarting it. Instead, it just sat there and windows showed the service as still running since NSSM was still running. This was not good, as this means if my script crashes, it won't restart and all my scheduled tasks would stop working. This is not the first time I've used NSSM for something like this, so I was confused why it was not working when it had in the past.

The fix

It turns out that this is an issue if you are using IO Redirection and either redirect stdout and stderr to the same file or have the Timestamp Logs option enabled. When the service is configured this way NSSM opens a pipe for these streams and then hangs when trying to close the pipe.

Reconfiguring the service to not use IO redirection or to redirect stderr and stdout to separate files without timestamping will resolve the issue and get NSSM correctly restarting the service on exit.

Finding the problem

Looking at the logs in event viewer didn't really help with determining why this was happening. There were no logs related to it not wanting to restart the process. Since the source code for NSSM is available on its website, I decided to download it and start looking through it to determine why it might be failing to restart the process.

In the source code, I eventually found the end_service function which is called when the program exits and handles restarting the program. I saw there were a couple conditions that would prevent it from restarting, so to debug the problem I added extra logging events to this function and compiled a new version of nssm and used that for my service.

After replicating the problem, I found that the function ran up until the clean_logging function but then quit. I added some more debug logging to this function to see where it was running into issues and re-compiled nssm. With this new version I was able to determine the program was getting hung on the close_handle call for stdout_pipe.

I don't know why this is happening, but it clued me into the fact that the issue is probably related to the IO logging configuration. I searched the code for where this handle is set up and found it is only setup when a few conditions are true like if the stdout and stderr files are the same, if timestamping is checked or .... I decided to re-configure my service so that none of these conditions were met and try again. This time, the process was successfully restarted.