I recently had this annoying issue with Mosquitto (a MQTT broker) that just wouldn't start.

The issue was simple - since version 2.x, Mosquitto will try to write a PID file which by default lives in /var/run/mosquitto/mosquitto.pid (in previous versions it doesn't crash if it can't create this file). This is a tmpfs drive so it resets with boot, and the Mosquitto user doesn't own the parent folder, so it can't create a subfolder in which it can store it's pid file.

The default systemd service definition for Mosquitto in Debian doesn't seem to create this folder before starting the service. The fix is simple, just place this file in /etc/systemd/system/mosquitto.service, run systemctl daemon-reload and restart the service with systemctl restart mosquitto.service.

  [Unit]
  Description=Mosquitto MQTT message broker
  Documentation=https://mosquitto.org/man/mosquitto-8.html
  After=network-online.target
  Wants=network-online.target systemd-networkd-wait-online.service

  [Service]
  Restart=on-failure
  RestartSec=2

  User=mosquitto
  Group=mosquitto

  # Here's the important change
  ExecStartPre=+/bin/mkdir -m 740 -p /var/log/mosquitto
  ExecStartPre=+/bin/chown mosquitto:mosquitto /var/log/mosquitto
  ExecStartPre=+/bin/mkdir -m 740 -p /var/run/mosquitto
  ExecStartPre=+/bin/chown mosquitto:mosquitto /var/run/mosquitto

  ExecStart=/usr/sbin/mosquitto --config-file /etc/mosquitto/mosquitto.conf
  ExecReload=/bin/kill -HUP $MAINPID

  PrivateTmp=true
  PrivateDevices=true
  ProtectHome=true
  ProtectSystem=full

  [Install]
  WantedBy=multi-user.target

…and Bob's your uncle.

Update: I stole the unit file from this issue on GitHub.