In certain environments, Chef Infra Server setups may be required to forward logging to an adjacent directory on the filesystem beyond /var/log. This may be because /var/log resides on a volume with insufficient space or because a monitoring/logging agent is configured to watch all applications running from a pre-designated location.
This runbook describes the necessary steps to repointing the services which write logs to a new location and persist this arrangement over successive reboots.
This applies to all versions of Chef Infra Server, but precludes testing of those with external services or additional components (such as aws-signing-proxy). Please consult your Chef CA/CSM team if you are concerned about customising your server. Note the example is applied to RHEL based Chef Infra Server instances but the same principle applies across all linux distributions.
Ensure Chef is currently functioning and writing logs to the reconfigured /var/log location:
Find and stop the necessary services from the init system on the instance:
systemctl list-unit-files | egrep "(chef|opscode)"
systemctl stop private_chef-runsvdir-start.service
systemctl stop chef-manage-runsvdir-start.service
Move the existing logs to the new location you have configured (in this example we'll use /opt/opscode_logs):
mv /var/log /opt/opscode_logs
Bind-mount the newly created empty directory and ensure there is an entry in mount:
mount -o bind /opt/opscode_logs /var/log
mount | grep var
(reponse) /opt/opscode_logs on /var/log type none (rw,bind)
At this point you are ready to restart the previously stopped services and start the Chef Infra Server application:
systemctl start private_chef-runsvdir-start.service
systemctl start chef-manage-runsvdir-start.service
To test that logging is now forwarding to the correct location, you can run test and in an adjacent window tail an application log to confirm that the change works:
If you want this to persist across reboots, you’ll just need to update your /etc/fstab with the bind mount as well:
echo '/opt/opscode_logs /var/log none bind 0 0' >> /etc/fstab
cat /etc/fstab | grep "/var/log"
(response) /opt/opscode_logs /var/log none bind 0 0