Fixing missing 2018 Reporting partition tables

Will Fisher -

Users that are have installed Reporting < 1.7.8 may see an increase in the disk space usage on their Chef server database. 

This is due to an issue with older versions of reporting failing to create data partition tables after 2018 and prevents the the "opscode-reporting remove-partitions" command from being able to clean up old reporting data.  There is also a bug in the way data was migrated from the node_run and node_run_details tables for 2016-12 and 2017-12 which may have left some old records in those tables in addition to any data that is not migrated due to missing partition tables.

These instructions are intended to help with migrating reporting data into the new partition tables and to then clean up the node_run and node_run_details tables. If the migration fails you will need to contact Chef Support (support@chef.io) to walk you through completing the process.

Requirements to apply the fix

  • chef-server >= 12.14.0
  • Enough disk space to duplicate the node_run and node_run_detail tables contents during the data migration.

Download the migration scripts

The migration scripts are available at https://tinyurl.com/y86mtgrb and you can preview them by opening the url in your browser.

git clone https://tinyurl.com/y86mtgrb reporting-migration
cd reporting-migration

There is also a "Download ZIP" button on the page for the above link if you cannot use git to download the migration scripts.

Installing new reporting version

  1. Download Reporting >= 1.8.0 for your OS/System from https://downloads.chef.io/reporting/stable
  1. Install the new opscode-reporting package on the chef-server

# Ubuntu example
dpkg -i opscode-reporting_1.8.0-1_amd64.deb
# RHEL/CENTOS 7 example
rpm -Uvh opscode-reporting-1.8.0-1.el7.x86_64.rpm

# reconfigure reporting and chef-server
opscode-reporting-ctl reconfigure
chef-server-ctl reconfigure

Performing reporting data migration

NOTE: This will replicate data records in the database while the migration is being performed so will need to make sure there is enough free space to duplicate the node_run and node_run_details tables.

  1. Check that there is enough free space to perform the migration. Run the following command and look for the size of the node_run and node_run_details tables and make sure there is enough free space to duplicate that data. The duplicate data will get cleaned up in one of the last steps.
echo '\d+' | su - opscode-pgsql -c 'psql opscode_reporting'
  1. We’ll need to make sure no new data is being ingested into the system while the data is being migrated.
chef-server-ctl stop nginx
  1. Use the following command to migrate existing data into the correct partition tables.
/opt/opscode/embedded/bin/ruby migrate_2018.rb
  1. Attach the new partitions:
cat node_run_final_migrate.sql | su - opscode-pgsql -c "psql -d opscode_reporting -a"
  1. Start NGINX and test that the chef-server is working properly.
chef-server-ctl start nginx
# from a client node
chef-client
  1. The old partition tables will be kept around with a suffix of _backup. When you are comfortable with the migration you can drop these tables.
Have more questions? Submit a request

Comments

Powered by Zendesk