Elasticsearch Template Mishaps

Sean Horn -

Elasticsearch indices can have mappings applied by templates as they are created. These mappings determine how particular pieces of an indexed document are stored in the search index.

If an elasticsearch index loses its mappings somehow, it will fall back to a default template. This can cause havoc in an Automate installation. The logstash service is supposed to update ES with the mappings for a given index type when the logstash service starts up. However, even if the logstash service is restarted and successfully uploads the correct template, the current day's index will not have the proper mapping as templates are only applied during an index creation. 

A bad(for Automate's purposes) default mapping is applied here

[2018-01-11T17:58:14,995][INFO ][o.e.c.m.MetaDataCreateIndexService] [MMOCHFELS01P] [insights-2018.01.12] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings []

Compare that with an index creation on my working system:

[2018-01-26T18:47:19,716][INFO ][o.e.c.m.MetaDataCreateIndexService] [automate] [insights-2018.01.26] creating index, cause [auto(bulk api)], templates [insights], shards [5]/[1], mappings [_default_]

That _default_ mapping is the one defined in that logstash config that sets entity_uuid to a searchable keyword instead of just text.

A workaround to get the proper template applied to the index for the current day is to make sure logstash is restarted, then delete the current day's index, and notice that that means any new data that comes in will result in the creation of a new daily index using the correct template....which should result in the newly created index having the proper mapping applied.

Zendesk 17644

Have more questions? Submit a request

Comments

Powered by Zendesk