We complete the actual data purge by deleting entire indices once all records have expired. For example, an index will rollover either when the average shard size breaches a threshold or when the age of the index exceeds the data retention period for the account/event type. We will wait until all documents in the index have expired, then delete the entire index.
Sometimes, the index doesn't rollover itself. In those cases, you can follow the instructions below to roll it over.
1. Use the rollover index API call:
http://<host:9080>/v1/admin/cluster/{cluster}/index/{index}/rollover
2. Complete the request with headers:
curl -XPOST http://<host:9080>/v1/admin/cluster/{cluster}/index/{index}/rollover
-H"Authorization: Basic <Base64 encoded ad.accountmanager.key.controller from events-service-api-store.properties>" -H"Content-Type: application/json" -H"Accept: application/json"
3. Once the command is successfully executed, verify it using the output of the following:
curl http://localhost:9200/_cat/indices?v
In Step 2, instead of ad.accountmanager.key.controller use ad. accountmanager.key.ops from events-service-api-store.properties file. Encode ad.accountmanager.key.ops value using https://www.base64encode.org/
For Event Service using Elastic Search 8, you also need to include the following parameter:
-d '{"numberOfShards": "2"}
Refer to the following article for more insights on ES8 index roll-over: https://community.appdynamics.com/t5/Share-a-tip/How-to-Roll-Over-an-index-manually-for-Event-Servic...