topic Re: Forcing eventgen to refresh normalised replacement files in Splunk Search

Forcing eventgen to refresh normalised replacement files

bowesmana — Wed, 25 Sep 2019 23:33:56 GMT

I am using eventgen to generate transaction type data, where I create an event in Splunk and then at some point in the future, I create a new event to revoke the initial event. It works like this

my first event is using a guid in the event
a saved search runs periodically and searches for a select set of these events and writes the guids to an outputlookup
this csv is then used by another eventgen sample to create the second revocation transaction

However, it seems that as eventgen randomises the replacement file at startup, when the saved search rewrites the outputlookup, eventgen is not aware of that change, so will continue to use the file. This means that the same guid can be potentially reused in (2) and new guids created in (1) are not candidates.

I can see this in the log file

Normalized replacement file /opt/splunk/etc/apps/app/lookups/guids.csv

is there a way to force this to refresh?

Re: Forcing eventgen to refresh normalised replacement files

lwu_splunk — Thu, 26 Sep 2019 02:40:50 GMT

TLDR:
You can not force it to use the updated replacement file.

Long answer:
EventGen is designed for large data volume generation. So EventGen will read the replacement file once and cache it for event token replacement. Think if when you generate every event, you read from file and replace it with a random value from the file, it will have unacceptable I/O latency.

If you think it is critical for your requirement, you can make a feature request here: https://github.com/splunk/eventgen/issues/new/choose

Thanks for your feedback.

Re: Forcing eventgen to refresh normalised replacement files

bowesmana — Thu, 26 Sep 2019 07:17:55 GMT

Thanks. That's as I suspected, so no big deal. I am shutting down my instance daily for the night and I don't need nightly data, so I have saved searches that will update the file before shutdown, so will be in place for the next restart.