Splunk Search

Forcing eventgen to refresh normalised replacement files

bowesmana
SplunkTrust
SplunkTrust

I am using eventgen to generate transaction type data, where I create an event in Splunk and then at some point in the future, I create a new event to revoke the initial event. It works like this

  1. my first event is using a guid in the event
  2. a saved search runs periodically and searches for a select set of these events and writes the guids to an outputlookup
  3. this csv is then used by another eventgen sample to create the second revocation transaction

However, it seems that as eventgen randomises the replacement file at startup, when the saved search rewrites the outputlookup, eventgen is not aware of that change, so will continue to use the file. This means that the same guid can be potentially reused in (2) and new guids created in (1) are not candidates.

I can see this in the log file

Normalized replacement file /opt/splunk/etc/apps/app/lookups/guids.csv

is there a way to force this to refresh?

Tags (1)
0 Karma
1 Solution

lwu_splunk
Splunk Employee
Splunk Employee

TLDR:
You can not force it to use the updated replacement file.

Long answer:
EventGen is designed for large data volume generation. So EventGen will read the replacement file once and cache it for event token replacement. Think if when you generate every event, you read from file and replace it with a random value from the file, it will have unacceptable I/O latency.

If you think it is critical for your requirement, you can make a feature request here: https://github.com/splunk/eventgen/issues/new/choose

Thanks for your feedback.

View solution in original post

lwu_splunk
Splunk Employee
Splunk Employee

TLDR:
You can not force it to use the updated replacement file.

Long answer:
EventGen is designed for large data volume generation. So EventGen will read the replacement file once and cache it for event token replacement. Think if when you generate every event, you read from file and replace it with a random value from the file, it will have unacceptable I/O latency.

If you think it is critical for your requirement, you can make a feature request here: https://github.com/splunk/eventgen/issues/new/choose

Thanks for your feedback.

bowesmana
SplunkTrust
SplunkTrust

Thanks. That's as I suspected, so no big deal. I am shutting down my instance daily for the night and I don't need nightly data, so I have saved searches that will update the file before shutdown, so will be in place for the next restart.

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...