Eventgen: Is there a known bug with backfill and replaytimestamp?

[japger_splunk]

I am trying to model transactions such that they are replayed using similar timing between events that are contained in the sample data file. The pause between each "block" of transactions also needs to be spaced out which is being correctly handled (every 600 seconds) in the below eventgen.conf file when ignoring the backfill. The token substitutions are also working as desired.

PROBLEM: The problem shows up when I try to backfill these events over a previous window. It seems that the backfill process does not account for the 600 second pause and is defaulting back to a 1 second spacing between blocks of events. Is this a known bug? Any ideas for a workaround?

eventgen.conf:

[test2.raw]
index=demo
host=myHost
source=test2.raw
breaker = \r*\n\r*\n
mode = replay
sampletype=raw
timeMultiple = 600
backfill = -10m
backfillSearch = index=demo source=test2.raw
outputMode = splunkstream
token.0.token = \d{4}-\d{2}-\d{2}T\d{2}.\d{2}.\d{2}
token.0.replacementType = replaytimestamp
token.0.replacement = %Y-%m-%dT%H:%M:%S
token.1.token = @@src_ip@@
token.1.replacementType = random
token.1.replacement = ipv4

test2.raw:

{"timestamp":"2015-09-04T15:45:00.454143Z","src_ip":"@@src_ip@@","comment":"web click #1"}
{"timestamp":"2015-09-04T15:46:01.454143Z","src_ip":"@@src_ip@@","comment":"web click #2"}
{"timestamp":"2015-09-04T15:47:02.454143Z","src_ip":"@@src_ip@@","comment":"web click #3"}
{"timestamp":"2015-09-04T15:48:03.454143Z","src_ip":"@@src_ip@@","comment":"web click #4"}

[csharp_splunk]

You don't need replaytimestamp (that feature should probably be removed, but it's being used by one consumer right now). Replaytimestamp is when you want to take a group of transactions using bundlelines and then randomize the time between them. It's specifically targeted at use cases where you want to run the Splunk transaction command and have a varied duration.

This may be a bug. Can you try with the Dev branch from https://github.com/splunk/eventgen and see if you're seeing that there? This branch is about to become master after a few years of development, and I'd like to see if you can reproduce it there. I'm relatively certain it should work properly on that branch.

[japger_splunk · Accepted solution]

No bug. After increasing the size of the backfill window and performing a more thorough analysis, I determined that the backfill window is respecting the specific timemultiple=600. The small period of time between the end of the backfill window and the beginning of the ongoing eventgen just happened to be producing 2 "blocks" that were within 1 second of each other. The above eventgen will reliably produce both backfill and ongoining "blocks". I should have caught that the timing between the last event in the backfill and the first event in the ongoining eventgen would be handled differently than as specified in the eventgen.conf file.

I will leave this example up there as a reference for how to replay events that need to be modeled as transactions.