Splunk Search

SA-Eventgen: detect timestamp in replay- vs. sample-mode

nekb1958
Path Finder

when i take my eventgen conf in sample-mode the timestamp is replaced with the actual time in the defined format. when i take the same conf and only change mode to replay, error:

ValueError: Can't find a timestamp (using patterns '['\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}']') in this event: '2012-01-30 23:59:04 QQc-O3J7xve CPUSys 1.046711451335729

what did i miss????

thanks for any help (i´m feeling stupid)
norbert

CONFIG

[short.splunk]  
mode = sample

outputMode = file  
fileName = e://tmp/meas-all.log

token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}  
token.0.replacementType = timestamp  
token.0.replacement = %b %d %H:%M:%S

DATA (== short.splunk)

2012-01-30 23:59:04 QQc-O3J7xve CPUUser 2.3843930719158464

2012-01-30 23:59:04 QQc-O3J7xve CPUSys 1.046711451335729

2012-01-30 23:59:04 sIkqmRX7zai CPUUser 0.57221331953932253

2012-01-30 23:59:04 sIkqmRX7zai CPUSys 0.37301278622911521

2012-01-30 23:59:02 QQc-O3J7xve MemFree 4326506496

2012-01-30 23:59:02 sIkqmRX7zai MemFree 3537133568

2012-01-30 23:58:04 sSGBns8SkM7 CPUUser 0.74475287745429919

2012-01-30 23:58:04 sSGBns8SkM7 CPUSys 0.56507473569084943

2012-01-30 23:58:02 sSGBns8SkM7 MemFree 7351476224

Tags (1)
0 Karma

fortiwhall
Explorer

I was having the hardest time with this today - thank you for the tip, @nekb1958.

I had switched from "sample" to "replay" as well and kept getting the same error over and over. I didn't understand @nekb1958's answer at first, but now I see what I'm supposed to do. I'm documenting it further below.

In my example, the timestamp looked like this:

Oct 16 10:59:54

In SAMPLE mode, you would use the three lines as follows, where the token regex is completely separate from the replacement format.

token.0.token = \w{3} \d{2} \d{2}:\d{2}:\d{2}
token.2.replacementType = timestamp  
token.2.replacement = %d/%b/%Y:%H:%M:%S.%f  

Here, the token does not need to match the replacement - the replacement variable is the POSIX format of the timestamp you want EventGen to insert into the result, while the token is the string you want to match. In the above case, I changed the timestamp to a new format just as an example.

In REPLAY mode, however, it seems the token regex and the replacement need to match.

token.0.token = \w{3} \d{2} \d{2}:\d{2}:\d{2}  
token.0.replacementType = timestamp  
token.0.replacement = %b %d %H:%M:%S  

That way, it knows where to look for a timestamp (via token regex) and it knows the POSIX format of the timestamp (via replacement format).

0 Karma

nekb1958
Path Finder

feeling stupid?

reading helps 😞

token.0.replacement = %b %d %H:%M:%S

isn´t (like i thought) the format (strptime) of the replaced timestamp, NO it is the format of the "input". stupid? yes 🙂

bye norbert

nekb1958
Path Finder

my original timestamp is in epoch, taking token.0.replacement = %s detect the right value, but has a problem to print epoch in replacement.
2013-08-08 14:33:37,548 DEBUG Match '1325372400' Format '%s' result: '2012-01-01 00:00:00'
2013-08-08 14:33:37,548 DEBUG Match '1325372402' Format '%s' result: '2012-01-01 00:00:02'

...
File "C:\Program Files\Splunk\etc\apps\SA-EventGen\lib\eventgensamples.py", line 921, in _getReplacement
replacementTime = replacementTime.strftime(self.replacement)
ValueError: Invalid format string

ok, now i know what works.

norbert

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...