All Apps and Add-ons

eventgen app not generating the data due to the "can't find the time stamp warning".

pavanae
Builder

I was using the sample tutorial 1 as eventgen.conf which is as below :-

[sample_tutorial 1.sample]
mode = replay
sampletype = csv
timeMultiple = 2

backfill = -15m

backfillSearch = index=main sourcetype=splunkd

outputMode = stdout

outputMode = splunkstream

splunkHost = localhost

splunkUser = admin

splunkPass = changeme

outputMode = file
fileName = /tmp/internal.log
token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

token.1.token = \d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}
token.1.replacementType = timestamp
token.1.replacement = %m-%d-%Y %H:%M:%S.%f

token.2.token = \d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}
token.2.replacementType = timestamp
token.2.replacement = %d/%b/%Y:%H:%M:%S.%f

token.3.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
token.3.replacementType = timestamp
token.3.replacement = %Y-%m-%d %H:%M:%S

token.4.token = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}
token.4.replacementType = timestamp
token.4.replacement = %Y-%m-%dT%H:%M:%S

Now the below were some of the events in my .csv file which I'd kept in samples directory

Oct 4 08:18:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............
Oct 4 08:19:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............

The below is the error I'm seeing regarding the timestamp

WARNING module='Sample' sample='exported_logs.csv': Can't find a timestamp (using patterns '['\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}', '\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}', '\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}', '\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}', '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}']') in this event: 'Oct 4 08:18:26 xyz.net Oct 4 08:18:26 xyzabc.net 1,2016/10/04 08:18:26,007801003385,THREAT,url,1,2016/10/04 08:18:26,1.2.3.4.1.111.32,0.0.0.0,0.0.0.0,eserv-unknownURL,x-fwd-for: 11.22.33.44,,web-browsing,vsys1,AM-trust,AM-untrust,ethernet1/2,ethernet1/1,AllThreatsAM,2016/10/04 08:18:26,33919488,1,56014,80,0,0,0x80000,tcp,alert,"vid-io.springserve.com/vd/i?

Based on the above error I understand that the timeformat in the exported_logs in .csv file were different from the eventgen.conf file. So how can i modify the timeformat stanzas in the eventgen.conf file to make it working?

0 Karma
1 Solution

csharp_splunk
Splunk Employee
Splunk Employee

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

View solution in original post

0 Karma

csharp_splunk
Splunk Employee
Splunk Employee

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

0 Karma

pavanae
Builder

what does %b %d mean ?

0 Karma
Get Updates on the Splunk Community!

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Admin Console: A Single, Unified Interface for All Your Cloud Admin Needs

WATCH NOWJoin us to learn how the admin console can save you time and give you more control over the Splunk® ...