eventgen app not generating the data due to the "can't find the time stamp warning".

[pavanae]

I was using the sample tutorial 1 as eventgen.conf which is as below :-

[sample_tutorial 1.sample]
mode = replay
sampletype = csv
timeMultiple = 2

backfill = -15m

backfillSearch = index=main sourcetype=splunkd

outputMode = stdout

outputMode = splunkstream

splunkHost = localhost

splunkUser = admin

splunkPass = changeme

outputMode = file
fileName = /tmp/internal.log
token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

token.1.token = \d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}
token.1.replacementType = timestamp
token.1.replacement = %m-%d-%Y %H:%M:%S.%f

token.2.token = \d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}
token.2.replacementType = timestamp
token.2.replacement = %d/%b/%Y:%H:%M:%S.%f

token.3.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
token.3.replacementType = timestamp
token.3.replacement = %Y-%m-%d %H:%M:%S

token.4.token = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}
token.4.replacementType = timestamp
token.4.replacement = %Y-%m-%dT%H:%M:%S

Now the below were some of the events in my .csv file which I'd kept in samples directory

Oct 4 08:18:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............
Oct 4 08:19:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............

The below is the error I'm seeing regarding the timestamp

WARNING module='Sample' sample='exported_logs.csv': Can't find a timestamp (using patterns '['\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}', '\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}', '\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}', '\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}', '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}']') in this event: 'Oct 4 08:18:26 xyz.net Oct 4 08:18:26 xyzabc.net 1,2016/10/04 08:18:26,007801003385,THREAT,url,1,2016/10/04 08:18:26,1.2.3.4.1.111.32,0.0.0.0,0.0.0.0,eserv-unknownURL,x-fwd-for: 11.22.33.44,,web-browsing,vsys1,AM-trust,AM-untrust,ethernet1/2,ethernet1/1,AllThreatsAM,2016/10/04 08:18:26,33919488,1,56014,80,0,0,0x80000,tcp,alert,"vid-io.springserve.com/vd/i?

Based on the above error I understand that the timeformat in the exported_logs in .csv file were different from the eventgen.conf file. So how can i modify the timeformat stanzas in the eventgen.conf file to make it working?

[csharp_splunk · Accepted solution]

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

[pavanae]

what does %b %d mean ?