All Apps and Add-ons

eventgen app not generating the data due to the "can't find the time stamp warning".

pavanae
Builder

I was using the sample tutorial 1 as eventgen.conf which is as below :-

[sample_tutorial 1.sample]
mode = replay
sampletype = csv
timeMultiple = 2

backfill = -15m

backfillSearch = index=main sourcetype=splunkd

outputMode = stdout

outputMode = splunkstream

splunkHost = localhost

splunkUser = admin

splunkPass = changeme

outputMode = file
fileName = /tmp/internal.log
token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

token.1.token = \d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}
token.1.replacementType = timestamp
token.1.replacement = %m-%d-%Y %H:%M:%S.%f

token.2.token = \d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}
token.2.replacementType = timestamp
token.2.replacement = %d/%b/%Y:%H:%M:%S.%f

token.3.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
token.3.replacementType = timestamp
token.3.replacement = %Y-%m-%d %H:%M:%S

token.4.token = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}
token.4.replacementType = timestamp
token.4.replacement = %Y-%m-%dT%H:%M:%S

Now the below were some of the events in my .csv file which I'd kept in samples directory

Oct 4 08:18:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............
Oct 4 08:19:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............

The below is the error I'm seeing regarding the timestamp

WARNING module='Sample' sample='exported_logs.csv': Can't find a timestamp (using patterns '['\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}', '\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}', '\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}', '\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}', '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}']') in this event: 'Oct 4 08:18:26 xyz.net Oct 4 08:18:26 xyzabc.net 1,2016/10/04 08:18:26,007801003385,THREAT,url,1,2016/10/04 08:18:26,1.2.3.4.1.111.32,0.0.0.0,0.0.0.0,eserv-unknownURL,x-fwd-for: 11.22.33.44,,web-browsing,vsys1,AM-trust,AM-untrust,ethernet1/2,ethernet1/1,AllThreatsAM,2016/10/04 08:18:26,33919488,1,56014,80,0,0,0x80000,tcp,alert,"vid-io.springserve.com/vd/i?

Based on the above error I understand that the timeformat in the exported_logs in .csv file were different from the eventgen.conf file. So how can i modify the timeformat stanzas in the eventgen.conf file to make it working?

0 Karma
1 Solution

csharp_splunk
Splunk Employee
Splunk Employee

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

View solution in original post

0 Karma

csharp_splunk
Splunk Employee
Splunk Employee

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

0 Karma

pavanae
Builder

what does %b %d mean ?

0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...