All Apps and Add-ons

How to use Eventgen for multiline logs?

phoenixdigital
Builder

Hi All,

Just trying to work out how to use eventgen for multiline logs such as oracles' hideous audit file.

Audit file /ora/app/oracle/admin/cdh/adump/oracle_audit_log.aud
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
ORACLE_HOME = /ora/app/oracle/product/11.2.0.4/db_1
System name:    SunOS
Node name:  hostname-oracle4
Release:    5.10
Version:    Generic_150400-26
Machine:    sun4v
Instance name: this_instance
Redo thread mounted by this instance: 1
Oracle process number: 90
Unix process pid: 5542, image: oracle@hostname-oracle4

Mon Nov  2 11:33:30 2015 +11:00
LENGTH: "223"
SESSIONID:[8] "23491997" ENTRYID:[1] "1" USERID:[6] "SIEM_TEST1" ACTION:[3] "44" RETURNCODE:[1] "0" LOGOFF$PREAD:[1] "0" LOGOFF$LREAD:[2] "24" LOGOFF$LWRITE:[1] "0" LOGOFF$DEAD:[1] "0" DBID:[9] "847685182" SESSIONCPU:[1] "4"

Mon Nov  2 11:33:33 2015 +11:00
LENGTH: "223"
SESSIONID:[8] "23491997" ENTRYID:[1] "1" USERID:[6] "SIEM_TEST1" ACTION:[3] "115" RETURNCODE:[1] "0" LOGOFF$PREAD:[1] "0" LOGOFF$LREAD:[2] "24" LOGOFF$LWRITE:[1] "0" LOGOFF$DEAD:[1] "0" DBID:[9] "847685182" SESSIONCPU:[1] "4"

I have tried this in evengen.conf

[oracle_audit_log.aud]
disabled = false
mode = replay
index=sec_database
sourcetype=oracle:audit:text
# breaker= ^\r\n
bundlelines = true

## Generate all events in sample
count = 0
earliest = -5m
latest = now
interval = 300

outputMode=spool
spoolFile = sample.oracle

# host.token
# host.replacement = hosts.list

## Replace timestamp
token.0.token = \w{3}\s+\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2}\s+\d{4}
token.0.replacementType = timestamp
token.0.replacement = %a %b %d %H:%M:%S %Y

But it throws this error

2015-11-03 13:55:51,350 INFO Retrieving eventgen configurations from /configs/eventgen
2015-11-03 13:55:51,894 INFO Creating timer object for sample 'oracle_audit_log.aud' in app 'TA_ob-3_oracle_eventgen'

2015-11-03 13:55:51,896 INFO Starting timers
2015-11-03 13:55:51,898 ERROR Can't find a timestamp (using patterns '['\\w{3}\\s+\\w{3}\\s+\\d{1,2}\\s+\\d{2}:\\d{2}:\\d{2}\\s+\\d{4}']') in this event: 'LENGTH: "223"
'.
2015-11-03 13:55:51,901 ERROR Exception in sample: oracle_audit_log.aud
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/SA-Eventgen/bin/eventgen.py", line 47, in run
    partialInterval = self.sample.gen()
  File "/opt/splunk/etc/apps/SA-Eventgen/lib/eventgensamples.py", line 506, in gen
    self._lastts = self._getTSFromEvent(self._rpevents[self._currentevent])
  File "/opt/splunk/etc/apps/SA-Eventgen/lib/eventgensamples.py", line 702, in _getTSFromEvent
    raise ValueError("Can't find a timestamp (using patterns '%s') in this event: '%s'." % (formats, event))
ValueError: Can't find a timestamp (using patterns '['\\w{3}\\s+\\w{3}\\s+\\d{1,2}\\s+\\d{2}:\\d{2}:\\d{2}\\s+\\d{4}']') in this event: 'LENGTH: "223"
'.

I'm guessing it is trying to find a timestamp on every line, but obviously one does not exist.

I even tried messing around with breaker.

I even removed all the junk at the top of the log with no luck.

Any ideas would be appreciated.

Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

Your breaker is wrong and it's looking for the timestamp in this one line:

LENGTH: "223"

Use this as your breaker and discard the header data in the sample log:

([\n\r]+)\w{3}\s+\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2}\s+\d{4}

phoenixdigital
Builder

Its been a while since I looked at this but I should note the error is in the eventgen code not in the Splunk ingestion config.

Eventgen can't handle generating sample data where it is multiple lines. It is breaking as eventgen tries to make fake logs from this sample data.

You are right though if I was having problems ingesting this data into Splunk. Changing the breaker would make this work just fine. However changing the breaker will have no effect on eventgen as far as I know.

0 Karma

jkat54
SplunkTrust
SplunkTrust

I meant with eventgen, not on data ingestion. The error message is saying it's failing to parse the timestamp on the length line... and I have little knowledge about event gen but figured if you put the proper breaker in there it could work.

0 Karma

phoenixdigital
Builder

Good to know. I haven't looked much at Eventgen since I posted this ~2 years ago it has likely changed a bit since then.

https://github.com/splunk/eventgen

Back then if eventgen didn't find a timestamp on every line it borked out as shown above.

0 Karma

mjm_bhatiarahul
New Member

Reviving this old thread... I am encountering a similar issue trying to replay a log file containing multi-line event.

Anyone has any suggestions on how to make this work?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Old threads should not be "revived". It's better to post a new question describing the problem you are having.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

mjm_bhatiarahul
New Member

Reviving this old thread... I am encountering a similar issue trying to replay a log file containing multi-line event.

Anyone has any suggestions on how to make this work?

0 Karma

jkat54
SplunkTrust
SplunkTrust

You should create your own question and provide the same level of details as the original poster. We cant tell you how to fix your problem without all of the same details and therefore you should start a new question.

0 Karma

mjm_bhatiarahul
New Member

Will do so... Thanks

0 Karma

lakshman239
SplunkTrust
SplunkTrust

can you confirm if you have used token.0.replacementType = timestamp stanza which covers all possible timeformats available in your multi-line event data?

Also, if you are in app, say myapps with your sample file, mysample.csv, you can run to check the actual error ( should also be in splunkd.log)

/opt/splunk/etc/apps/myapps/samples$ /opt/splunk/bin/python ../../eventgen/bin/eventgen.py -v -s mysample.csv ../local/eventgen.conf

0 Karma

lakshman239
SplunkTrust
SplunkTrust

could you do one of these two?

  1. try %:z to match the +11:00
  2. in the same file, have _time and provide timestamp there
0 Karma

KarunK
Contributor

I am also facing the same issue with eventgen when i am trying to generate Windows HostMon data. I think the breaker is the causing the problem. Any suggetions for mutliline event generation ?

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!