topic Re: How to split a file into events based on multiple criteria? in Getting Data In

How to split a file into events based on multiple criteria?

Hazel — Wed, 27 Apr 2011 11:31:23 GMT

Hello

I have a log file that has two different types of events in it.

Events that look like this

 <logtime="2011-04-25T23:05:16.389" level=INFO file="file1.C" line="280">
<![CDATA[ 
Submitted xys
]]></log>

Events that look like this
```
 26-Apr-2011 09:44:52; INFO.... 
```

I want my splunk props.conf to split events every time it sees either of these, but I can only get it to work with one or the others.

If I stick with the default and have no props.conf then it splits events based on 2. as this is the standard DATETIME event.

I know I can also use a props.conf to get it to break on ^<log, but I can't get it to do both at the same time.

Can anyone help?

For example, I thought something like this below (I've hard coded the 26-Apr just to get this scenario right and then i can put it as something more generic for any date)

[source::.../test.tester2] 
CHECK_METHOD = entire_md5
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = (^<log|^26-Apr)

But this is only picking up the first scenario and does not split into a new event when it sees something starting with 26-Apr. Any ideas?

Thanks
Hazel

Re: How to split a file into events based on multiple criteria?

hazekamp — Wed, 27 Apr 2011 16:26:44 GMT

Have you tried a LINE_BREAKER that will match either record:

[source::...test.tester2]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(<logtime|\d{2}-\w{3}-\d{4}\s+\d{2}:\d{2}:\d{2};)

Re: How to split a file into events based on multiple criteria?

Hazel — Thu, 28 Apr 2011 15:13:07 GMT

Hello. I added this in, and now when a file comes through that matches this, the splunkd running the props.conf (on my indexer as I am using light forwarders), crashes out. I have confirmed this as splunkd will be fine and then as soon as I send through a file that matches to perform the above props.conf, splunkd crashes immediately....

Re: How to split a file into events based on multiple criteria?

hazekamp — Thu, 28 Apr 2011 17:22:09 GMT

Interesting...try the updated line breaker above...

Re: How to split a file into events based on multiple criteria?

Hazel — Tue, 03 May 2011 14:41:44 GMT

Hello, this worked!! Thanks 🙂