topic Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing? in Getting Data In

Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

wightjw — Wed, 13 Apr 2016 19:18:38 GMT

Equallogic and Compellent use non-standard syslog formats when sending events. Are there pre-defined Splunk configurations (props.conf and transforms.conf) that will correctly parse these events?

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

daphelps — Fri, 13 May 2016 16:14:59 GMT

We also have EqualLogic, are capturing the syslogs, and are annoyed when some events are split in to two. Using WireShark, I captured the syslogs and looks like the "offending" characters are "\x0d\x0a", which in the ASCII world are (Carriage return)(Line feed), respectfully. I'm new to Splunk and haven't had the training. If you can develop a solution be for I, I'd love to see it.

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

damode — Fri, 20 Oct 2017 01:10:49 GMT

I am facing this same issue. Were you able to resolve it ?

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

xavierashe — Tue, 24 Oct 2017 11:29:02 GMT

If you can post some sample logs, I can help you out.

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

xavierashe — Tue, 29 Sep 2020 16:27:26 GMT

I would start by setting these in the props.conf. See if that clears it up.

SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = true

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

damode — Tue, 24 Oct 2017 23:09:44 GMT

Hi @xavierashe,
please find below sample, It should appear as one event but it is being split into two and since "Requested target not found" line does not have host ip, it is going to wrong sourcetype.

Requested target not found.

Oct 24 00:00:32 x.x.x.x 80000:20000:MgmtExec:20-Oct-2017 07:14:06.083084:targetAttr.cc:593:ERROR::7.4.3:iSCSI login to target 'x.x.x.x:3000, fqn.2001-05.com.equallogic:4-42a846-545b0c93bdf59ed0-test-dr' from initiator 'x.x.x.x:40000, fqn.1998-01.com:nel-esx1-326532g1' failed for the following reason:

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

esix_splunk — Tue, 29 Sep 2020 16:24:57 GMT

This solution should work.. Or you can look at a combination of

SHOULD_LINEMERGE = TRUE
BREAK_ONLY_BEFORE = ^\w+

Depending on what is following that second line feed (\r\n), adjust the regex to match the original first time, which is your timestamp...

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

xavierashe — Wed, 25 Oct 2017 12:46:54 GMT

First question, are you collecting this syslog via a syslog server (syslog-ng or rsyslog) or directly into Splunk via syslog input?

If I understand your post, it looks like the messages are coming in out of order, right? The reason I asked the first question is that will help us figure out how to get these two messages put back together.

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

damode — Wed, 25 Oct 2017 23:19:38 GMT

I am collecting directly into Splunk HF.
Yes, only this particular type of event is coming out of order.

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

xavierashe — Fri, 27 Oct 2017 14:17:41 GMT

To write a complete TA for this, I would need a many more events, but this should get you started. I made some assumpts about the data.

props.conf

TIME_FORMAT = %b-%Y %m %H:%M:%S.%f
EXTRACT-Equallogic_logginglevel = (?P<logging_level>INFO|WARN|ERROR|FATAL|TRACE)
EXTRACT-Equallogic_event = \d{2}:\d{2}:\d{2}\.\d+:\S+:(?P<event>.+?) '(?P<src_ip>.+?):(?P<src_port>\d+), (?P<src>.+?)'.+?'(?P<dest_ip>.+?):(?P<dest_port>\d+), (?P<dest>.+?)' (?P<status>.*?)$
EVAL-reason = if(status="failed for the following reason:","Requested target not found","")

I would strongly suggest you stop using Splunk's syslog collector and use syslog-ng instead. It might fix your out of order problem, and you can take out that eval statement. This is a great write-up on how to get that done: https://www.splunk.com/blog/2016/03/11/using-syslog-ng-with-splunk.html