Getting Data In
Highlighted

Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

New Member

Equallogic and Compellent use non-standard syslog formats when sending events. Are there pre-defined Splunk configurations (props.conf and transforms.conf) that will correctly parse these events?

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

New Member

We also have EqualLogic, are capturing the syslogs, and are annoyed when some events are split in to two. Using WireShark, I captured the syslogs and looks like the "offending" characters are "\x0d\x0a", which in the ASCII world are (Carriage return)(Line feed), respectfully. I'm new to Splunk and haven't had the training. If you can develop a solution be for I, I'd love to see it.

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Builder

I am facing this same issue. Were you able to resolve it ?

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Contributor

I would start by setting these in the props.conf. See if that clears it up.

SHOULDLINEMERGE = true
BREAK
ONLYBEFOREDATE = true

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Splunk Employee
Splunk Employee

This solution should work.. Or you can look at a combination of

SHOULD_LINEMERGE = TRUE
BREAKONLYBEFORE = ^\w+

Depending on what is following that second line feed (\r\n), adjust the regex to match the original first time, which is your timestamp...

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Contributor

If you can post some sample logs, I can help you out.

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Builder

Hi @xavierashe,
please find below sample, It should appear as one event but it is being split into two and since "Requested target not found" line does not have host ip, it is going to wrong sourcetype.

Requested target not found.

Oct 24 00:00:32 x.x.x.x 80000:20000:MgmtExec:20-Oct-2017 07:14:06.083084:targetAttr.cc:593:ERROR::7.4.3:iSCSI login to target 'x.x.x.x:3000, fqn.2001-05.com.equallogic:4-42a846-545b0c93bdf59ed0-test-dr' from initiator 'x.x.x.x:40000, fqn.1998-01.com:nel-esx1-326532g1' failed for the following reason:

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Contributor

First question, are you collecting this syslog via a syslog server (syslog-ng or rsyslog) or directly into Splunk via syslog input?

If I understand your post, it looks like the messages are coming in out of order, right? The reason I asked the first question is that will help us figure out how to get these two messages put back together.

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Builder

I am collecting directly into Splunk HF.
Yes, only this particular type of event is coming out of order.

0 Karma
Highlighted

Re: Are there pre-defined props and transforms.conf configurations for Equallogic and Compellent syslog parsing?

Contributor

To write a complete TA for this, I would need a many more events, but this should get you started. I made some assumpts about the data.

props.conf

TIME_FORMAT = %b-%Y %m %H:%M:%S.%f
EXTRACT-Equallogic_logginglevel = (?P<logging_level>INFO|WARN|ERROR|FATAL|TRACE)
EXTRACT-Equallogic_event = \d{2}:\d{2}:\d{2}\.\d+:\S+:(?P<event>.+?) '(?P<src_ip>.+?):(?P<src_port>\d+), (?P<src>.+?)'.+?'(?P<dest_ip>.+?):(?P<dest_port>\d+), (?P<dest>.+?)' (?P<status>.*?)$
EVAL-reason = if(status="failed for the following reason:","Requested target not found","")

I would strongly suggest you stop using Splunk's syslog collector and use syslog-ng instead. It might fix your out of order problem, and you can take out that eval statement. This is a great write-up on how to get that done: https://www.splunk.com/blog/2016/03/11/using-syslog-ng-with-splunk.html

0 Karma