Getting Data In

timestamp and line breaks

Contributor

The timestamp and linebreaking doesn't seem to be working as expected. They are nagios/pnp4nagios logs.
I get a burst of events similar to the below data every few seconds/minutes and it seems the first line of each data burst is being recognized for the TIMET timestamp but all other events within that data burst aren't being handled correctly.

TIMET::1506034709 = timestamp in epoch time
DATATYPE:: = start/end of event

Data is sent in this format: DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\t

Here's the data:

DATATYPE::HOSTPERFDATA  TIMET::1506034709   HOSTNAME::host1 HOSTPERFDATA::time=0.000342s;;;0.000000;20.000000   HOSTCHECKCOMMAND::check_tcp!255.255.25.25!443   HOSTSTATE::UP   HOSTSTATETYPE::HARD HOSTOUTPUT::TCP OK - 0.000 second response time on 255.255.25.25 port 443   
DATATYPE::HOSTPERFDATA  TIMET::1506034713   HOSTNAME::host2 HOSTPERFDATA::time=0.000368s;;;0.000000;20.000000   HOSTCHECKCOMMAND::check_tcp!255.255.25.256!443  HOSTSTATE::UP   HOSTSTATETYPE::HARD HOSTOUTPUT::TCP OK - 0.000 second response time on 255.255.25.256 port 443

Here's the sourcetype config: - timestamp/linebreak

[nagios:core:perfdata]
event_breaks: (I've tried auto and every line)
BREAK_ONLY_BEFORE = ([\r\n]+)DATATYPE
SHOULD_LINEMERGE = true
TIME_FORMAT =  %s
TIME_PREFIX = TIMET::
lookahead 128
0 Karma
1 Solution

Contributor

I had to modify the props.conf on the cluster/indexers and that seemed to get it working. I was in the SH mucking around with the props.conf
Thanks for the reminder.

$SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf

BREAK_ONLY_BEFORE=DATATYPE::
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = TIMET::

View solution in original post

Contributor

I had to modify the props.conf on the cluster/indexers and that seemed to get it working. I was in the SH mucking around with the props.conf
Thanks for the reminder.

$SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf

BREAK_ONLY_BEFORE=DATATYPE::
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = TIMET::

View solution in original post

Splunk Employee
Splunk Employee

Ha, he/she who's never done that, speak up now or be silent forever! 🙂
Glad you got it working.

0 Karma

Splunk Employee
Splunk Employee

Is there a line breaker in the source events at all? From your post there is, so standard line breaking (using CRLF) should work. If it doesn't, there is no line feed in the source.
You can try BREAK_ONLY_BEFORE=DATATYPE::
Unless you are dealing with multi-line events, set SHOULD_LINEMERGE=false
Line 7 in your props.conf above is not a valid setting, it should be MAX_TIMESTAMP_LOOKAHEAD=128.

Also, you configured that where parsing occurs (indexer, heavy forwarder), correct?

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!