topic Re: timestamp and line breaks in Getting Data In

timestamp and line breaks

rewritex — Thu, 21 Sep 2017 23:34:30 GMT

The timestamp and linebreaking doesn't seem to be working as expected. They are nagios/pnp4nagios logs.
I get a burst of events similar to the below data every few seconds/minutes and it seems the first line of each data burst is being recognized for the TIMET timestamp but all other events within that data burst aren't being handled correctly.

TIMET::1506034709 = timestamp in epoch time
DATATYPE:: = start/end of event

Data is sent in this format: DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\t

Here's the data:

DATATYPE::HOSTPERFDATA  TIMET::1506034709   HOSTNAME::host1 HOSTPERFDATA::time=0.000342s;;;0.000000;20.000000   HOSTCHECKCOMMAND::check_tcp!255.255.25.25!443   HOSTSTATE::UP   HOSTSTATETYPE::HARD HOSTOUTPUT::TCP OK - 0.000 second response time on 255.255.25.25 port 443   
DATATYPE::HOSTPERFDATA  TIMET::1506034713   HOSTNAME::host2 HOSTPERFDATA::time=0.000368s;;;0.000000;20.000000   HOSTCHECKCOMMAND::check_tcp!255.255.25.256!443  HOSTSTATE::UP   HOSTSTATETYPE::HARD HOSTOUTPUT::TCP OK - 0.000 second response time on 255.255.25.256 port 443

Here's the sourcetype config: - timestamp/linebreak

[nagios:core:perfdata]
event_breaks: (I've tried auto and every line)
BREAK_ONLY_BEFORE = ([\r\n]+)DATATYPE
SHOULD_LINEMERGE = true
TIME_FORMAT =  %s
TIME_PREFIX = TIMET::
lookahead 128

Re: timestamp and line breaks

s2_splunk — Fri, 22 Sep 2017 00:08:53 GMT

Is there a line breaker in the source events at all? From your post there is, so standard line breaking (using CRLF) should work. If it doesn't, there is no line feed in the source.
You can try BREAK_ONLY_BEFORE=DATATYPE::
Unless you are dealing with multi-line events, set SHOULD_LINEMERGE=false
Line 7 in your props.conf above is not a valid setting, it should be MAX_TIMESTAMP_LOOKAHEAD=128.

Also, you configured that where parsing occurs (indexer, heavy forwarder), correct?

Re: timestamp and line breaks

rewritex — Tue, 29 Sep 2020 15:54:36 GMT

I had to modify the props.conf on the cluster/indexers and that seemed to get it working. I was in the SH mucking around with the props.conf
Thanks for the reminder.

$SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf

BREAK_ONLY_BEFORE=DATATYPE::
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = TIMET::

Re: timestamp and line breaks

s2_splunk — Fri, 22 Sep 2017 01:49:01 GMT

Ha, he/she who's never done that, speak up now or be silent forever! 🙂
Glad you got it working.