topic Re: universal forwarder trying to parse the data in Getting Data In

universal forwarder trying to parse the data

reggie_123 — Thu, 29 Sep 2016 04:44:20 GMT

I have a UF monitoring a couple of files on a AIX box.
The UF is forwarding the data to a HF, I verified this in outputs.conf.
There are no props.conf present for that input on the UF, only at the HF, and they are obviously not being honored.
For some strange reason, I see "Breaking event" and "DateParserVerbose" errors on the UF.
How come the parsing phase takes place on the UF and not only the forwarding of the data ? I didn't get this behavior on any of my other UFs.
This is not an indexed_extraction.

Re: universal forwarder trying to parse the data

somesoni2 — Thu, 29 Sep 2016 05:00:23 GMT

Could you post some of the error samples that you see in UF? Also, when you say you're seeing those in UF, it means you see those in physical splunkd.log file on UF?

Re: universal forwarder trying to parse the data

reggie_123 — Thu, 29 Sep 2016 05:03:46 GMT

Exactly, on the UF box, on it's splunkd log.
Examples:
"DateParserVerbose - Time parsed (Mon May 30 21:00:00 2016) is too far away from the previous event's time"
"AggregatorMiningProcessor - Breaking event because limit of 256 has been exceeded"
I expect to see such messages only on the HF

Re: universal forwarder trying to parse the data

yannK — Thu, 29 Sep 2016 16:00:11 GMT

The only data that will be parsed on an Universal or Lightweight forwarders (and all forwarders) will be the sourcetypes using INDEXED_EXTRACTIONS. that do tailing time structured data parsing.
usually : xml, json, IIS, etc ...

see http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileswithstructureddata

Check your data format and sourcetype.
then if it is the case, you can prevent the errors by tuning your sourcetype parsing on the forwarder props.conf directly (like the MAX_EVENTS to raise to more than 256)