Unable to linebreak a forwarded json input but works when file monitored locally

[Cuyose]

This is odd, I have a json log file that can be copied and added manually or monitored locally from a standalone instance applying the my_json sourcetype. the only thing this sourcetype initially needed to work from the autoselected _json sourcetype is TRUNCATE = 0 and defining the timestamp field.

Everything looks perfect when manually indexing the file or when monitoring the file locally. However if I use a forwarder to send the data to the same exact server hat has no issues indexing the data manually, none of the events are broken out, nor are the timestamps found.

[Cuyose · Accepted solution]

I figured this out, apparently some structured sourcetypes need to also have the props on the actual forwarder?! Even though this was an indexed extraction, specifying only truncate = 0 and the custom timestamp field json element attribute value, having the props on the indexer and search heads was not sufficient. Once the props.conf was also pushed to the forwarder sending the file it worked.

[Cuyose]

Ill add the info I came across that describes this behavior

Since splunk 6, some sources can be parsed for structured data (like headers, or json) and be populated at the forwarder level. see http://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Extractfieldsfromfileheadersatindextime#Forwa... Those setting have to be on the forwarders (and indexers if they monitor files)

Caveats
Splunk Enterprise does not parse structured data that has been forwarded to an indexer

When you forward structured data to an indexer, Splunk Enterprise does not parse this data once it arrives at the indexer, even if you have configured props.conf on that indexer with INDEXED_EXTRACTIONS. Forwarded data skips the following queues on the indexer, which precludes any parsing of that data on the indexer:

parsing
aggregation
typing

The forwarded data must arrive at the indexer already parsed. To achieve this, you must also set up props.conf on the forwarder that sends the data. This includes configuration of INDEXED_EXTRACTIONS and any other parsing, filtering, anonymizing, and routing rules. Universal forwarders are capable of performing these tasks solely for structured data. See "Forward data extracted from header files" earlier in this topic.

http://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Extractfieldsfromfileheadersatindextime
http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

[adonio]

where did you put your props.conf?