topic Re: Bug in spath ? Not recovering every fields in Splunk Search

Bug in spath ? Not recovering every fields

sbsbb — Thu, 17 Oct 2013 11:24:19 GMT

Hi have a query, that try to get all the fields from an xml doc.
For some reason, spath seems to ignore some of the fields, in the example below, this is the case with field LinienText under Splunk 5.0.2 :

Re: Bug in spath ? Not recovering every fields

sbsbb — Fri, 18 Oct 2013 08:08:33 GMT

In Fact it is because of the limitation of spath from 5000 characters, I have 10000...

for me it is really a problem don't being able to overwrite this setting as user... like "spath limit=10000"

Re: Bug in spath ? Not recovering every fields

capnjosh — Tue, 13 Jan 2015 21:32:57 GMT

I had been hitting the same problem: some events had xml that was longer than 5000 characters, and spath wasn't extracting all the fields I knew were in there.

Here's how to fix it:
Override the spath character limit in $splunk_home%/etc/system/local/limits.conf.

My exact edit was to add the below config section to /opt/splunk/etc/system/local/limits.conf (since it wasn't there be default in 4.3.3). I pulled this from /opt/splunk/etc/system/default/limit.conf:

[spath]

number of characters to read from an XML or JSON event when auto extracting

extraction_cutoff = 10000

I copied and pasted my answer from here: http://answers.splunk.com/answers/179968/parsing-very-long-json-lines.html. I hope that's ok. These two Splunk answers were the top 2 in Google searches for, so I figured I'd contribute my solution in case it helps others.