Dashboards & Visualizations

improve xmlkv performance

Explorer

Hi, I is there any way to improve the performance extracting fields from xml logs?

Here are the results from my queries

source="txn.log" host="none" sourcetype= "txn log" | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)"
* This search has completed and has returned 1,965 results by scanning 1,965 events in 0.946 seconds

-
source="txn.log" host="none" sourcetype= "txn log" | rex "(?(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\,\d{3})) (?DEBUG|INFO|WARN|ERROR) (?[.]) (?s)(?.)" | xmlkv
* This search has completed and has returned 1,965 results by scanning 1,965 events in 12.125 seconds.

0 Karma
1 Solution

Legend

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?

View solution in original post

0 Karma

Legend

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?

View solution in original post

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!