improve xmlkv performance

[marcosrios]

Hi, I is there any way to improve the performance extracting fields from xml logs?

Here are the results from my queries

source="txn.log" host="none" sourcetype= "txn log" | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)"
* This search has completed and has returned 1,965 results by scanning 1,965 events in 0.946 seconds

-
source="txn.log" host="none" sourcetype= "txn log" | rex "(?(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\,\d{3})) (?DEBUG|INFO|WARN|ERROR) (?[.]) (?s)(?.)" | xmlkv
* This search has completed and has returned 1,965 results by scanning 1,965 events in 12.125 seconds.

[lguinn2 · Accepted solution]

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?