Dashboards & Visualizations

improve xmlkv performance

marcosrios
Explorer

Hi, I is there any way to improve the performance extracting fields from xml logs?

Here are the results from my queries

source="txn.log" host="none" sourcetype= "txn log" | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)"
* This search has completed and has returned 1,965 results by scanning 1,965 events in 0.946 seconds

-
source="txn.log" host="none" sourcetype= "txn log" | rex "(?(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\,\d{3})) (?DEBUG|INFO|WARN|ERROR) (?[.]) (?s)(?.)" | xmlkv
* This search has completed and has returned 1,965 results by scanning 1,965 events in 12.125 seconds.

0 Karma
1 Solution

lguinn2
Legend

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?

View solution in original post

0 Karma

lguinn2
Legend

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...