Dashboards & Visualizations

improve xmlkv performance

marcosrios
Explorer

Hi, I is there any way to improve the performance extracting fields from xml logs?

Here are the results from my queries

source="txn.log" host="none" sourcetype= "txn log" | rex "(?<timestamp>(\d{4}-\d{2}-\d{2} \d{2}\:\d{2}\:\d{2}\,\d{3})) (?<loglevel>DEBUG|INFO|WARN|ERROR) (?<origin>\[.*\]) (?s)(?<content>.*)"
* This search has completed and has returned 1,965 results by scanning 1,965 events in 0.946 seconds

-
source="txn.log" host="none" sourcetype= "txn log" | rex "(?(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\,\d{3})) (?DEBUG|INFO|WARN|ERROR) (?[.]) (?s)(?.)" | xmlkv
* This search has completed and has returned 1,965 results by scanning 1,965 events in 12.125 seconds.

0 Karma
1 Solution

lguinn2
Legend

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?

View solution in original post

0 Karma

lguinn2
Legend

xmlkv tries to dynamically find all key-value pairs within the search results, so it can be quite slow.

You are already extracting some fields with rex - you could extract additional fields in the same way. The more fields that exist in the xml, the longer xmlkv will run. So using rex to extract only the fields that you need can be faster.

What do you plan to do after the xmlkv step?

0 Karma
Get Updates on the Splunk Community!

Buttercup Games: Further Dashboarding Techniques

Hello! We are excited to kick off a new series of blogs from SplunkTrust member ITWhisperer, who demonstrates ...

Message Parsing in SOCK

Introduction This blog post is part of an ongoing series on SOCK enablement. In this blog post, I will write ...

Exploring the OpenTelemetry Collector’s Kubernetes annotation-based discovery

We’ve already explored a few topics around observability in a Kubernetes environment -- Common Failures in a ...