Splunk Dev

Extract fields at ingest

timtekk
Explorer

I have UFs installed on some sql servers that forward certain events (according to eventID) to my Splunk. 

I have created a search query to parse out the data need to make a nice table. However, ideally I'd like to do this at ingest time instead of at search. 

I was told by my manager to research props.conf and transforms.conf and here I am 🙂 Not sure if that is the proper route or if there are other suggestions. Thank you. 

 

index="wineventlog"
| rex field=EventData_Xml (server_principal_name:(?<server_principal_name>\S+))
| rex field=EventData_Xml (server_instance_name:(?<server_instance_name>\S+))
| rex field=EventData_Xml (action_id:(?<action_id>\S+))
| rex field=EventData_Xml (succeeded:(?<succeeded>\S+))
| table _time, action_id, succeeded, server_principal_name, server_instance_name
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Search-time extractions are preferred over index-time extractions, because they use less storage (none) and don't slow down indexing.

You can extract fields automatically at search time by adding EXTRACT settings in the sourcetypes's props.conf stanza.

[xmlwineventlog]
EXTRACT-s_p_n = (server_principal_name:(?<server_principal_name>\S+)) in EventData_Xml 
EXTRACT-s_i_n = (server_instance_name:(?<server_instance_name>\S+)) in EventData_Xml 
EXTRACT-a_i = (action_id:(?<action_id>\S+)) in EventData_Xml 
EXTRACT-succeeded = (succeeded:(?<succeeded>\S+)) in EventData_Xml 

 

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Search-time extractions are preferred over index-time extractions, because they use less storage (none) and don't slow down indexing.

You can extract fields automatically at search time by adding EXTRACT settings in the sourcetypes's props.conf stanza.

[xmlwineventlog]
EXTRACT-s_p_n = (server_principal_name:(?<server_principal_name>\S+)) in EventData_Xml 
EXTRACT-s_i_n = (server_instance_name:(?<server_instance_name>\S+)) in EventData_Xml 
EXTRACT-a_i = (action_id:(?<action_id>\S+)) in EventData_Xml 
EXTRACT-succeeded = (succeeded:(?<succeeded>\S+)) in EventData_Xml 

 

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...