Splunk Dev

Extract fields at ingest

timtekk
Explorer

I have UFs installed on some sql servers that forward certain events (according to eventID) to my Splunk. 

I have created a search query to parse out the data need to make a nice table. However, ideally I'd like to do this at ingest time instead of at search. 

I was told by my manager to research props.conf and transforms.conf and here I am 🙂 Not sure if that is the proper route or if there are other suggestions. Thank you. 

 

index="wineventlog"
| rex field=EventData_Xml (server_principal_name:(?<server_principal_name>\S+))
| rex field=EventData_Xml (server_instance_name:(?<server_instance_name>\S+))
| rex field=EventData_Xml (action_id:(?<action_id>\S+))
| rex field=EventData_Xml (succeeded:(?<succeeded>\S+))
| table _time, action_id, succeeded, server_principal_name, server_instance_name
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Search-time extractions are preferred over index-time extractions, because they use less storage (none) and don't slow down indexing.

You can extract fields automatically at search time by adding EXTRACT settings in the sourcetypes's props.conf stanza.

[xmlwineventlog]
EXTRACT-s_p_n = (server_principal_name:(?<server_principal_name>\S+)) in EventData_Xml 
EXTRACT-s_i_n = (server_instance_name:(?<server_instance_name>\S+)) in EventData_Xml 
EXTRACT-a_i = (action_id:(?<action_id>\S+)) in EventData_Xml 
EXTRACT-succeeded = (succeeded:(?<succeeded>\S+)) in EventData_Xml 

 

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Search-time extractions are preferred over index-time extractions, because they use less storage (none) and don't slow down indexing.

You can extract fields automatically at search time by adding EXTRACT settings in the sourcetypes's props.conf stanza.

[xmlwineventlog]
EXTRACT-s_p_n = (server_principal_name:(?<server_principal_name>\S+)) in EventData_Xml 
EXTRACT-s_i_n = (server_instance_name:(?<server_instance_name>\S+)) in EventData_Xml 
EXTRACT-a_i = (action_id:(?<action_id>\S+)) in EventData_Xml 
EXTRACT-succeeded = (succeeded:(?<succeeded>\S+)) in EventData_Xml 

 

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...