Extract fields at ingest

[timtekk]

I have UFs installed on some sql servers that forward certain events (according to eventID) to my Splunk. 

I have created a search query to parse out the data need to make a nice table. However, ideally I'd like to do this at ingest time instead of at search. 

I was told by my manager to research props.conf and transforms.conf and here I am 🙂 Not sure if that is the proper route or if there are other suggestions. Thank you. 

index="wineventlog"
| rex field=EventData_Xml (server_principal_name:(?<server_principal_name>\S+))
| rex field=EventData_Xml (server_instance_name:(?<server_instance_name>\S+))
| rex field=EventData_Xml (action_id:(?<action_id>\S+))
| rex field=EventData_Xml (succeeded:(?<succeeded>\S+))
| table _time, action_id, succeeded, server_principal_name, server_instance_name

[richgalloway · Accepted solution]

Search-time extractions are preferred over index-time extractions, because they use less storage (none) and don't slow down indexing.

You can extract fields automatically at search time by adding EXTRACT settings in the sourcetypes's props.conf stanza.

[xmlwineventlog]
EXTRACT-s_p_n = (server_principal_name:(?<server_principal_name>\S+)) in EventData_Xml 
EXTRACT-s_i_n = (server_instance_name:(?<server_instance_name>\S+)) in EventData_Xml 
EXTRACT-a_i = (action_id:(?<action_id>\S+)) in EventData_Xml 
EXTRACT-succeeded = (succeeded:(?<succeeded>\S+)) in EventData_Xml