topic Re: Regex Field Extraction in Data Model in Splunk Search

Regex Field Extraction in Data Model

han — Mon, 10 Jan 2022 10:12:02 GMT

Hi all. I'm fairly new to Splunk and regex. I've got many event logs and I'm making use of data models beforing generating different visualisations.

The fields discovered isn't good enough for my usecase thus I need to extract specific fields. Right now, using the following regex

(?<field_name>(([a-zA-Z]+(\.[a-zA-Z]+)+)_([a-zA-Z]+(|[a-zA-Z]+)+)|/^([^.]+)/))

, I'm able to extract this pattern

ABC|DEF|GHI

most accurately.

Subsequently, I would like extract each respective word into its own field. In total 3 different fields for ABC, DEF and GHI respectively.

Is there a way I extract each individual word? How can perform regex expression on top of my current regex expression result? Thank you.

Re: Regex Field Extraction in Data Model

isoutamo — Mon, 10 Jan 2022 11:45:37 GMT

excellent place to check and training with your regex is regex101.com. In there you could try with visual feedback how those are working.

To separate those to own variables can be do at least next ways.

... base search ... | rex "(?<field 1>[rex for field one])...(?<f2>[rex for f2])...(?<f3>[rex for f3])"

... base search ... | rex "(?<field 1>[rex for field one])" | rex "(?<f2>[rex for f2])" | rex "(?<f3>[rex for f3])"

But which one is better for you? As usually it depends on what your events are. Are those always in same order and all in place (then probably first one is better). Or can order be different and are those all present? In this case probably second one is better. Of course in second one you must separate those events from each other to get values to correct field name. For that reason you must add some additional strings etc. to those rex to match to the correct part of event.

And one option to use is add those extractions to props.conf and transforms.com especially when you need those in several queries.

Two other commands which can help you https://docs.splunk.com/Documentation/Splunk/8.2.4/SearchReference/Extract and https://docs.splunk.com/Documentation/Splunk/8.2.4/SearchReference/Erex

r. Ismo

Re: Regex Field Extraction in Data Model

han — Mon, 10 Jan 2022 15:09:15 GMT

Hey @isoutamo. Thanks for the suggestion. However, my question is specific to forming the regex expression in the Regular Expression box within "App/Search/data_model_editor" of Splunk, and not about forming a search query as you've suggested.