topic Re: How to generate the regex to extract distinct values of this field? in Splunk Search

How to generate the regex to extract distinct values of this field?

lsy9891 — Wed, 28 Aug 2019 07:19:22 GMT

Hi,

I have events with the field WindowsIdentity. Some examples of this field values are:

WindowsIdentity: IIS APPPOOL\login20.monster.com
IIS APPPOOL\ jobs.monster.com
IIS APPPOOL\ hiring.channels.monster.com_jcm
IIS APPPOOL\ wwwcs.channels.monster.com

I tried extracting it with the IFX and I used it like this: rex field=WindowsIdentity "(?P\w+)" but it extracts IIS instead of the text highlighted in bold?

Re: How to generate the regex to extract distinct values of this field?

KailA — Wed, 28 Aug 2019 11:28:37 GMT

Hello,

You can try this regex

| rex field=WindowsIdentity "(?<yourNewField>[\w\.]+)\.monster"

This is an example :

| makeresults
| eval WindowsIdentity = "IIS APPPOOL\ jobs.monster.com"
| rex field=WindowsIdentity "(?<yourNewField>\w+)\.monster"

Re: How to generate the regex to extract distinct values of this field?

lsy9891 — Thu, 29 Aug 2019 02:34:24 GMT

Hi, the rex command extracted some important info from the URL. For example,

IIS APPPOOL\career-advice.monster.com should be extracted to career-advice and not channels. Basically, anything that's between \ and monster.com. How should I fix that?

Re: How to generate the regex to extract distinct values of this field?

memarshall63 — Thu, 29 Aug 2019 03:27:19 GMT

 | rex field=WindowsIdentity "IIS APPPOOL\\(?P<yourNewField>[.\w]+).monster"

Re: How to generate the regex to extract distinct values of this field?

lsy9891 — Thu, 29 Aug 2019 05:26:20 GMT

I got Error in 'rex' command: Encountered the following error while compiling the regex 'IIS APPPOOL(?P[.\w]+).monster': Regex: unmatched closing parenthesis

Re: How to generate the regex to extract distinct values of this field?

KailA — Thu, 29 Aug 2019 08:18:01 GMT

Hello,

This regex is a bit greedy but its doing exactly what you ask:

| makeresults
 | eval WindowsIdentity = "IIS APPPOOL\career-advice.monster.com"
 | rex field=WindowsIdentity "\\\\(?<yourNewField>.*)\.monster"

If you need another regex that is a bit more safe you can use this one :

| makeresults
| eval WindowsIdentity = "IIS APPPOOL\career-advice.monster.com"
| rex field=WindowsIdentity "\\\\(?<yourNewField>[\w\-\.]+)\.monster"

And if it's not matching a particular character you can add it to this part [\w\-\.]+

Re: How to generate the regex to extract distinct values of this field?

memarshall63 — Thu, 29 Aug 2019 12:04:22 GMT

Yep. I like @KailA 's last one best. I'm suffering from \\ (backslash-backslash) whiplash. 🐵 Suggest letting https://regex101.com/ help you get to the end of it.

Re: How to generate the regex to extract distinct values of this field?

wenthold — Thu, 29 Aug 2019 13:07:10 GMT

I'm not entirely clear how your messages are actually formatted - is "WindowsIdentity" the name of the field and also part of the field? are all these lines part of the same message and you want to skip the first value - "WindowsIdentity: IIS APPPOOL\login20.monster.com"?

Maybe this will work:

{ ... base search ... }
| rex max_match=100 field=WindowsIdentity "IIS APPPOOL\\\+\s?(?<App>.*?)\.monster\.com"
| eval app_count=mvcount(App) | eval App=mvindex(App,1,app_count)