Splunk Search

Wildcards and Regex(s) in Windows OS (UF) Monitor Paths (inputs.conf)?

kfeagans_splunk
Splunk Employee
Splunk Employee

Without any examples of Windows UF Monitor Paths (Universal Forwarder), it's pretty tough to figure out just what works and what doesn't. For example, the mixing of regex and wildcards, especially on Window is daunting. (for me at least).

IE: inputs.conf

[monitor://C:\OnState\Instances\k*01\ControlCP\]
whitelist = CPlog\d{0,3}\.txt$

Directory listing looks like:
C:\OnState\Instances\kfeagans01\ControlCP\CPlog01.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog02.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog03.txt
etc.

This I couldn't get to work, and it's generally because of mixing wildcards and regex between the monitor path and whitelist. Am I wrong here? There is an order to wildcards/regex(s) that must be followed that is outlined here (but without Windows examples, it's not useful): Wildcards and regular expression metacharacters

Thanks, Kelly

1 Solution

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

View solution in original post

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

ppablo
Retired

No problem Kelly 🙂 and awesome, you got it! More karma points for you!

0 Karma

kfeagans_splunk
Splunk Employee
Splunk Employee

Oh duh ... check mark next to the answer. 🙂 Accepted?

kf

kfeagans_splunk
Splunk Employee
Splunk Employee

Hi Patrick .. thanks! How to accept the answer? I don't readily see a button or pulldown or?

Thanks!

kf

0 Karma

ppablo
Retired

Hi @kfeagans_splunk

Glad you found a solution to your question! Please be sure to accept your answer to your post so other people with similar issues will refer to it for help 🙂 Thanks!

Patrick

Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...