Splunk Search

Wildcards and Regex(s) in Windows OS (UF) Monitor Paths (inputs.conf)?

kfeagans_splunk
Splunk Employee
Splunk Employee

Without any examples of Windows UF Monitor Paths (Universal Forwarder), it's pretty tough to figure out just what works and what doesn't. For example, the mixing of regex and wildcards, especially on Window is daunting. (for me at least).

IE: inputs.conf

[monitor://C:\OnState\Instances\k*01\ControlCP\]
whitelist = CPlog\d{0,3}\.txt$

Directory listing looks like:
C:\OnState\Instances\kfeagans01\ControlCP\CPlog01.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog02.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog03.txt
etc.

This I couldn't get to work, and it's generally because of mixing wildcards and regex between the monitor path and whitelist. Am I wrong here? There is an order to wildcards/regex(s) that must be followed that is outlined here (but without Windows examples, it's not useful): Wildcards and regular expression metacharacters

Thanks, Kelly

1 Solution

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

View solution in original post

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

ppablo
Retired

No problem Kelly 🙂 and awesome, you got it! More karma points for you!

0 Karma

kfeagans_splunk
Splunk Employee
Splunk Employee

Oh duh ... check mark next to the answer. 🙂 Accepted?

kf

kfeagans_splunk
Splunk Employee
Splunk Employee

Hi Patrick .. thanks! How to accept the answer? I don't readily see a button or pulldown or?

Thanks!

kf

0 Karma

ppablo
Retired

Hi @kfeagans_splunk

Glad you found a solution to your question! Please be sure to accept your answer to your post so other people with similar issues will refer to it for help 🙂 Thanks!

Patrick

Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...