Splunk Search

Wildcards and Regex(s) in Windows OS (UF) Monitor Paths (inputs.conf)?

kfeagans_splunk
Splunk Employee
Splunk Employee

Without any examples of Windows UF Monitor Paths (Universal Forwarder), it's pretty tough to figure out just what works and what doesn't. For example, the mixing of regex and wildcards, especially on Window is daunting. (for me at least).

IE: inputs.conf

[monitor://C:\OnState\Instances\k*01\ControlCP\]
whitelist = CPlog\d{0,3}\.txt$

Directory listing looks like:
C:\OnState\Instances\kfeagans01\ControlCP\CPlog01.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog02.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog03.txt
etc.

This I couldn't get to work, and it's generally because of mixing wildcards and regex between the monitor path and whitelist. Am I wrong here? There is an order to wildcards/regex(s) that must be followed that is outlined here (but without Windows examples, it's not useful): Wildcards and regular expression metacharacters

Thanks, Kelly

1 Solution

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

View solution in original post

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

ppablo
Retired

No problem Kelly 🙂 and awesome, you got it! More karma points for you!

0 Karma

kfeagans_splunk
Splunk Employee
Splunk Employee

Oh duh ... check mark next to the answer. 🙂 Accepted?

kf

kfeagans_splunk
Splunk Employee
Splunk Employee

Hi Patrick .. thanks! How to accept the answer? I don't readily see a button or pulldown or?

Thanks!

kf

0 Karma

ppablo
Retired

Hi @kfeagans_splunk

Glad you found a solution to your question! Please be sure to accept your answer to your post so other people with similar issues will refer to it for help 🙂 Thanks!

Patrick

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...