Splunk Search

Splunk Data Input Whitelist: Regex not matching

beat_grob
Engager

I am trying to add a directory input monitor to Splunk. In this directory I have many different CSV files. Since there are different types of CSV files in there, I'd like to create a data input for each CSV type.

The file I am trying to filter is named like this:
11046819.2017-07-07_23-26-33.messages.csv
11046868.2017-07-08_00-58-38.messages.csv

The files I am trying to avoid are named like this:
11046868.1.2017-07-08_15_23_01.transform_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.print_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.positional_attributes.csv
11046868.1.2017-07-08_15_23_01.policy_properties.cfg.csv

Currently there are 8753 files in the directory, of which only 1094 are relevant. My Splunk Data Input configuration looks like this:
data input configuration

After creating the input it shows me that it found 8753 files (amounting to the total number of files in that directory). However, it did not index any records, when I search for entries from that input type, no record is being displayed.

If I remove the whitelist regex all files are being indexed and I can see them in the log viewer. But that doesn't allow me to create different extraction strategies for the different CSV file types.

I have also tried using "*.messages.csv" and "*messages.csv", all of which produce the same outcome. What am I doing wrong?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

beat_grob
Engager

Thanks for the response. I had tried that, but it didn't work. After creating and assigning a new index, it started to work.

0 Karma
Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...