Splunk Search

Splunk Data Input Whitelist: Regex not matching

beat_grob
Engager

I am trying to add a directory input monitor to Splunk. In this directory I have many different CSV files. Since there are different types of CSV files in there, I'd like to create a data input for each CSV type.

The file I am trying to filter is named like this:
11046819.2017-07-07_23-26-33.messages.csv
11046868.2017-07-08_00-58-38.messages.csv

The files I am trying to avoid are named like this:
11046868.1.2017-07-08_15_23_01.transform_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.print_properties.cfg.csv
11046868.1.2017-07-08_15_23_01.positional_attributes.csv
11046868.1.2017-07-08_15_23_01.policy_properties.cfg.csv

Currently there are 8753 files in the directory, of which only 1094 are relevant. My Splunk Data Input configuration looks like this:
data input configuration

After creating the input it shows me that it found 8753 files (amounting to the total number of files in that directory). However, it did not index any records, when I search for entries from that input type, no record is being displayed.

If I remove the whitelist regex all files are being indexed and I can see them in the log viewer. But that doesn't allow me to create different extraction strategies for the different CSV file types.

I have also tried using "*.messages.csv" and "*messages.csv", all of which produce the same outcome. What am I doing wrong?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

These two regex strings work on regex101.com with your sample file names.

.*\.messages\.csv

\d+\.\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}\.messages\.csv
---
If this reply helps you, Karma would be appreciated.

beat_grob
Engager

Thanks for the response. I had tried that, but it didn't work. After creating and assigning a new index, it started to work.

0 Karma
Get Updates on the Splunk Community!

The OpenTelemetry Certified Associate (OTCA) Exam

What’s this OTCA exam? The Linux Foundation offers the OpenTelemetry Certified Associate (OTCA) credential to ...

From Manual to Agentic: Level Up Your SOC at Cisco Live

Welcome to the Era of the Agentic SOC   Are you tired of being a manual alert responder? The security ...

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 4)

Welcome back to Splunk Classroom Chronicles, our ongoing series where we shine a light on what really happens ...