Getting Data In

Why is wildcard not working in log file name for input.conf?

phamxuantung
Communicator

Hello, I have the input.conf for several log files as

 

[monitor:///u01/mnt/log-1/data/trafficmanager/access/*]
index = myindex
sourcetype = csvtype
initCrcLength = 1048576

 

The log file name is structured as access_worker_*_YYYY_mm_dd.log. For example: access_worker_5_03_21.log, access_worker_6_03_21.log, access_worker_5_03_20.log, etc.

The stanza that I put in don't work so I try for a specific file name, such as

 

[monitor:///u01/mnt/log-1/data/trafficmanager/access/access_worker_5_03_21.log]
index = myindex
sourcetype = csvtype
initCrcLength = 1048576

 

 Then the log was pull in no problem. The problem that I see is the way I use my wildcard, somehow don't catch all the log file that I want to monitor.

Can anyone point out how to fix this problem?

Labels (2)
Tags (2)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @phamxuantung ,

there's no reason for this behavior,

please try with this header:

[monitor:///u01/mnt/log-1/data/trafficmanager/access/access_worker_*.log]

Ciao.

Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @phamxuantung ,

are you sure that the problem is the filename?

check if the not read file has the same content (event if a different file name) of an already ingested file, because Splunk doesn't index twice a file.

If the files to read always have diferent filename, you could try to add the following option:

crcSalt = <SOURCE>

Ciao.

Giuseppe

0 Karma

phamxuantung
Communicator

The files always have different name and the content are also different. The different between my 2 config (where one work and one don't) is the path and file name

[monitor:///u01/mnt/log-1/data/trafficmanager/access/*]

-> This one don't work

vs

[monitor:///u01/mnt/log-1/data/trafficmanager/access/access_worker_5_03_21.log]

-> This does but can only index that specific file "access_worker_5_03_21.log" but not the other with similar name.

So the problem I think lie in how I use my wildcard.

I added crcSalt =<SOURCE> but it don't work.

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @phamxuantung ,

there's no reason for this behavior,

please try with this header:

[monitor:///u01/mnt/log-1/data/trafficmanager/access/access_worker_*.log]

Ciao.

Giuseppe

phamxuantung
Communicator

Your stanza work and the log files are indexed as normal. I just wonder why Splunk can't catch all file with the wildcard, so weird.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @phamxuantung ,

as I said it's a strange behavior that I never experienced before, anyway I'm happy that you solved your issue.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...