Getting Data In
Highlighted

How to set up file monitoring for this folder structure?

New Member

Hi, I want to setup the file monitoring for all the files starting with "mq-" or "secs-" or "err-" in below directory:

/var/mtapps/ashl/logs/[folder A]/[folder B]/[folder C]/

because there are many folder A folder B and Folder C with different names so we setup as below:

alt text

but it doesn't work. all 3 kinda files doesn't go into the splunk server.

how can I resolve this? Thanks in advance!

Tags (3)
0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

Builder

Can you try this in inputs.conf -

[monitor:///var/mtapps/ashl/logs/*/*/*/mq-*]
index = index_name
sourcetype = sourcetype_name
crcSalt=<SOURCE>

[monitor:///var/mtapps/ashl/logs/*/*/*/secs-*]
index = index_name
sourcetype = sourcetype_name
crcSalt=<SOURCE>

[monitor:///var/mtapps/ashl/logs/*/*/*/err-*]
index = index_name
sourcetype = sourcetype_name
crcSalt=<SOURCE>
0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

New Member

yes. this is how we setup, but it doesnt work. you can see my screenshot below.
alt text

0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

Builder

Can you check if there is any permission issue for these logs. Since splunk is unable to access them.

index=_internal sourcetype=splunkd host=<log_server> "var/mtapps/ashl/logs"

Also try running the command to check if these logs are monitored on the log server -

./splunk list monitor -auth admin:<password>
0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

New Member

sorry for not making myself clear. yes, it can access. in the inputstatus, I can see it run through all folders for mq-* look up (strange thing is it only manage to run mq-, but not err- and secs-. Even for mq- files, I wait for more than 5 hours (set up at 9:30am today and now is 3:13pm) it only manage to process 1 file with 91.7% uploaded) this doesn't look correct to me. so I suspect it is because of the monitor string setup is not correct.

/var/mtapps/ashl/logs/Rorze/SorterRSC/RBWSA22200/mq-sRBWSA22200.log.tu

file position 24332126
file size 26533134
parent /var/mtapps/ashl/logs////mq-
percent 91.70
type open file

0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

Builder

Can you try having a single monitor for all -

[monitor:///var/mtapps/ashl/logs/.../.../.../(mq|secs|err)-*)]

Splunk should have read the events as and when the logs are written. Try checking internal logs if they print any errors and adding crcSalt(as shown above).

0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

New Member

Hi Dineshraj9,

I want to monitor them under 3 different source type. is it possible to do that?

0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

Motivator

Hi,

you can try this,

[monitor:///var/mtapps/ashl/logs/*/*/*/mq-*]
index = index_name
sourcetype = sourcetype_name1
crcSalt=
blacklist.1=secs-.*
blacklist.2=err-.*

[monitor:///var/mtapps/ashl/logs/*/*/*/secs-*]
index = index_name
sourcetype = sourcetype_name2
crcSalt=
blacklist.1=mq-.*
blacklist.2=err-.*

[monitor:///var/mtapps/ashl/logs/*/*/*/err-*]
index = index_name
sourcetype = sourcetype_name3
crcSalt=
blacklist.1=mq-.*
blacklist.2=secs-.*

Regards,
Badri Srinivas B

0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

Motivator

at the end is mq-, secs- and err-*

0 Karma
Highlighted

Re: How to set up file monitoring for this folder structure?

New Member

I've just change the input.conf to below:

[monitor:///var/mtapps/ashl/logs/.../.../.../mq-*]

recursive=true

sourcetype = mqhist
index = automation
disabled=1

[monitor:///var/mtapps/ashl/logs/.../.../.../err-*]

recursive=true

sourcetype = hosterror
index = automation
disabled=0

[monitor:///var/mtapps/ashl/logs/.../.../.../secs-*]

recursive=true

sourcetype = secs
index = automation
disabled=1

if I enable all 3 soucetypes, none of files will go in. but if I enable only one, it works for that sourcetype.

how can I do in order to enable all 3 types???

0 Karma