topic Monitoring File Directories but not indexing file contents in Getting Data In

Monitoring File Directories but not indexing file contents

Highlander22 — Mon, 02 Nov 2020 21:45:35 GMT

Hi all,

Sorry for the really newb question (because I am one).

I have Splunk Enterprise running on my standalone PC to evaluate it. I have managed to get Splunk to monitor my PC's volumes, directories & files OK...but the **bleep** thing insists on trying to index the contents of every file too. Obviously, this is very resource-intensive and I really don't want the file contents indexed.

How do I stop it indexing the file contents? Or, even better, tell it not to index any file contents except for specific file extensions?

Thanks for any help.

Re: Monitoring File Directories but not indexing file contents

richgalloway — Tue, 03 Nov 2020 15:03:21 GMT

Indexing files is Splunk's reason for being.

Have a look at the fschange monitor in inputs.conf. It's a deprecated feature so it may so away in a future release, but it may be want you seek.

Re: Monitoring File Directories but not indexing file contents

gcusello — Tue, 03 Nov 2020 15:12:17 GMT

Hi @Highlander22,

you could create a scripted input that runs (e.g. every hour) a script containing the "dir" command (in Windows) or the "ls" command (in Linux).

in this way you have the list of the files of a folder, without indexing file contents, and you can compare the lists in different time periods.

ciao.

Giuseppe

Re: Monitoring File Directories but not indexing file contents

Highlander22 — Tue, 03 Nov 2020 16:56:48 GMT

Thanks @gcusello & @richgalloway 😊👍 for taking the time to respond.

I think I have the solution: I already had the "File Meta-data" app but was using the "Files & Directories" "Data Input" (which does index the contents of the file/all files in the directory) rather than the "File Meta-data" "Data Input" (which "Import[s] file and directory meta-data (size, modification dates, etc.)". This also adds a new "File Meta-data" entry in the "Add Data" "Monitor" sources too.

I have it timed to run every 24 hours so will have to wait 24 hours for it to have triggered at some time in the interim. The documentation on that "File Meta-data" app is rather spartan to say the least but I think I am on the right path. Just need a bit of trial and error now, I think.

I have upvoted both of you though 😊.

@gcusello- In my old programming days, yes, I would have created such a script but I expected that such integrated tools like Splunk should be able to integrate directly with the FS by now to at least do that for me 😊.

@richgalloway- If I can't get my expected solution to work (it bloody well should work), I will have a look at yours next. There's always different ways of skinning the same cat...though, yes, this is quite a primitive cat to have Splunk get its teeth into.