Getting Data In

multiple fschange issues

tawollen
Path Finder

I have a few issues when trying to use fschange.

  1. even though fullEvent = true & sendEventMaxSize = -1, I am still getting one line per event/file.

  2. even though I have souretype = changed_files, I am getting other sourcetypes. (I get csv-2 for CSV files, conf-too_small, etc). I do get changed_files when source=fschangemonitor, but not when I look for the changes to the files themselves

  3. Even though I have whitelist/blacklist, I am still getting files that are not listed in whitelist (e.g. path="/opt/splunk/etc/system/local/.inputs.conf.swp or web.conf.old")

  4. trying to monitor /opt/splunk/etc/system/local & /opt/splunk/etc/system/local/authentication with one directory.

I have reviewed the following pages, and they seem to contradict each other in the format for placement of options and stanza order. http://www.splunk.com/base/Documentation/latest/AppManagement/Configurationmonitoring http://www.splunk.com/base/Documentation/latest/Admin/Monitorchangestoyourfilesystem

[fschange:/opt/splunk/etc/system/local]
sourcetype = changed_files
index = test
filters = configs,terminal-blacklist
recurse = true
followLinks = false
signedaudit = false
pollPeriod=30
fullEvent = true
sendEventMaxSize = -1
delayInMills = 1000

[filter:whitelist:configs]
regex1 = \.conf$
regex2 = \.py$
regex3 = \.csv$
regex4 = authentication

[filter:blacklist:terminal-blacklist]
regex1 = .?
Tags (1)
0 Karma
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

This has been reported to support and is a known issue in 4.1.4 +.

See the following: splunk.com/base/Documentation/4.1.4/ReleaseNotes

You may be able to workaround this by creating a whitelist that excludes explicitly the files you'd normally blacklist.

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

This has been reported to support and is a known issue in 4.1.4 +.

See the following: splunk.com/base/Documentation/4.1.4/ReleaseNotes

You may be able to workaround this by creating a whitelist that excludes explicitly the files you'd normally blacklist.

tawollen
Path Finder

I am getting fields that include files that a) are not in the whitelist, b) have not been deleted (or changed)

The fschange part of the stanza is now:

[fschange:/opt/splunk/etc/system/local]
index = test

fullEvent = true

filters = configs,terminal-blacklist
recurse = true
pollPeriod=60
delayInMills = 1000

Wed Nov 3 18:00:01 2010 action=delete, path="/opt/splunk/etc/system/local/web.conf-taw"
Wed Nov 3 18:00:01 2010 action=delete, path="/opt/splunk/etc/system/local/transforms.conf.bak"
Wed Nov 3 18:00:01 2010 action=delete, path="/opt/splunk/etc/system/local/props.conf.bak"

0 Karma

tawollen
Path Finder

I removed regex4, and that seemed to fix the issue with blacklisted files getting indexed (authentication is a directory I have under system/local). I may just have to do multiple fschange stanzas

When I removed all filters, Splunk indexed "README" file, that showed up all in one event and with the sourcetype=misc_text.

So, it seems that if the sourcetype is csv-*, or *_too_small, it won't put it all in one event.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...