multiple fschange issues

[tawollen]

I have a few issues when trying to use fschange.

1. even though fullEvent = true & sendEventMaxSize = -1, I am still getting one line per event/file.

2. even though I have souretype = changed_files, I am getting other sourcetypes. (I get csv-2 for CSV files, conf-too_small, etc). I do get changed_files when source=fschangemonitor, but not when I look for the changes to the files themselves

3. Even though I have whitelist/blacklist, I am still getting files that are not listed in whitelist (e.g. path="/opt/splunk/etc/system/local/.inputs.conf.swp or web.conf.old")

4. trying to monitor /opt/splunk/etc/system/local & /opt/splunk/etc/system/local/authentication with one directory.

I have reviewed the following pages, and they seem to contradict each other in the format for placement of options and stanza order. http://www.splunk.com/base/Documentation/latest/AppManagement/Configurationmonitoring http://www.splunk.com/base/Documentation/latest/Admin/Monitorchangestoyourfilesystem

[fschange:/opt/splunk/etc/system/local]
sourcetype = changed_files
index = test
filters = configs,terminal-blacklist
recurse = true
followLinks = false
signedaudit = false
pollPeriod=30
fullEvent = true
sendEventMaxSize = -1
delayInMills = 1000

[filter:whitelist:configs]
regex1 = \.conf$
regex2 = \.py$
regex3 = \.csv$
regex4 = authentication

[filter:blacklist:terminal-blacklist]
regex1 = .?

[jbsplunk · Accepted solution]

This has been reported to support and is a known issue in 4.1.4 +.

See the following: splunk.com/base/Documentation/4.1.4/ReleaseNotes

You may be able to workaround this by creating a whitelist that excludes explicitly the files you'd normally blacklist.

[tawollen]

I am getting fields that include files that a) are not in the whitelist, b) have not been deleted (or changed)

The fschange part of the stanza is now:

[fschange:/opt/splunk/etc/system/local]
index = test

fullEvent = true

filters = configs,terminal-blacklist
recurse = true
pollPeriod=60
delayInMills = 1000

Wed Nov 3 18:00:01 2010 action=delete, path="/opt/splunk/etc/system/local/web.conf-taw"
Wed Nov 3 18:00:01 2010 action=delete, path="/opt/splunk/etc/system/local/transforms.conf.bak"
Wed Nov 3 18:00:01 2010 action=delete, path="/opt/splunk/etc/system/local/props.conf.bak"

[tawollen]

I removed regex4, and that seemed to fix the issue with blacklisted files getting indexed (authentication is a directory I have under system/local). I may just have to do multiple fschange stanzas

When I removed all filters, Splunk indexed "README" file, that showed up all in one event and with the sourcetype=misc_text.

So, it seems that if the sourcetype is csv-*, or *_too_small, it won't put it all in one event.