topic multiple fschange on the same files in Getting Data In

multiple fschange on the same files

imacdonald2 — Mon, 06 Jun 2011 22:01:22 GMT

Will this cause any issues, I would like to do

[fschange:/etc]
index = linux-security
recurse = true
followLinks = false
signedaudit = false
fullEvent = false
delayInMills = 1000
pollPeriod = 600
sourcetype = linux_etc_fschange
hashMaxSize = 1048576

and

[filter:whitelist:files]
regex1 = ^passwd$
regex2 = ^group$


[fschange:/etc/]
index = linux-security
recurse = true
followLinks = false
signedaudit = false
fullEvent = true
delayInMills = 1000
pollPeriod = 600
sourcetype = linux_etc_full_fschange
hashMaxSize = 1048576
filters = files

I know you are not support to use two monitory inputs against the same files, but I didn't see anything in the documentation that says this would be a bad idea for fschange. My goal is to monitor /etc/passwd and /etc/group for changes and to log the files on a change so I can easily review user adds/changes/deletes.

Re: multiple fschange on the same files

jbsplunk — Mon, 06 Jun 2011 22:11:35 GMT

This isn't a great idea for fschange. You are going to run into some odd behaviors that might not be entirely consistent. Also, I don't think you need the 'hashMaxSize' value, so I would suggest you leave it off. I've seen this cause continual updates to files on certain systems and wouldn't use this unless it is explicitly needed.

I would suggest that you remove your whitelist and sourcetyping from inputs.conf as it seems your goal is to get different sourcetypes applied to these inputs. You can use props/transforms to take care of sourcetyping.

So keep the fschange stuff that you've got to monitor the entire /etc/ folder, I think that should work. The unix app does something similar, which you can look at if you'd like, its similar to this:

[monitor:///etc]
_whitelist=REGEX

[config_file]
LINE_BREAKER = ^()$
TRUNCATE = 1000000
SHOULD_LINEMERGE = false
DATETIME_CONFIG = NONE
CHECK_METHOD = modtime
KV_MODE = none
pulldown_type = true
SEGMENTATION-all      = whitespace-only
SEGMENTATION-inner    = whitespace-only
SEGMENTATION-outer    = whitespace-only
SEGMENTATION-standard = whitespace-only
LEARN_MODEL = false

[source::/etc/(passwd|group)]
sourcetype=config_file
CHECK_METHOD = modtime

setting the sourcetype to config_file and then calling this line breaker indexes the whole file.

Re: multiple fschange on the same files

imacdonald2 — Mon, 06 Jun 2011 22:21:12 GMT

So the goal is to be able to do a diff on just /etc/passwd and /etc/group from with in splunk, but have all other files in /etc be logged for changes but not store the contents of the files. if having 2 fschange directives on the same directory is not recommended, do you have any suggestions?

I will be using the same sourcetype when I roll this to prod, its different just for testing

After posting I discovered that doing a whitelist on its own didn't seem to limit it to just those two files, so I added the following

[filter:blacklist:blfiles]
regex1 = .*

Re: multiple fschange on the same files

jbsplunk — Mon, 06 Jun 2011 23:03:42 GMT

I edited my answer to reflect your goals. I THINK this has a good chance of being a solution.

Re: multiple fschange on the same files

imacdonald2 — Tue, 07 Jun 2011 17:52:52 GMT

Thanks, I am testing it.

And to confirm what you were saying having two [fschange:/etc/] entries does not work, it keeps adding and removing all the files in the /etc/

Re: multiple fschange on the same files

imacdonald2 — Wed, 08 Jun 2011 02:20:10 GMT

I have this working if I split it out into

[source::/etc/passwd]
and
[source::/etc/group]

but

[source::/etc/(passwd|group)]

Isn't working any suggestions?

Re: multiple fschange on the same files

imacdonald2 — Wed, 08 Jun 2011 03:03:14 GMT

Interesting the following works

[source::*/etc/(passwd|group)]

but

[source::/etc/(passwd|group)]

doesn't

Re: multiple fschange on the same files

jbsplunk — Wed, 08 Jun 2011 05:37:20 GMT

That is interesting, but I didn't test the specific configuration. Glad to hear a wildcard takes care of it though.