I am trying to implement file integrity monitoring. I have configured fschange as follows:
[fschange:/opt/bea/10_sp0_ltf/production/properties]
sourcetype = wls_monitor
index = fileint
disabled = false
_whitelist = \.xml$
recurse = false
pollPeriod = 600
fullEvent = true
sendEventMaxSize = -1
I'm having two problems. The first is that the sourcetype is not being set to "wls_monitor" but is instead showing up in the index as "xml-5".
The second issue is that each line of the monitored file is showing up as an individual event instead of the modified file being it's own event.
In addition, the events aren't showing up as having a source of fschangemonitor.
There are a couple problems here.
You should be seeing events with the wls_monitor
sourcetype, but that will only contain the actual audit events (notifying you of a new file, removed file, or modified file),
Here is an example of what these audit events will look like:
Wed Apr 28 21:52:42 2010 action=add, path="/path/to/myfile.cab", isdir=0, size=1401528, gid=65534, uid=1002, modtime="Fri Sep 19 13:57:19 2008", mode="r--r--r--", hash=d7l6zIDYsY7GyoC23M6QX5xQNaci3ulHyuqKn3kb0YM=
fschange also has the option of letting you store a copy of the file when a new file is created, or when an existing file is modified. I'm assuming that you want to do this based on your question. So you should have fullEvent=true
as part of your config. Now, when the content of your event is fed into splunk, it uses the same source indentify rules as a "normal" input (for example, inputs using [monitor://]
). So you will need to setup a matching entry in props.conf
to establish your indexing options (such as sourcetype). I'm assuming that your looking at XML config files, so you will want these files indexed in one big chunk.
Also, _whitelist
is only applicable for [monitor://]
entries, not [fschange:]
entries.
Try something like this:
inputs.conf:
[fschange:/opt/bea/10_sp0_ltf/production/properties] sourcetype = wls_monitor index = fileint filters = xml_files, terminal-blacklist recurse = false pollPeriod = 600 fullEvent = true [filter:whitelist:xml_files] regex1 = \.xml$ [filter:blacklist:terminal-blacklist] regex1 = .?
props.conf:
[source:/opt/bea/.../properties/*.xml] sourcetype = wls_xml [wls_xml] LINE_BREAKER = ^()$ TRUNCATE = 1000000 SHOULD_LINEMERGE = false DATETIME_CONFIG = NONE CHECK_METHOD = modtime KV_MODE = none LEARN_MODEL = false
You could also use the config_file
sourcetype which splunk provides in the unix
app, instead of using the wls_xml
sourcetype that I made up.
Docs:
I think I found it. I had source:
Also, this page is where I got the bad info on _whitelist.
http://www.splunk.com/base/Documentation/4.1.1/AppManagement/Configurationmonitoring
I'm getting closer. I'm now seeing events with a source of fschangemonitor. This is good. However, when the file is changed it's now being indexed as both single lines and as one large file. In both cases they still have a sourcetype xml-5.
I do have the following in local/props.conf
[source:/opt/bea/10_sp0_ltf/production/properties/*.xml]
sourcetype = wls_xml
[wls_xml]
LINE_BREAKER = ^()$
TRUNCATE = 1000000
SHOULD_LINEMERGE = false
DATETIME_CONFIG = NONE
CHECK_METHOD = modtime
KV_MODE = none
LEARN_MODEL = false
Thanks for the help!
There are a couple problems here.
You should be seeing events with the wls_monitor
sourcetype, but that will only contain the actual audit events (notifying you of a new file, removed file, or modified file),
Here is an example of what these audit events will look like:
Wed Apr 28 21:52:42 2010 action=add, path="/path/to/myfile.cab", isdir=0, size=1401528, gid=65534, uid=1002, modtime="Fri Sep 19 13:57:19 2008", mode="r--r--r--", hash=d7l6zIDYsY7GyoC23M6QX5xQNaci3ulHyuqKn3kb0YM=
fschange also has the option of letting you store a copy of the file when a new file is created, or when an existing file is modified. I'm assuming that you want to do this based on your question. So you should have fullEvent=true
as part of your config. Now, when the content of your event is fed into splunk, it uses the same source indentify rules as a "normal" input (for example, inputs using [monitor://]
). So you will need to setup a matching entry in props.conf
to establish your indexing options (such as sourcetype). I'm assuming that your looking at XML config files, so you will want these files indexed in one big chunk.
Also, _whitelist
is only applicable for [monitor://]
entries, not [fschange:]
entries.
Try something like this:
inputs.conf:
[fschange:/opt/bea/10_sp0_ltf/production/properties] sourcetype = wls_monitor index = fileint filters = xml_files, terminal-blacklist recurse = false pollPeriod = 600 fullEvent = true [filter:whitelist:xml_files] regex1 = \.xml$ [filter:blacklist:terminal-blacklist] regex1 = .?
props.conf:
[source:/opt/bea/.../properties/*.xml] sourcetype = wls_xml [wls_xml] LINE_BREAKER = ^()$ TRUNCATE = 1000000 SHOULD_LINEMERGE = false DATETIME_CONFIG = NONE CHECK_METHOD = modtime KV_MODE = none LEARN_MODEL = false
You could also use the config_file
sourcetype which splunk provides in the unix
app, instead of using the wls_xml
sourcetype that I made up.
Docs:
Yeah. I have no officialness, but I can tell that's how I've seen it behave. (In fact I just fixed an issue caused by this on one of my systems yesterday.) I do agree that you would normally want to index the file as one big event, but you don't have to. For example, say your watching a directory *.csv
config files. In that case you would probably want to index the file on a row-by-row basis instead of just one big event.
Are you sure? I haven't spent much time with fschange, but I thought it always generated one event per file update.