I got quite some events coming in, so exemplarily I copied two, one with action=add and one with action=delete
Interesting enough the add / update events are correctly classified and the sourcetype matches the app, but for the delete events, the sourcetyp does not match the app and I don't actually see a reason why.
Maybe anybody has an idea on what might be the issue ?
Events
7/31/13 11:03:50.610 AM
Wed Jul 31 11:03:50 2013 action=delete, path="/opt/appA/config/resource/file"
host=host1 sourcetype=appB source=fschangemonitor index=core action=delete
Wed Jul 31 11:11:56 2013 action=add, path="/opt/appA/config/resource/file", isdir=0, size=1199, gid=5340, uid=5340, modtime="Thu Jun 13 10:51:58 2013", mode="r--r-----", hash=W0ReV7n8TIIUmccmbIX4xHHPiWHNNH2j1HQ3PK0qlqg=
host=host1 sourcetype=appA source=fschangemonitor index=core action=add
inputs.conf
[fschange:/opt/config/AppAsomename/resource]
index=core
followLinks=false
recurse=true
pollPeriod=60
hashMaxSize=100000000
fullEvent=true
sourcetype= appAsomename
Props.conf
[source::/opt/config/AppAsomename/resource/...]
LEARN_SOURCETYPE=false
LEARN_MODEL=false
sourcetype= appAsomename
inputs.conf
[fschange:/opt/config/AppA/resource]
index=core
followLinks=false
recurse=true
pollPeriod=60
hashMaxSize=100000000
fullEvent=true
sourcetype= appA
props.conf
[source::/opt/config/AppA/resource/...]
LEARN_SOURCETYPE=false
LEARN_MODEL=false
sourcetype= appA
Seems to be related to this one, anybody can confirm this ?
http://answers.splunk.com/answers/40799/fs-change-keeps-adding-and-deleting-files-from-monitoring?pa...
It seems that those delete events never actually happened, so there are some kind of ghost delete events (version version 5.0.2, build 149561 )
It seems to be some kind of circular action delete-add-delete-add that actually does not happen on the filesystem
Actually the only two stanzas that include the mentioned path are the two posted (I had to change the names for privacy reasons)
Both directory names start with the same word but the second app has some letters added to (pretty much like AppAsomename) it's end, but besides that there is no overlapping as far as I could see.
This mismatch ONLY happens for delete events NOT for add / update events, with the same hosts reporting the stuff and the same config (as far as I know)
Can you include whatever relevant props apply to 'appB'? I don't think Splunk is going to invent sourcetypes on its own. I'd suggest that there may be fschange: stanzas with overlapping paths, and that it's the second one (appB) which is picking up the delete.