I have two types of log files, one is supposed to record security related events (sourcetype = sec) and the other to record the rest system events (sourcetype = sys).
But I found a small set of security events cross written to both types of the files. One of such log example is as follows (I displayed host and sourcetype using splunk)
2011-06-06T18:16:29 127.0.0.1 [NET-SECURITY]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/service radiusd restart
host=TT_June2011 Options| sourcetype = sec Options
2011-06-06T18:16:29 127.0.0.1 [NET-SECURITY]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/service radiusd restart
host=TT_June2011 Options| sourcetype = sys Options
How to identify all the log events/entries written to both of the files. At least is it possible to find if the security events written to the "sys" type file is a subset of all the security events written to the "sec" type file?
The suggested "punct" does not work. do I need to write python script to compare and identify the duplicate messages written to both files?
I found a solution throught this post.
http://splunk-base.splunk.com/answers/432/how-do-i-find-all-duplicate-events
The search that works is
sourcetype=sec OR sourcetype=sys | eval dupfield=_raw | transaction dupfield maxspan=1s keepevicted=true | where mvcount(sourcetype) > 1
I found a solution throught this post.
http://splunk-base.splunk.com/answers/432/how-do-i-find-all-duplicate-events
The search that works is
sourcetype=sec OR sourcetype=sys | eval dupfield=_raw | transaction dupfield maxspan=1s keepevicted=true | where mvcount(sourcetype) > 1
what about splunk special field punct ?
You could try a search similar to this :
sourcetype="sec" OR sourcetype="sys" | where mvcount(punct) > 1 | where mvcount(sourcetype) > 1
thanks. why did I receive "0" matching event after running the above suggested search. I obtained "0" matching event even after the first where, i.e. after where mvcount(punct) > 1? However, I can observe the idential log entries in the both sourcetypes (they are from differnet files).