topic Re: How can I show the difference in a string field from one day to another? in Splunk Search

How can I show the difference in a string field from one day to another?

bscavotto — Wed, 09 May 2018 15:45:52 GMT

I have a powershell script that audits some files and creates an Windows application event log with the filepaths of any matches in the Message field. It runs once per day and creates a single event log everyday. I only want the diff from one day to the next as new things are found.

My query so far only gives me the split listing of the Message field contents and the count but how do I get it to only show me the new entries as compared to the previous day?

sourcetype=WinEventLog:Application SourceName=MessageAudit |eval NewMessage=split(Message,"\\\\") |eval NewMessageCount=mvcount(NewMessage) |table NewMessage,NewMessageCount | search NewMessageCount>36
(There are 36 consistent false positives)

Thanks.

Re: How can I show the difference in a string field from one day to another?

skoelpin — Wed, 09 May 2018 17:49:17 GMT

Perhaps the diff command would work here

sourcetype=WinEventLog:Application SourceName=MessageAudit |eval NewMessage=split(Message,"\\\\") |eval NewMessageCount=mvcount(NewMessage) |table NewMessage,NewMessageCount| diff pos1=... pos2=... | search NewMessageCount>36

https://docs.splunk.com/Documentation/Splunk/7.1.0/SearchReference/Diff

Re: How can I show the difference in a string field from one day to another?

bscavotto — Wed, 09 May 2018 18:24:33 GMT

Thanks, I've reviewed that document. I'm not clear on what would go into the pos1 and pos2 fields.

Re: How can I show the difference in a string field from one day to another?

bscavotto — Wed, 09 May 2018 18:35:14 GMT

Plus, I'm not diffing data in the same output. This is output from each day separately.

Re: How can I show the difference in a string field from one day to another?

somesoni2 — Wed, 09 May 2018 19:24:22 GMT

Try like this

sourcetype=WinEventLog:Application SourceName=MessageAudit earliest=-1d@d latest=now
| fields _time Message |eval Message=split(Message,"\\\\") | mvexpand Message
| stats values(_time) as time by Message 
| rename COMMENT as "Below line will filter results to show new paths that were received today"
| whhere mvcount(time)=1 AND time>=relative_time(now(),"@d")

Re: How can I show the difference in a string field from one day to another?

bscavotto — Wed, 09 May 2018 20:10:17 GMT

Thanks, I appreciate the help.