How can I show the difference in a string field from one day to another?

[bscavotto]

I have a powershell script that audits some files and creates an Windows application event log with the filepaths of any matches in the Message field. It runs once per day and creates a single event log everyday. I only want the diff from one day to the next as new things are found.

My query so far only gives me the split listing of the Message field contents and the count but how do I get it to only show me the new entries as compared to the previous day?

sourcetype=WinEventLog:Application SourceName=MessageAudit |eval NewMessage=split(Message,"\\\\") |eval NewMessageCount=mvcount(NewMessage) |table NewMessage,NewMessageCount | search NewMessageCount>36
(There are 36 consistent false positives)

Thanks.

[somesoni2 · Accepted solution]

Try like this

sourcetype=WinEventLog:Application SourceName=MessageAudit earliest=-1d@d latest=now
| fields _time Message |eval Message=split(Message,"\\\\") | mvexpand Message
| stats values(_time) as time by Message 
| rename COMMENT as "Below line will filter results to show new paths that were received today"
| whhere mvcount(time)=1 AND time>=relative_time(now(),"@d")

[bscavotto]

Thanks, I appreciate the help.

[skoelpin]

Perhaps the diff command would work here

sourcetype=WinEventLog:Application SourceName=MessageAudit |eval NewMessage=split(Message,"\\\\") |eval NewMessageCount=mvcount(NewMessage) |table NewMessage,NewMessageCount| diff pos1=... pos2=... | search NewMessageCount>36

https://docs.splunk.com/Documentation/Splunk/7.1.0/SearchReference/Diff

[bscavotto]

Thanks, I've reviewed that document. I'm not clear on what would go into the pos1 and pos2 fields.

[bscavotto]

Plus, I'm not diffing data in the same output. This is output from each day separately.