Compare two sets of data from different times

[sranga]

Hi

I am at a loss on how to approach this problem. Lets say we have the following data:

Input 1: Contains list of non-numeric Ids of data that should be picked up by a processing module 30 days from now
Input 2: Contains list of non-numeric Ids of data that the processing module picked up today

1) Is there anyway to show the diff between the list of today (Input 2) compared with what we "expected" this list to be 30 days back (Input 1)?
2) If the inputs contain a "count" instead of the actual list of data, would it be an easier comparison? I would still be interested in getting the Ids that didn't get picked up today though

Would I be able to use the "diff" command for this? Thanks for your help.

[Stephen_Sorkin · Accepted solution]

diff isn't the right way to approach it since it compares two individual events. set diff can be used for this but is often hard to use. Let's assume that you have two searches that identify the events, and we'll call them <search_1> and <search_2>.

Your base search would then be:

<search_1>
| eval input_type="baseline"
| append [search <search_2> | eval input_type="today"]
| chart count by id input_type

Now to answer your question, you'd add to this search: | search baseline>0 today=0.

The problem is marginally easier if you already have a field to distinguish the two types of events, as you'd just search (<search_1>) OR (<search_2>) | chart count by id distinguishing_field.

[Stephen_Sorkin]

Yes, you are correct. I've edited my answer.

[sranga]

Thanks. In the search command you specified in your answer | append [search <search_1>. Should that be search_2 instead of search_1? Input-1 & Input-2 would be two separate searches.

[Stephen_Sorkin]

By search_1, search_2, I mean input 1, input 2 from your statement. If they're the same input, let me know and I'll add to my answer to handle that case.

[sranga]

Thanks. Would that be search_2 in the "append" command? If I summary index the searches, would that be able to provide the distinguishing_field?