How to flag when new value is created in field

[manderson7]

I'm really overthinking this, but am lost.

I need to show when new correlation searches are introduced into the environment. I have a lookup with the current correlation searches, along w/ relevant data, and last time updated. How do I compare current rest call results with the lookup and show the most recently created searches?

Any help would be appreciated.

[somesoni2 · Accepted solution]

Something like this

| rest <your rest call> | eval from="rest"
| append [| inputlookup yourlookup | eval from="lookup"]
| stats values(from) as from ..add other fields you need.. by yourPrimaryKeyField
| where mvcount(from)=1 AND from="rest"

[manderson7]

Yep, overthinking. Thank you.

In addition, how would I go about creating a "add on date" field to the lookup once the new search is added to it.

Basically, I'd like to get the dates added for new correlation searches from this time forward in said lookup.

[somesoni2]

Since the rest call might now give the timestamp, you can add a field with current time on the rest query, then append lookup data and take the latest records. Something like this

| rest <your rest call> | eval add_on_date=now()
| append [| inputlookup yourlookup | eval add_on_date=coalesce(add_on_date,0)]
| stats max(add_on_date) as add_on_date ...other fields that you need in lookup... by yourPrimaryKeyField(s) | outputlookup yourlooup

[manderson7]

Thanks for the help so far, this is what I have

|relevant rest call
| eval from="rest"
| eval added_on_date=now()
| fields title, search, updated SplunkApp dataSource actions search added_on_date
| rename title AS Title 
| append 
    [| inputlookup append=true correlation_searches_history.csv 
    | eval from="lookup"
    | eval added_on_date=coalesce(added_on_date,0)
    ] 
| stats max(added_on_date) AS added_on_date values(from) as from values(updated)  by Title

and the search keeps updating the added_on_date in the lookup (when I run outputlookup) to when the search runs. How do I avoid that?