The Lookup cache has been generated with 90 days baseline before Search 2 in which "dest" field is not "null" for any user and the "dest" field is expected not to be "null" at anytime. then later, it is saved as report and scheduled to run every hour (Query 2).
Search 1:
index=x
| stats count earliest(_time) as firstTime latest(_time) as lastTime last(dest) as new_dest by user
| outputlookup test.csv
Search 2:
index=x
| stats count earliest(_time) as firstTime latest(_time) as lastTime last(dest) as new_dest by user
| inputlookup append=t test.csv
| eval destination = coalesce(new_dest,dest)
| stats min(firstTime) as firstTime, max(lastTime) as lastTime values(destination) as Destination by user
| outputlookup test
So the issue is, when it runs for last hour, the "dest" field in the lookup gets updated for users who were seen in last hour only and "dest" becomes null for rest of the users after appending. so the need is to retain the old "dest" values for users who have not seen in the last hour, but I don't see any issue with "_time" fields.
