Search Time Issue

[wilcomply13]

I'm having a bit of issue with my current logic. Ideally my lookup would contain three months of data, however when this search is executed I am only receiving the previous 15 mins of data. I presumed that the 'earliest' specification would only apply to the base search and not put requirements on the inputlookup, I was incorrect.

In an ideal setting the base search runs every 15 minutes and pulls in recent events, lookup is appended to the results, stats recalculates latest events for public_ips, anything older than 3months is discarded, and the lookup is updated.

Can anyone advise on my time settings so that the lookup appended results are not restricted to the 15min time frame in the base search?

index=firewall earliest=-15m 
| fields user src_host private_ip public_ip 
| inputlookup user_tracking.csv append=true 
| stats latest(_time) as latestTime by user src_host public_ip private_ip 
| where latestTime>relative_time(now(),"-3mon") 
| outputlookup user_tracking.csv

[richgalloway]

How did you conclude the inputlookup command is restricted to 15 minutes?  That command has no time constraint.  Lookup files are not time-sequenced and often have no time field at all.  Their purpose is to enrich data, not restrict it, so it doesn't make sense to limit their output as you suggest they do.

Please share the fields in the lookup and in the data so we can help troubleshoot.

[wilcomply13 · Accepted solution]

I looked at this so long I didn't think about the field name for time in the lookup. 

I coalesced the base search and the lookup time fields and was able to resolve the issue

| eval time = coalesce(_time, latestTime)
| stats latest(time) as latestTime by......