I have a csv that gets loaded weekly... timestamp for events are on load. However, this file has multiple time fields (first discovered, last seen, etc.). I am attempting to find those events (based on the fields) that are greater than 30 days, for example. had this working fine, until I introduced a lookup. I am attempting to show results grouping them by owner (stats) but only those events that are 30 days from first discovered until now(). If I add | where Days > 30, results show every event from the fiel. But I know they are there... anonymized query below. What am I doing wrong?
Sample fields being eval'ed:
First Discovered: Jul 26, 2023 16:50:26 UTC
Last Observed: Jul 19, 2024 09:06:32 UTC
index=stuff source=file Severity="Critical" | lookup detail.csv "IP Address" OUTPUTNEW Manager
| eval First_DiscoveredTS = strptime("First Discovered", "%b %d, %Y %H:%M:%S %Z"), Last_ObservedTS = strptime("Last Observed", "%b %d, %Y %H:%M:%S %Z"), firstNowDiff = (now() - First_DiscoveredTS)/86400, Days = floor(firstNowDiff)
| stats by Manager | where Days > 30
This has nothing to do with now(). Your strptime receives a literal string "First Detected" and tries to calculate time. The result is null, of course.
Change double quote to single quote.
index=stuff source=file Severity="Critical"
| lookup detail.csv "IP Address" OUTPUTNEW Manager
| eval First_DiscoveredTS = strptime('First Discovered', "%b %d, %Y %H:%M:%S %Z"),
Last_ObservedTS = strptime('Last Observed', "%b %d, %Y %H:%M:%S %Z"),
firstNowDiff = (now() - First_DiscoveredTS)/86400, Days = floor(firstNowDiff)
| stats count by Manager Days
| where Days > 30
Play with this emulation and compare with real data
| makeresults format=csv data="First Discovered, Last Observed
\"Jul 26, 2023 16:50:26 UTC\", \"Jul 19, 2024 09:06:32 UTC\""
| appendcols
[makeresults format=csv data="Manager
foo"]
``` the above emulates
index=stuff source=file Severity="Critical"
| lookup detail.csv "IP Address" OUTPUTNEW Manager
```
Output from this is
| Manager | Days | count |
| foo | 362 | 1 |
Thank you. The quotes made all the difference, silly mistake.
This has nothing to do with now(). Your strptime receives a literal string "First Detected" and tries to calculate time. The result is null, of course.
Change double quote to single quote.
index=stuff source=file Severity="Critical"
| lookup detail.csv "IP Address" OUTPUTNEW Manager
| eval First_DiscoveredTS = strptime('First Discovered', "%b %d, %Y %H:%M:%S %Z"),
Last_ObservedTS = strptime('Last Observed', "%b %d, %Y %H:%M:%S %Z"),
firstNowDiff = (now() - First_DiscoveredTS)/86400, Days = floor(firstNowDiff)
| stats count by Manager Days
| where Days > 30
Play with this emulation and compare with real data
| makeresults format=csv data="First Discovered, Last Observed
\"Jul 26, 2023 16:50:26 UTC\", \"Jul 19, 2024 09:06:32 UTC\""
| appendcols
[makeresults format=csv data="Manager
foo"]
``` the above emulates
index=stuff source=file Severity="Critical"
| lookup detail.csv "IP Address" OUTPUTNEW Manager
```
Output from this is
| Manager | Days | count |
| foo | 362 | 1 |