Knowledge Management

Collecting to a summary index not working

jizzmaster
Path Finder

am unable to collect data into a summary index. Getting odd behavior.

This works:

index=security sourcetype=dbx2 source=ca_owned_resource inactive=0 status="In Service" |stats count(resource_name) as asset_mgmt.cmdb.active |collect index=summary

This does not:

index=security sourcetype=nmap_xml state=up earliest=-5d |stats first(state) as State by ip |stats count(State) as asset_mgmt.nmap_xml.ip.live |collect index=summary

Both searches, per se, work fine. But the collection part does not work on the latter query. Each search query provides a number value as expected. When I look at the summary index (index=summary asset_mgmt) right after running each query, only the first search comes up. It does not matter the user I run it as; typical user, power user, or admin. I have also tried this on different searchheads but have the same result. Overall, I have three search strings that will not record into the summary index, and one that will.

0 Karma
1 Solution

jizzmaster
Path Finder

Turns out that it was indexing, just with a completely wrong date and timestamp. I could not find much rhyme or reason to when it applied the timestamp, but it was not the first or the last timestamp of the data it was pulling from.

Anyway, adding "|eval _time=now()" worked perfectly. Timestamps and summary indexing now being applied as I expect it to be. Timestamps are from when the saved search begins, not dependent on timestamps within the logs it is searching on.

View solution in original post

jizzmaster
Path Finder

Turns out that it was indexing, just with a completely wrong date and timestamp. I could not find much rhyme or reason to when it applied the timestamp, but it was not the first or the last timestamp of the data it was pulling from.

Anyway, adding "|eval _time=now()" worked perfectly. Timestamps and summary indexing now being applied as I expect it to be. Timestamps are from when the saved search begins, not dependent on timestamps within the logs it is searching on.

ConnorG
Path Finder

There's a few paragraphs in the documentation that may shed some light:

If you apply the collect command to events that do not have timestamps, it designates a time for all of the events using the earliest (or minimum) time of the search range. For example, if you use collect over the past four hours (range: -4h to +0h), it assigns a timestamp four hours previous to the time the search was launched to all of the events without a timestamp.

If you use collect with an all-time search and the events do not have timestamps, Splunk Enterprise uses the current system time for the timestamps.

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...

Announcing the 1st Round Champion’s Tribute Winners of the Great Resilience Quest

We are happy to announce the 20 lucky questers who are selected to be the first round of Champion's Tribute ...

We’ve Got Education Validation!

Are you feeling it? All the career-boosting benefits of up-skilling with Splunk? It’s not just a feeling, it's ...