I have DNS logs from both Windows and Unix BIND. What I am trying to do is create a quick way for admins to query 90 days worth of these DNS logs to see if a particular domain name has ever been looked up. My thought was to do search of each index, do a "stats count by domain" and put the results potentially in a summary index. It was suggested that I look at Report Acceleration as well. I'm running into a couple issues.
I cant save my query as accelerated. I do not see any checkbox in version 6.0. Here is the search: index=dnslog | stats count by domain_name
It would be useful for the analysts to have something like a "first seen" date to go with the domains returned in the searches but I cannot figure out a way to do that.
Any help would be greatly appreciated.
I would highly recommend using report acceleration, it is self repairing. Summary Indexing is entirely reliant on the scheduled search performing as expected.
Generally I only use summary indexing for summarized information that needs to be stored longer than a year.
Report Acceleration is still available in 6.0, and your search does qualify (uses a transforming command, stats). If you go to Settings > Searches & Reports > Your Search, do you not see the check box?
To calculate the first seen date you can use earliest() in your stats command. It would look like this...
| stats count earliest(eval(strftime(time, "%H:%M:%S"))) AS "first seen" by domainname
You can modify the time format to display month, date, year and all that other goodness.
Replaces this question:
Is there *possibly* a match for my data in this bucket?
With this question:
Is there definitely a match for my data in this bucket?
Astronomical increases in performance are possible (but, if you do it wrong, things can actually be slower).
Admin must manually preform backfill.
Late-arriving events are of GREAT concern
Drilldown to raw data no longer works (must do Advanced dashboard programming to force it to switch back to raw data).
Is somewhat fragile and requires reactive housekeeping (many things can cause SI data outages).
Is sensitive to TimeZone and the user who runs the "Populating Search".
Has a minor "every-hour" impact to Indexers and consumes a small amount of disk space but no license.
Speeds up searches by replacing huge amounts of raw data with much smaller amounts of aggregate ("summarized") data in the Summary Index.
Requires an Admin to reverse-engineer the desired end-usage of the data in order to construct an appropriate "Populating Search" to do this conversion.
Requires that all existing KO uses of the raw data be manually converted (e.g. saved searches, macros, dashboard panels) to pull data from the SI instead of raw data.
+adding one more
Report acceleration saved in - $SPLUNKHOME/var/lib/splunk/index1/summary whereas
Summary indexing will be saved in summary index($SPLUNKHOME/var/lib/splunk/summarydb). Temporarily search results are stored in $SPLUNKHOME/var/spool/splunk/.stash