Splunk Search

Creating a lookup file to provide data to a dashboard search with time stamps

arrcee
New Member

I have an application that generates a value that I pull the highest value for each day.
Right now the entire app log is in my indexer, but I only need to store the highest app value and the date stamp for that date. I'd like to do this to save space on my indexer since I have limited resources to use.
I initially created a csv based lookup file with a search that pulled the _time and app values I need, but I then realized I cannot properly address the time field in a search that uses the lookup file.
Here is an example of how that data looks:
_time app_value
2019-02-26 783
2019-02-27 16091
2019-02-28 11870
2019-03-01 1575
2019-03-02 433
2019-03-03 224
2019-03-04 1893
2019-03-05 10223
2019-03-06 11116
2019-03-07 12822
2019-03-08 1835
2019-03-09 44

I need to either configure this lookup file or possibly a KV store in order for me to be able to pull the app values in a search by a given hour period, day, week, month, year to date, current year, previous year, etc. I have access to over two years worth of data in my indexer to build the initial lookup file. I plan on appending the lookup file each data with the new value.

It would be a bonus if I could use the time picker to choose my date ranges. I have tried a few searches with date references in the search but those did not work. I no longer have those failed test searches.

I have IMO spent too much time trying to get this to work on my own, so I am reaching out to the community for help.
Thank You!!

0 Karma

woodcock
Esteemed Legend

You are mistaken; you can do a time-based lookup and that is exactly what you should do:
https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Defineatime-basedlookupinSplunkWeb

0 Karma

arrcee
New Member

I will look into this option as well. Thank You!

0 Karma

woodcock
Esteemed Legend

Be sure to come back here and let us know what you ended up doing and close out the question.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi arrcee,
have you ever thought of using a Summary Index instead of a lookup?
You could schedule a daily search (for example at 1.00) a daily search obtaining a result every day with all the information you need and then save this result in a Summary Index, consuming a really negligible amount of disk space and having the info that you they serve.
You can do this using the collect command (see https://docs.splunk.com/Documentation/Splunk/7.3.0/SearchReference/Collect).
Then you can have these information searching on this Summary index.

Bye.
Giuseppe

0 Karma

arrcee
New Member

I have not. Thank you for the suggestion. I'll look into that.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...