Knowledge Management

kvstore, inputlookup and time-bounds

lfrit
New Member

I'm trying to set up a kvstore lookup where the results from inputlookup can be filtered using the regular time-pickers available on the web GUI or with the latest= and earliest= modifiers.

$ collections.conf
[testkv]
enforceTypes = true
field.action = string
field.ts = time

$ transforms.conf
[testkv]
external_type = kvstore
fields_list =  action, ts
time_field = ts
;time_format = %s.%3N
;time_format = %s.%Q

The ts field contains a UNIX epoch with milliseconds so 10+3 digits.

Regardless what I select "Last 15 minutes", "Last 4 hours" I always get the whole kvstore content.

First of all, is that doable in general and, if yes, any ideas on what's wrong? 🙂

0 Karma
1 Solution

woodcock
Esteemed Legend

Sure, but not in a normal way. Do it like this:

| makeresults
| addinfo
| map
    [| inputlookup testkv
    | search ts>=$info_min_time$ AND ts<=$info_max_time$]

View solution in original post

0 Karma

woodcock
Esteemed Legend

Sure, but not in a normal way. Do it like this:

| makeresults
| addinfo
| map
    [| inputlookup testkv
    | search ts>=$info_min_time$ AND ts<=$info_max_time$]
0 Karma

frechette
Explorer

Why should you have to explicitly filter by time with a "search" or "where" command for a kvstore lookup when you don't have to with a regular search from an index?! This is a terrible approach. If it's the only approach to filtering a kvstore lookup by time then shame on Splunk.

0 Karma

dnitschke_splun
Splunk Employee
Splunk Employee

You can also add the time filter into the WHERE clause of inputlookup, e.g.

| inputlookup testkv WHERE
[| makeresults count=1
| addinfo
| eval info_max_time=if(info_max_time=="+Infinity", 2147483647, info_max_time)
| eval search="( (ts>=" . info_min_time . ") AND (" . "ts<" . info_max_time . ") )"
| table search ]

0 Karma

lfrit
New Member

Many thanks! That's a really interesting approach 🙂

I've just added a small workaround to handle the "All time" case and it seems to work as expected, I can simply create a dedicated macro now to make it more handy.

 | makeresults
 | addinfo
 | eval info_max_time=if(info_max_time=="+Infinity", 9999999999999, info_max_time)
 | map
     [| inputlookup testkv
     | search ts>=$info_min_time$ AND ts<=$info_max_time$]

Do you know any sort trick to cast that "+Infinity" so I can directly compare it with my ts field?

0 Karma

woodcock
Esteemed Legend

I should have caught that. I would do it exactly as you have done

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...