Is there some reason why using the lookup
command doesn't seem to be working properly after stats
?
The search I'm trying to use, is like this:
sourcetype=access_combined | stats count by my_field | lookup my_lookup my_field OUTPUT out1
However out1
is never populated. If I move the lookup
before the stats
command (which now means I have to group by this value, and protect against null
values), then the search works. Here is an example:
sourcetype=access_combined | lookup my_lookup my_field OUTPUT out1 | eval out1=coalesce(out1, "default") | stats count by my_field, out1
Is there some fundamental concept I'm missing here, or is this a bug?
Thanks to jrodman for pointing me in the right direction on this one.
My problem is because my lookup is date effective, that is to say, I've defined time_field
in my transforms.conf
entry, like so:
[my_lookup] filename = my_lookup_file.csv time_field = my_time_field time_format = %s
So because of my stats
command dropped the _time
field, splunk was unable to available to complete the lookup. Therefore, since I don't need to be strict about date-effective lookups in my search, I can use the following to get the results I'm looking for:
sourcetype=access_combined | stats count, min(_time) as _time by my_field | lookup my_lookup my_field OUTPUT out1
There's no fixed requirement for when lookup should be invoked. There are two possibilities here.
Perhaps you should provide a bit more about how the lookup is defined?
There's no fixed requirement for when lookup should be invoked. There are two possibilities here.
Perhaps you should provide a bit more about how the lookup is defined?
I'm a bit murky on exactly when we should complain about stuff and when we shouldn't. With all the things that can get defined by various parties, we can't complain about everything by default. However, in this case it sure seems like you're explicitly running the lookup command, and it should probably provide feedback. I'll file some sort of defect tomorrow, hopefully. It wouldn't hurt to report it to support as a "please pass this along".
Thanks. I figured out my issue. I was doing a date-effective lookup and I stats
was stripping out _time
. I've updated the post. I am somewhat surprised that lookup
didn't complain about missing fields like I've seen it do in other cases, but I suppose there is no single "right way" to handle the situation of a missing _time
field, and it certainly makes sense why lookup
failed to give me what I was looking for.