Is there some reason why using the
lookup command doesn't seem to be working properly after
The search I'm trying to use, is like this:
sourcetype=access_combined | stats count by my_field | lookup my_lookup my_field OUTPUT out1
out1 is never populated. If I move the
lookup before the
stats command (which now means I have to group by this value, and protect against
null values), then the search works. Here is an example:
sourcetype=access_combined | lookup my_lookup my_field OUTPUT out1 | eval out1=coalesce(out1, "default") | stats count by my_field, out1
Is there some fundamental concept I'm missing here, or is this a bug?
Thanks to jrodman for pointing me in the right direction on this one.
My problem is because my lookup is date effective, that is to say, I've defined
time_field in my
transforms.conf entry, like so:
[my_lookup] filename = my_lookup_file.csv time_field = my_time_field time_format = %s
So because of my
stats command dropped the
_time field, splunk was unable to available to complete the lookup. Therefore, since I don't need to be strict about date-effective lookups in my search, I can use the following to get the results I'm looking for:
sourcetype=access_combined | stats count, min(_time) as _time by my_field | lookup my_lookup my_field OUTPUT out1
There's no fixed requirement for when lookup should be invoked. There are two possibilities here.
Perhaps you should provide a bit more about how the lookup is defined?
Thanks. I figured out my issue. I was doing a date-effective lookup and I
stats was stripping out
_time. I've updated the post. I am somewhat surprised that
lookup didn't complain about missing fields like I've seen it do in other cases, but I suppose there is no single "right way" to handle the situation of a missing
_time field, and it certainly makes sense why
lookup failed to give me what I was looking for.
I'm a bit murky on exactly when we should complain about stuff and when we shouldn't. With all the things that can get defined by various parties, we can't complain about everything by default. However, in this case it sure seems like you're explicitly running the lookup command, and it should probably provide feedback. I'll file some sort of defect tomorrow, hopefully. It wouldn't hurt to report it to support as a "please pass this along".