Splunk Search

Splunk bug with lookups matching on fields that include spaces

phoenixdigital
Builder

OK one of our devs discovered a weird bug where if a lookup is being performed on a CSV where the field to match contained spaces.

The results are consistently weird between Linux and Windows.

It doesn't matter if the CSV lookup is

  • Unix or Windows format (ie CRLF or just LF newlines)
  • Placing the lookup field in quotes makes no difference.

I have a sample app here which demonstrates the bug using internal Splunk data.

http://www.phoenixdigital.com/splunk/lookup_bug.spl

The screenshot shows the bug in action.
(see attached screenshot)

You can see in this example the group=pipeline data does not appear in the results on the left however it does on the data on the right when you change

group_is_it_searchable=no
to
group_is_it_searchable=*no

I will put the major components here for clarity though.

props.conf

[splunkd]

LOOKUP-1-first = group_extend group OUTPUT group_spaced, group_bug_prediction
LOOKUP-2-second = spacing_issue group_spaced OUTPUT group_is_it_searchable

transforms.conf

[group_extend]
filename = group_extend.csv

[spacing_issue]
filename = spacing_issue.csv

group_extend.csv (This lookup creates a field 'group_spaced' which will be used for the next lookup)

group,group_spaced,group_bug_prediction
pipeline,pipe line,this will break
queue,queue,this will work
per_host_thruput,per host thruput,this will break
tpool,tpool,this will work
deploy-connections,deploy connections,this will break
map,map,this will work
per_source_thruput,per source thruput,this will break

spacing_issue,csv (The second lookup which provides inconsistent results)

group_spaced,group_is_it_searchable
queue,yes
per host thruput,no
tpool,yes
pipe line,no
deploy connections,no
map,yes
per source thruput,no
Tags (5)
0 Karma

sylim_splunk
Splunk Employee
Splunk Employee

This turned out to be an issue of "nested lookups" which is not supported in any of splunk versions as of this writing.

You would see inconsistent results - The situation is it works under certain instance and not on another. Please avoid any chain lookups.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi phoenixdigital, this looks like a well thought out and demonstrated bug report. As ppablo mentioned, you should submit a report at http://www.splunk.com/r/bugs. I'll be interested to hear how it goes! 😄

0 Karma

phoenixdigital
Builder

Thanks meubel and ppablo submitted as a bug pending response................

0 Karma

ppablo
Community Manager
Community Manager

Hi @phoenixdigital

Did you already submit the bug here?
http://www.splunk.com/r/bugs

Posting on Answers won't really help to get it looked at. Also, if other users have previously submitted this bug as well, it'll help build traction on the issue.

0 Karma

phoenixdigital
Builder

I will post there just wanted to see if there was anything obviously wrong with my test before I did.

0 Karma

ppablo
Community Manager
Community Manager

ohh ok no worries. Thanks for being thoughtful 🙂

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!