topic proper format for lookup table files in Splunk Search

proper format for lookup table files

jalfrey — Tue, 12 Mar 2013 21:37:56 GMT

I'm working on defining a new lookup table. I found the tutorial and example files. http://docs.splunk.com/Documentation/Splunk/5.0/Knowledge/Usefieldlookupstoaddinformationtoyourevents
When using the web interface to upload the file I always get the following error:
"Encountered the following error while trying to save: In handler 'lookup-table-files': File is binary and not gzipped"

Things I have tried so far:
- Checking the line endings. I tried CR, LF, and CR\LF
- Looking for special characters. There are no accents or odd characters. Also no odd end of file stuff.
- The file type is UTF-8

Searching the spunk answers I see others asking the question but no clear fix.

Re: proper format for lookup table files

vincesesto — Tue, 12 Mar 2013 22:08:43 GMT

Hey jalfrey, I had a lot of issues getting lookup tables to work when I first started using them. Is the file simply a csv file and can you post a couple of lines as an example?

Re: proper format for lookup table files

lguinn2 — Tue, 12 Mar 2013 22:33:04 GMT

From the docs (mid-page)

"The CSV files used as lookups must be created with UNIX-style line endings ("\n"). Splunk will not correctly read lookup files saved using Macintosh ("\r") or Windows line endings ("\r\n")."

You might find the dos2unix utility helpful. Some editors (like Komodo Edit) will let you choose the line-ending style in the preferences or options.

Re: proper format for lookup table files

jalfrey — Tue, 12 Mar 2013 22:41:42 GMT

Xcode has the option to set the line endings. I also used dos2unix. Neither worked for me.

Re: proper format for lookup table files

jalfrey — Tue, 12 Mar 2013 22:45:50 GMT

I'm tying to upload the file as a .csv. I see that it supports .zip. I tried zipping it but that did not fix it either.

Re: proper format for lookup table files

lguinn2 — Tue, 12 Mar 2013 23:08:33 GMT

Wow - what's in this file... and where are you putting it? Can you open it with Text Edit?

Re: proper format for lookup table files

lguinn2 — Tue, 12 Mar 2013 23:24:16 GMT

Also, can you copy the file onto the Splunk server (with ftp or something) - and then see if you can edit it there with a local text editor? Does it look okay then?

Re: proper format for lookup table files

jalfrey — Mon, 17 Jun 2013 20:10:32 GMT

ok so it turns out that the formatting was not a problem. The feature of being able to upload the file via the GUI is currently broken. You must manually stage the $PLUNK_INSTALLs/etc/apps/search/lookups

Re: proper format for lookup table files

michael_peters — Tue, 29 Apr 2014 17:26:11 GMT

In 6.0.1 to 6.0.3 the upload feature does work, and you do have to have unix like line "\n" endings.

To make the file acceptable via komodo open the original csv then open a new file and copy and paste the text, just resaving will not work.

For those interested the feature is enabled in Komodo by default and is found at:
Preferences>New Files>Specify the end-of-line (EOL) inidicator for newly created files:

I will upload a screenshot once I have more karma.

Re: proper format for lookup table files

kknopp — Mon, 13 Oct 2014 18:39:38 GMT

I used TextWrangler on my mac to get things in the right Unix format. That resolved my issues.

Re: proper format for lookup table files

aakwah — Wed, 14 Jan 2015 16:43:39 GMT

This issue happened with me when I've a column that has German letters ü,ß,ä ...
after I removed this column from csv file it uploaded successfully