Splunk Search

proper format for lookup table files

jalfrey
Communicator

I'm working on defining a new lookup table. I found the tutorial and example files. http://docs.splunk.com/Documentation/Splunk/5.0/Knowledge/Usefieldlookupstoaddinformationtoyourevent...
When using the web interface to upload the file I always get the following error:
"Encountered the following error while trying to save: In handler 'lookup-table-files': File is binary and not gzipped"

Things I have tried so far:
- Checking the line endings. I tried CR, LF, and CR\LF
- Looking for special characters. There are no accents or odd characters. Also no odd end of file stuff.
- The file type is UTF-8

Searching the spunk answers I see others asking the question but no clear fix.

Tags (2)

aakwah
Builder

This issue happened with me when I've a column that has German letters ü,ß,ä ...
after I removed this column from csv file it uploaded successfully

0 Karma

michael_peters
Path Finder

In 6.0.1 to 6.0.3 the upload feature does work, and you do have to have unix like line "\n" endings.

To make the file acceptable via komodo open the original csv then open a new file and copy and paste the text, just resaving will not work.

For those interested the feature is enabled in Komodo by default and is found at:
Preferences>New Files>Specify the end-of-line (EOL) inidicator for newly created files:

I will upload a screenshot once I have more karma.

jalfrey
Communicator

ok so it turns out that the formatting was not a problem. The feature of being able to upload the file via the GUI is currently broken. You must manually stage the $PLUNK_INSTALLs/etc/apps/search/lookups

lguinn2
Legend

From the docs (mid-page)

"The CSV files used as lookups must be created with UNIX-style line endings ("\n"). Splunk will not correctly read lookup files saved using Macintosh ("\r") or Windows line endings ("\r\n")."

You might find the dos2unix utility helpful. Some editors (like Komodo Edit) will let you choose the line-ending style in the preferences or options.

kknopp
Path Finder

I used TextWrangler on my mac to get things in the right Unix format. That resolved my issues.

0 Karma

lguinn2
Legend

Also, can you copy the file onto the Splunk server (with ftp or something) - and then see if you can edit it there with a local text editor? Does it look okay then?

0 Karma

lguinn2
Legend

Wow - what's in this file... and where are you putting it? Can you open it with Text Edit?

0 Karma

jalfrey
Communicator

I'm tying to upload the file as a .csv. I see that it supports .zip. I tried zipping it but that did not fix it either.

0 Karma

jalfrey
Communicator

Xcode has the option to set the line endings. I also used dos2unix. Neither worked for me.

0 Karma

vincesesto
Communicator

Hey jalfrey, I had a lot of issues getting lookup tables to work when I first started using them. Is the file simply a csv file and can you post a couple of lines as an example?

0 Karma