Getting Data In

Columns missing while using the Add Data to upload a csv file

jerrin
Explorer

I have uploaded a csv file through the Splunk Web which has 157 columns and 4000 events.
I noticed that while directing to the index only 105 columns are visible with 4000 events.
Is there any limitation in splunk? And this is not the first time too I noticed this. Kindly help.

Tags (2)
0 Karma

pgoldweic
Communicator

I seem to be experiencing the same issue as reported here with a very small/simple csv file that suggests to me that only csv columns which have *at least* one non-empty value in them are being imported.  Could somebody please confirm whether this is expected behavior (hopefully not) or some sort of bug?. 

0 Karma

woodcock
Esteemed Legend

Almost always CSV files should be uploaded as lookup files, not event data. I am not sure what happened when you indexed this file, but try using it as lookup file instead and they try |inputlookup YourLookupFileName.csv and see if it works better (it probably will).

0 Karma

jerrin
Explorer

Thanks @woodcock. But this do not give a clarity why the columns are truncated.
I ingest csv file from the AWS S3 into splunk always.
These feeds do not qualify to be a Lookup.
Please help to understand why the truncation of columns.

0 Karma

to4kawa
Ultra Champion
|inputlookup your_AWS_S3.csv
|head 1
|transpose

and

 index=your_index
|head 1
|transpose

When compared, there may be characters that cannot be used in field names.

0 Karma
Get Updates on the Splunk Community!

.conf23 Registration is Now Open!

Time to toss the .conf-etti 🎉 —  .conf23 registration is open!   Join us in Las Vegas July 17-20 for ...

Don't wait! Accept the Mission Possible: Splunk Adoption Challenge Now and Win ...

Attention everyone! We have exciting news to share! We are recruiting new members for the Mission Possible: ...

Unify Your SecOps with Splunk Mission Control

In today’s post, I'm excited to share some recent Splunk Mission Control innovations. With Splunk Mission ...