Getting Data In

Columns missing while using the Add Data to upload a csv file

jerrin
Explorer

I have uploaded a csv file through the Splunk Web which has 157 columns and 4000 events.
I noticed that while directing to the index only 105 columns are visible with 4000 events.
Is there any limitation in splunk? And this is not the first time too I noticed this. Kindly help.

Tags (2)
0 Karma

pgoldweic
Communicator

I seem to be experiencing the same issue as reported here with a very small/simple csv file that suggests to me that only csv columns which have *at least* one non-empty value in them are being imported.  Could somebody please confirm whether this is expected behavior (hopefully not) or some sort of bug?. 

0 Karma

woodcock
Esteemed Legend

Almost always CSV files should be uploaded as lookup files, not event data. I am not sure what happened when you indexed this file, but try using it as lookup file instead and they try |inputlookup YourLookupFileName.csv and see if it works better (it probably will).

0 Karma

jerrin
Explorer

Thanks @woodcock. But this do not give a clarity why the columns are truncated.
I ingest csv file from the AWS S3 into splunk always.
These feeds do not qualify to be a Lookup.
Please help to understand why the truncation of columns.

0 Karma

to4kawa
Ultra Champion
|inputlookup your_AWS_S3.csv
|head 1
|transpose

and

 index=your_index
|head 1
|transpose

When compared, there may be characters that cannot be used in field names.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...