Getting Data In

Columns missing while using the Add Data to upload a csv file

jerrin
Explorer

I have uploaded a csv file through the Splunk Web which has 157 columns and 4000 events.
I noticed that while directing to the index only 105 columns are visible with 4000 events.
Is there any limitation in splunk? And this is not the first time too I noticed this. Kindly help.

Tags (2)
0 Karma

pgoldweic
Communicator

I seem to be experiencing the same issue as reported here with a very small/simple csv file that suggests to me that only csv columns which have *at least* one non-empty value in them are being imported.  Could somebody please confirm whether this is expected behavior (hopefully not) or some sort of bug?. 

0 Karma

woodcock
Esteemed Legend

Almost always CSV files should be uploaded as lookup files, not event data. I am not sure what happened when you indexed this file, but try using it as lookup file instead and they try |inputlookup YourLookupFileName.csv and see if it works better (it probably will).

0 Karma

jerrin
Explorer

Thanks @woodcock. But this do not give a clarity why the columns are truncated.
I ingest csv file from the AWS S3 into splunk always.
These feeds do not qualify to be a Lookup.
Please help to understand why the truncation of columns.

0 Karma

to4kawa
Ultra Champion
|inputlookup your_AWS_S3.csv
|head 1
|transpose

and

 index=your_index
|head 1
|transpose

When compared, there may be characters that cannot be used in field names.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...