Getting Data In

Columns missing while using the Add Data to upload a csv file

jerrin
Explorer

I have uploaded a csv file through the Splunk Web which has 157 columns and 4000 events.
I noticed that while directing to the index only 105 columns are visible with 4000 events.
Is there any limitation in splunk? And this is not the first time too I noticed this. Kindly help.

Tags (2)
0 Karma

pgoldweic
Communicator

I seem to be experiencing the same issue as reported here with a very small/simple csv file that suggests to me that only csv columns which have *at least* one non-empty value in them are being imported.  Could somebody please confirm whether this is expected behavior (hopefully not) or some sort of bug?. 

0 Karma

woodcock
Esteemed Legend

Almost always CSV files should be uploaded as lookup files, not event data. I am not sure what happened when you indexed this file, but try using it as lookup file instead and they try |inputlookup YourLookupFileName.csv and see if it works better (it probably will).

0 Karma

jerrin
Explorer

Thanks @woodcock. But this do not give a clarity why the columns are truncated.
I ingest csv file from the AWS S3 into splunk always.
These feeds do not qualify to be a Lookup.
Please help to understand why the truncation of columns.

0 Karma

to4kawa
Ultra Champion
|inputlookup your_AWS_S3.csv
|head 1
|transpose

and

 index=your_index
|head 1
|transpose

When compared, there may be characters that cannot be used in field names.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...