- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are getting multiple errors like this
Corrupt csv header in CSV file , 2 columns with the same name
However we have so many CSV files that finding them will be all but impossible.
Can someone provide advice on how to find them?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Assuming that your OS is unix/linux, assuming that your CSV files use standard filenaming conventions (i.e. *.csv), assuming that your CSV files are standard with a header on the first line, assuming that the source files still exist, you can use the following CLI commands to identify problematic files:
find ${SPLUNK_HOME}/etc/apps/*/lookups -name *.csv -exec head -1 {} \; | tr ',' '\n' | sort| uniq -d
This will tell you the duplicated field, e.g. "foo". Then take that and do this to find the file (or a small pile to peek through):
for FILE in $(find ${SPLUNK_HOME}/lookups -name *.csv -exec grep -il foo {} \;); do echo ${FILE}; head -1 ${FILE} | tr ',' '\n' | sort | uniq -d; done
Here are some other tips:
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Assuming that your OS is unix/linux, assuming that your CSV files use standard filenaming conventions (i.e. *.csv), assuming that your CSV files are standard with a header on the first line, assuming that the source files still exist, you can use the following CLI commands to identify problematic files:
find ${SPLUNK_HOME}/etc/apps/*/lookups -name *.csv -exec head -1 {} \; | tr ',' '\n' | sort| uniq -d
This will tell you the duplicated field, e.g. "foo". Then take that and do this to find the file (or a small pile to peek through):
for FILE in $(find ${SPLUNK_HOME}/lookups -name *.csv -exec grep -il foo {} \;); do echo ${FILE}; head -1 ${FILE} | tr ',' '\n' | sort | uniq -d; done
Here are some other tips:
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So the first one command, every word it brings back is a duplicated one?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Exactly.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well see we are trying to find specific keywords, so I know like one I'm trying to test. When I run your second command, it pulls in a ton of CSV files. Checking one, and the word isn't in the CSV header at all?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Oh I see it now, the word is in the CSV file itself. But I'm only concerned with the headers, is that not what the alert means?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Yes. I updated my answer to help better.
