Getting Data In
Highlighted

unarchive_cmd doesn't work for CSV file

Path Finder

In 6.2.0, I have written a pre-processor script for a particular CSV log format, to produce a useable timestamp. The script is designed to be used as a 'unarchive_cmd' script for the input. I.e., from the CLI, it functions like this:
cat | convert.pl

The problem: no matter how much I play with inputs.conf and props.conf, the unarchive_cmd is ignored. Here is inputs.conf:
[monitor:///var/cool_csv_logs/file.csv]
disabled = 0
followTail = 0
host = some_host
index = scratch
sourcetype = cool_csv

and props.conf:
`[coolcsv]
SHOULD
LINEMERGE = false
pulldowntype = true
INDEXED
EXTRACTIONS = csv
KVMODE = node
category = Structured
invalid
cause = archive

[source::/var/coolcsvlogs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl`

Why is the unarchive_cmd not working?

Tags (2)
0 Karma
Highlighted

Re: unarchive_cmd doesn't work for CSV file

Path Finder

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

View solution in original post

Highlighted

Re: unarchive_cmd doesn't work for CSV file

Path Finder

Here's a more readable version:

Here is inputs.conf:

[monitor:///var/cool_csv_logs/file.csv] 
disabled = 0 
followTail = 0 
host = some_host 
index = scratch 
sourcetype = cool_csv

and props.conf:

[cool_csv] 
SHOULD_LINEMERGE = false 
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl

... should be changed to file.log

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.