Getting Data In

unarchive_cmd doesn't work for CSV file

dmillis
Splunk Employee
Splunk Employee

In 6.2.0, I have written a pre-processor script for a particular CSV log format, to produce a useable timestamp. The script is designed to be used as a 'unarchive_cmd' script for the input. I.e., from the CLI, it functions like this:
cat | convert.pl

The problem: no matter how much I play with inputs.conf and props.conf, the unarchive_cmd is ignored. Here is inputs.conf:
[monitor:///var/cool_csv_logs/file.csv]
disabled = 0
followTail = 0
host = some_host
index = scratch
sourcetype = cool_csv

and props.conf:
`[cool_csv]
SHOULD_LINEMERGE = false
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl`

Why is the unarchive_cmd not working?

Tags (2)
0 Karma
1 Solution

dmillis
Splunk Employee
Splunk Employee

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

View solution in original post

dmillis
Splunk Employee
Splunk Employee

Here's a more readable version:

Here is inputs.conf:

[monitor:///var/cool_csv_logs/file.csv] 
disabled = 0 
followTail = 0 
host = some_host 
index = scratch 
sourcetype = cool_csv

and props.conf:

[cool_csv] 
SHOULD_LINEMERGE = false 
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl

... should be changed to file.log

0 Karma

dmillis
Splunk Employee
Splunk Employee

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

Get Updates on the Splunk Community!

New Splunk Observability innovations: Deeper visibility and smarter alerting to ...

You asked, we delivered. Splunk Observability Cloud has several new innovations giving you deeper visibility ...

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...