Getting Data In

unarchive_cmd doesn't work for CSV file

dmillis
Splunk Employee
Splunk Employee

In 6.2.0, I have written a pre-processor script for a particular CSV log format, to produce a useable timestamp. The script is designed to be used as a 'unarchive_cmd' script for the input. I.e., from the CLI, it functions like this:
cat | convert.pl

The problem: no matter how much I play with inputs.conf and props.conf, the unarchive_cmd is ignored. Here is inputs.conf:
[monitor:///var/cool_csv_logs/file.csv]
disabled = 0
followTail = 0
host = some_host
index = scratch
sourcetype = cool_csv

and props.conf:
`[cool_csv]
SHOULD_LINEMERGE = false
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl`

Why is the unarchive_cmd not working?

Tags (2)
0 Karma
1 Solution

dmillis
Splunk Employee
Splunk Employee

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

View solution in original post

dmillis
Splunk Employee
Splunk Employee

Here's a more readable version:

Here is inputs.conf:

[monitor:///var/cool_csv_logs/file.csv] 
disabled = 0 
followTail = 0 
host = some_host 
index = scratch 
sourcetype = cool_csv

and props.conf:

[cool_csv] 
SHOULD_LINEMERGE = false 
pulldown_type = true
INDEXED_EXTRACTIONS = csv
KV_MODE = node
category = Structured
invalid_cause = archive

[source::/var/cool_csv_logs/file.csv]
unarchive_cmd = /opt/splunk/bin/convert.pl

... should be changed to file.log

0 Karma

dmillis
Splunk Employee
Splunk Employee

Well, David, it turns out that you cannot use a file suffix of .csv (or .txt) with unarchive_cmd.

Try changing your source filename to file.log

(This works!)

Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...