topic Re: Trouble ingesting multi line VB script output in Deployment Architecture

Trouble ingesting multi line VB script output

JamesRConley — Tue, 29 Sep 2020 14:14:05 GMT

Currently we are using VB scripts to query the WMI namespace within windows for data collection I have written a simple script to pull filesystem information in csv format. The issue is that Splunk will only ingest the header and the first row of data and nothing further.

Example data:
filesystem,label,type,format,capacitymb,freespace,pctused
C:,OSDisk,Local Fixed Disk,NTFS,475914,384259,19.26
H:,Users,Network Connection,NTFS,2097022,553788,73.59
P:,public,Network Connection,NTFS,5242750,2229850,57.47

inputs.conf
[script://.\bin\filesystem.bat]
disabled = 0
interval = 60
source = filesystem
sourcetype = windows:filesystem
index = custom_temp

props.conf
[windows:filesystem]
INDEXED_EXTRACTIONS=CSV
SHOULD_LINEMERGE = false
FIELD_HEADER_REGEX=(^filesystem,.*)

Re: Trouble ingesting multi line VB script output

guilmxm — Tue, 29 Sep 2020 14:10:58 GMT

Hi,

Have you been though the data preview to configure and preview the ingestion ? (Settings / Add data)

You data is just regular CSV:

[ csv ]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Structured
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true
HEADER_FIELD_LINE_NUMBER=1

I added the "HEADER_FIELD_LINE_NUMBER", this expect the header to be the first line.
If you look at the data preview, you will see other possibilities like using a regular expression.

Note you cannot have more than 1 header in the CSV event, Splunk won't manage multi csv files at once. (does not seem to your case)

Guilhem

Re: Trouble ingesting multi line VB script output

JamesRConley — Tue, 29 Sep 2020 14:14:08 GMT

I saved off my script output to a csv file and then imported it into Splunk using the add data function. It of course added everything perfectly as it should. I copied the props.conf sample which is shown below. I then jammed this into the props.conf into the project on my forwarder and restarted. I am still not getting any additional lines in Splunk, only the first line after the header of my command.

[windows:filesystem]
CHARSET=UTF-8
FIELD_HEADER_REGEX=(^filesystem,.*)
INDEXED_EXTRACTIONS=csv
KV_MODE=none
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=false
category=Structured
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true
FIELD_QUOTE="
FIELD_DELIMITER=,

I feel like this is an issue with the way Splunk interprets the output of a line break from a .bat file.

Re: Trouble ingesting multi line VB script output

JamesRConley — Tue, 23 May 2017 18:11:31 GMT

Ok I was just being stupid here. I forgot the scripts on the forwarder run as "administrator" and the output of my script only had a single line. Everything is working as intended I was just testing my script as myself instead of as the administrator.

Thank you