topic Re: Input a CSV in Getting Data In

Input a CSV

ofedorov — Mon, 28 Sep 2020 10:03:26 GMT

I'm trying to index a .CSV, created by tasklist.

CVS's headers and fields never get properly recognized and it gets indexed as a whole array:

"Image Name","PID","Session Name","Session#","Mem Usage","Status","User Name","CPU Time","Window Title"

"System Idle Process","0","Services","0","24 K","Unknown","NT AUTHORITY SYSTEM","2:07:39","N/A"

"System","4","Services","0","300 K","Unknown","N/A","0:00:07","N/A" "smss.exe","344","Services","0","1,204 K","Unknown","N/A","0:00:00","N/A" "csrss.exe","448","Services","0","5,028 K","Unknown","N/A","0:00:00","N/A" "csrss.exe","504","Console","1","3,772 K","Unknown","N/A","0:00:00","N/A"

"wininit.exe","512","Services","0","4,500 K","Unknown","N/A","0:00:00","N/A"

"winlogon.exe","540","Console","1","4,476 K","Unknown","N/A","0:00:00","N/A"

"services.exe","604","Services","0","8,700 K","Unknown","N/A","0:00:02","N/A"
"lsass.exe","612","Services","0","13,624 K","Unknown","N/A","0:00:01","N/A"

"lsm.exe","620","Services","0","6,016 K","Unknown","N/A","0:00:00","N/A"

inputs.conf

[batch://$SPLUNK_HOME\TEMP]

move_policy = sinkhole

interval = 60

source = transformfile

sourcetype = transformfile

disabled = 0

props.conf

[source::TRANSFORMFILE]

CHECK__FOR _HEADER=TRUE

SHOULD _LINEMERGE = false

TRANSFORM-transformfile = phy_csv

transforms.conf

[phy_csv]

DELIMS=","

FIELDS="Image Name", "PID", "Session Name", "Session#", "Mem Usage", "Status", "User Name", "CPU Time", "Window Title"

Any help here?

Re: Input a CSV

joshd — Mon, 28 Sep 2020 10:03:28 GMT

in your props.conf it should be TRANSFORMS-transformfile and not TRANSFORM-transformfile and it looks like you have two underscores in the CHECK_FOR_HEADER attribute along with a space in the SHOULD_LINE_MERGE attribute

also you may wish to use the sourcetype stanza when specifying it in props.conf instead of the source stanza

Re: Input a CSV

ofedorov — Fri, 04 Nov 2011 15:44:11 GMT

Thanks for the catch, mate.
But that didn't do the trick, indexed data is still in one chunk.
Maybe there is a way to strip those quote marks during .cmd output? Then the headers might get recognized properly.

Re: Input a CSV

joshd — Mon, 28 Sep 2020 10:03:32 GMT

Since you're specifying your own transform have you tried it without the CHECK_FOR_HEADER attribute in the props.conf?

Re: Input a CSV

joshd — Fri, 04 Nov 2011 16:19:49 GMT

Here's your config files rewritten correcting the minor mistakes, this should work...

inputs.conf :

[batch://$SPLUNK_HOMETEMP]
move_policy = sinkhole
interval = 60
source = transformfile
sourcetype = transformfile
disabled = 0

props.conf :

[transformfile]
SHOULD_LINEMERGE = false
TRANSFORMS-transformfile = phy_csv

transforms.conf :

[phy_csv]
DELIMS=","
FIELDS="Image Name","PID","Session Name","Session#","Mem Usage","Status","User Name","CPU Time","Window Title"

Re: Input a CSV

ofedorov — Mon, 28 Sep 2020 10:03:35 GMT

Thanks for the hint, but that didn't work either.
With "[batch://$SPLUNK_HOMETEMP]" file never gets picked up - changed to "[batch://$SPLUNK_HOME"bkslash"TEMP]".
Output still remains the same v_v

Re: Input a CSV

joshd — Fri, 04 Nov 2011 16:39:30 GMT

Sorry maybe I'm misunderstanding... your file is never getting indexed to begin with? Have you tried using a monitor stanza and the crcSalt attribute?

Re: Input a CSV

ofedorov — Fri, 04 Nov 2011 17:45:51 GMT

Oh, no it gets indexed.
File is being pulled from TEMP folder all right, but in Splunk it appears as a single chunk of data, never been broken into a table.

Re: Input a CSV

ofedorov — Fri, 04 Nov 2011 17:46:48 GMT

I've tried both with and without it - outcome is always the same.