Getting Data In

Why is my configuration not extracting CSV at index-time?

gschmitz
Path Finder

Hi all,
I'm trying to use INDEXED_EXTRACTIONS = CSV but for some reason it's just not working. My input looks as follows


***SPLUNK*** sourcetype=csv source=index/host/query.sql

"SESSION_ID","LOGON_TIME","SCHEMA_NAME","TOTAL_SESSION_MEMORY"

"119","2014-08-22 11:04:03","SYS","813704"


and my props.conf


[csv]

DATETIME_CONFIG=NONE

INDEXED_EXTRACTIONS=CSV

TRANSFORMS-index=index-as-first-folder


None of the four fields are extracted, but the TRANSFORMS as well as the DATETIME_CONFIG take effect. Can anybody spot a mistake?

0 Karma

gschmitz
Path Finder

It's got nothing to do with the CSV, but sets the index to the name of the first folder in my source 🙂

0 Karma

jrodman
Splunk Employee
Splunk Employee

By default, the triple-splat ***SPLUNK*** magic cookie is not enabled for logfiles, and is probably breaking the default csv handling. It's enabled for the first line for scripted inputs by default, if I recall correctly. I'm not aware if if INDEXED_EXTRACTIONS can work for scripted inputs. The design requires a certain amount of seeking around which makes it hard to do fully generic stream processing.

0 Karma

gschmitz
Path Finder

Ok. I tried PSV and TSV as well. No success. Opened case 187571.

0 Karma

gschmitz
Path Finder

Hi,

the output is actually from a script which changes the sourcetype repeatedly through the run of the script. So it's respected since the DATETIME_CONFIG is applied (the event was earlier in sent to August, so I'm pretty sure).

Lastly, I have another scripted input which uses TSV, but in this case I don't use ***SPLUNK***. Either INDEXED_TRANSACTIONS is really determined based on the first line, or CSV is broken. I may just change the delimiters to pipes or tabs and check it out later.

Best Regards.

0 Karma

woodcock
Esteemed Legend

Because the file has no header, you should use INDEXED_EXTRACTIONS = csv with the FIELD_NAMES option:
http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Extractfieldsfromfileheadersatindextime

Perhaps the magic cookie will be honored later down the line or perhaps it will be treated as an event (and you can strip it with a transform to null queue).

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Would you please share the relevant transform?

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Should our Deployment Servers have the Search Head server role on them?

all of our stuff is on premcurrently our dedicated Deployment Servers also have the Search Head role on them, ...

Why am I unable to create dropdown static option that is "All" static options ...

Hi, I am trying to get a static option that is "All" the individual static options combined.  The mCode ...

Where can we set the script (host=CSG196) so can we deploy the script in host?

Hi  We face a challenge We have created one alert in which we are monitoring one of the windows service ...