Getting Data In

Why is my configuration not extracting CSV at index-time?

gschmitz
Path Finder

Hi all,
I'm trying to use INDEXED_EXTRACTIONS = CSV but for some reason it's just not working. My input looks as follows


***SPLUNK*** sourcetype=csv source=index/host/query.sql

"SESSION_ID","LOGON_TIME","SCHEMA_NAME","TOTAL_SESSION_MEMORY"

"119","2014-08-22 11:04:03","SYS","813704"


and my props.conf


[csv]

DATETIME_CONFIG=NONE

INDEXED_EXTRACTIONS=CSV

TRANSFORMS-index=index-as-first-folder


None of the four fields are extracted, but the TRANSFORMS as well as the DATETIME_CONFIG take effect. Can anybody spot a mistake?

0 Karma

gschmitz
Path Finder

It's got nothing to do with the CSV, but sets the index to the name of the first folder in my source 🙂

0 Karma

jrodman
Splunk Employee
Splunk Employee

By default, the triple-splat ***SPLUNK*** magic cookie is not enabled for logfiles, and is probably breaking the default csv handling. It's enabled for the first line for scripted inputs by default, if I recall correctly. I'm not aware if if INDEXED_EXTRACTIONS can work for scripted inputs. The design requires a certain amount of seeking around which makes it hard to do fully generic stream processing.

0 Karma

gschmitz
Path Finder

Ok. I tried PSV and TSV as well. No success. Opened case 187571.

0 Karma

gschmitz
Path Finder

Hi,

the output is actually from a script which changes the sourcetype repeatedly through the run of the script. So it's respected since the DATETIME_CONFIG is applied (the event was earlier in sent to August, so I'm pretty sure).

Lastly, I have another scripted input which uses TSV, but in this case I don't use ***SPLUNK***. Either INDEXED_TRANSACTIONS is really determined based on the first line, or CSV is broken. I may just change the delimiters to pipes or tabs and check it out later.

Best Regards.

0 Karma

woodcock
Esteemed Legend

Because the file has no header, you should use INDEXED_EXTRACTIONS = csv with the FIELD_NAMES option:
http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Extractfieldsfromfileheadersatindextime

Perhaps the magic cookie will be honored later down the line or perhaps it will be treated as an event (and you can strip it with a transform to null queue).

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Would you please share the relevant transform?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...

Splunk and Fraud

Watch Now!Watch an insightful webinar where we delve into the innovative approaches to solving fraud using the ...