I am exporting an S3 bucket with only CSV and when I specified the sourcetype as CSV, I have all my events, but no fields about that event (aka the header is like an event). However, when I am importing my CSV in Splunk Web, the header is correctly inserted.
Any idea about how fix that?
Can you post your props config for this input?
my inputs.conf is :
[aws_s3://zx] aws_account = yx bucket_name = xy character_set = auto ct_blacklist = ^(?:Describe|List|Get) host_name = s3.amazonaws.com initial_scan_datetime = 2016-04-10T16:58:20+0200 key_name = devops/ max_items = 100000 max_retries = 3 polling_interval = 60 recursion_depth = -1 sourcetype = csv ct_excluded_events_index = index = data disabled = 0
Did you make any progress I am observing the same behaviour.
Hi Menaham, I believe this issue will be resolved by creating a props.conf with a [csv] stanza (this input's sourcetype, can be anything you want), and then setting the "INDEXED_EXTRACTIONS = CSV" config at that stanza. More info can be found in the Structured Data section here : http://docs.splunk.com/Documentation/Splunk/latest/Admin/propsconf
Please let me know if this answers your question!
It's not working, seems to be a bug with the add-on
Hello Menahem, I am trying to understand your expectation better. Could you explain further about your meaning behind "no fields about that event (aka the header is like an event)"
It is my understanding that the header of the file is used for field extractions. It would then be processed as a separate event and the fields that were extracted from the header will appear to the left under "Interesting Fields."
The problem is very simple, i don't have the different field's (even in "Interesting Fields") that is in the header of the file
I've added some information as a reply to the main question as I'm seeing the same issue but to clarify the problem is that when reading the file from S3 the header is not used for field extraction and is treated as if it were a separate event. So for a file like:
Using AWS S3 you get events like this:
Event 1 Header1,Header2,Header3
Event 2 Row1Value1,Row1Value2,Row1Value3
Event 3 Row2Value1,Row2Value2,Row2Value3
And no searchtime field extraction occurs
but if you add the file locally (just using the Add Data dialogue):
Event 1 Row1Value1,Row1Value2,Row1Value3
Event 2 Row2Value1,Row2Value2,Row2Value3
and the header field is used for field extraction at searchtime.