This works perfect in the cases where MESSAGE contains two double quotes. in the cases (like the example i provided) where the MESSAGE field contains multiple double quotes Splunk can't seem to break the event properly.
But i would expect that to be broken into three separate events. From what i can gather Splunk has issues with the multiple double quotation marks and completely ignores the line_breaker regex.
If i change FIELD_QUOTE = " and instead put FIELD_QUOTE = None it does seem to break the events like it should, but the auto-extracted MESSAGE field will then contain quotation marks. With FIELD_QUOTE=" Splunk removes the double quotes around the event but fails to break the event properly.
I have also tried to change the double quotes inside the outer double quotes with SEDCMD like this:
Causing event breaking to be done correctly (because of FIELD_QUOTE = None i suspect) and one event will now be correctly:
USER1;Windows 10 Enterprise;20201128 06:05:27,862;5;0;File not accessible ('90s'). Error: 'Try again'.
The MESSAGE field on the left hand side under "Interesting fields" still contains double quotes. Despite the event above where all double quotes is either changed or removed, the MESSAGE field will have the value like this:
"File not accessible ("90s"). Error: "Try again"."
- This seem to indicate that whatever i try Splunk will extract the field before any change i do and therefore make it near impossible to fix this issue. I have not set INDEXED_EXTRACTION = CSV and have left KV_MODE = none to try to keep Splunk from extracting fields at both indexing and search time, but it still extracts the fields. My guess is that this has to do with the header. As long as Splunk is looking for and using the header in the CSV file it will also auto-extract the fields. I have not found a way to change this behavior.
In my desperation i also tried to write a transforms stanza to remove the header, then remove the HEADER_FIELD_DELIMITER and HEADER_FIELD_QUOTE settings, but STILL Splunk is extracting the header and the fields that goes with them. There is the HEADER_FIELD_LINE_NUMBER setting, but this can't be set to "none" and defaults to 0. Even if i set this to HEADER_FIELD_LINE_NUMBER = 1 and have the transforms to remove the header, Splunk extracts the HEADER fields automatically.
Does anyone have any tips for fixing this issue? What i want is the events to be broken down correctly with 1 event per line and to have the MESSAGE field values without quotation marks.
The obvious and best solution would be for the system to change it's logger and never include multiple double quotation marks in the MESSAGE text, but this is not possible i'm afraid, so i'm stuck trying to find a solution in Splunk. Note again that this is a single instance Splunk Enterprise installation with no forwarder (local input).