Would it possible for you to turn on KV mode as XML to extract search time field extraction.
If not you might have to try xpath or spath SPL command. Once you feed XML data as a field to xpath or spath, you can define the xml path for field extraction. http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath
You could get around this by doing a search time field extraction using the
rex command. I would recommend taking a sample of your data and going to
www.regex101.com and paste your sample data in then build a regular expression which matches all your conditions. Once you've done this, you can go to Splunk and format it to include the field name
index=your_base_search | rex \d+\[(?<FieldName>[A-Z0-9]+)
Thank you both for you answers.
@Skoelpin, I have used the regex method before and it works, however with regex101 sometime the regex does not work in Splunk, and if the events are not consistent then the results are missing.
@Niketnilay, you pointed me to the right resource, I found [.... | xmlkv ...] that works!
Thank you both!!
I'm interested in seeing what's not working. Can you give me an example of a regular expression that works in regex101 but not in Splunk?
I don't have one at the moment, but I have used the regex101 editor to get (regex) the field/values I wanted. Then I put the regex is Splunk and it did not work. When I compared the regex101 expression to the "Extract New Fields" regex the syntax was different. This does not happen a lot but it has caused problems. I will dig around for the regex101 expression but it was a few months ago and probably discarded it. My suggestion would be to compare the regex101 to the regex created by the "Extract New Fields" GUI.
The automatic field extractor in Splunk is very primitive and needs a lot of work. You do need to create the fieldname inside the regular expression after building it in regex101, perhaps you did not include the
<FIELD_NAME> inside the capture group (which Splunk requires)?
perhaps, most of the time the 101 results worked, ultimately I need the original example to rule out any possible error.