Today we have messages from our application like this:
2018-May-1 12:00:00.000 [Thread=4d2ce108-c322-49ff-bcc0-380d777f939f] INFO com.MyClass - method=search,customer=1234,user=Tester,time=0.044
Splunk auto-extraction handles the key=value pairs perfectly without the need to define specific abstractions.
We are moving these apps into cloud-hosting (PCF specifically) which in turn is wrapping our own logs in a JSON object, as follows:
{"app_id":"ABC1234","app_name":"myApp","msg":"2018-May-1 12:00:00.000 [Thread=4d2ce108-c322-49ff-bcc0-380d777f939f] INFO com.MyClass - method=search,customer=1234,user=Tester,time=0.044","source":"APP/PROC/WEB"}
While Splunk's auto-extractor recognizes this as JSON and parses out fields from the JSON wrapper (app_id, app_name, msg and source), auto extraction of our real "meat" fields within msg is not automatically performed. Defining each specific key=value extraction isn't something I want to do since they change.
Performing | rename msg as _raw | extract pairdelim=",", kvdelim="=" inline to a search effectively does what I want, that is it keeps the field extractions from the wrapper (which contain important metadata), while then re-parsing out the "msg" field (my understanding is Splunk's auto extractor only works on _raw, hence the rename of msg to _raw). However, I would like if this could be configured somehow on the props.conf or transforms.conf so it was automatic.
Is it possible to do this? That is, effectively have the auto extractor run twice, first parsing JSON and then picking out one of the fields and re-parsing that field?
