In search, how do you extract key=value pairs from a specific field?

[kjoyner]

I have events that are added to an index using the Splunk Logging Driver for Docker. I would like to extract key-value pairs from the line field. I couldn't figure out how to do this with extract since it doesn't accept a field as a parameter. I tried to do it with rex and other methods but couldn't figure out how to get it to add individual key-value pairs and make the keys available as fields.

For example, I have the following event:

{"line":"2016-10-18 21:20:18.802  INFO [edge-server,a6ac3cdcebf511a,a6ac3cdcebf511a,false] 1 --- [io-8765-exec-23] c.q.s.s.edge.filters.PostLoggingFilter   : Stats: [url=http://services.quicken.com/mint-xevent/processRequest.xevent, method=POST, query=clientType=Quicken\u0026clientID=%7b385f58af-91f8-11e6-82dd-806e6f6e6963%7d\u0026platform=QUICKEN_DESKTOP\u0026clientVersion=26.1.2.7\u0026buildNumber=7\u0026systemName=Windows\u0026systemVersion=6.2.9200.2.\u0026apiProtocol=3.0.0\u0026deviceName=Fields-PC, size=48, status=200, time=186, ip=10.11.10.29, phase=post]\n","stream":"stdout","time":"2016-10-18T21:20:18.802172642Z"}

From the line field, I would like to extract the following keys: url, method, query, size, status, time, ip, phase. I could then use those fields in other Splunk functions to generate statistics.

I'm looking at how to make this extraction dynamic (different events may have different key-value pairs). The only thing in common is that they are in the line field and have key=value syntax. If more than one, then they are separated by a ,.

[sundareshr]

Try this run-anywhere sample

| makeresults | eval x="{\"line\":\"2016-10-18 21:20:18.802  INFO [edge-server,a6ac3cdcebf511a,a6ac3cdcebf511a,false] 1 --- [io-8765-exec-23] c.q.s.s.edge.filters.PostLoggingFilter   : Stats: [url=http://services.quicken.com/mint-xevent/processRequest.xevent, method=POST, query=clientType=Quicken\u0026clientID=%7b385f58af-91f8-11e6-82dd-806e6f6e6963%7d\u0026platform=QUICKEN_DESKTOP\u0026clientVersion=26.1.2.7\u0026buildNumber=7\u0026systemName=Windows\u0026systemVersion=6.2.9200.2.\u0026apiProtocol=3.0.0\u0026deviceName=Fields-PC, size=48, status=200, time=186, ip=10.11.10.29, phase=post]\n\",\"stream\":\"stdout\",\"time\":\"2016-10-18T21:20:18.802172642Z\"}" | rex max_match=0 field=x "(?<key>\w+)=(?<value>[^,]+)" | eval z=mvzip(key, value, "~") | mvexpand z | rex field=z "(?<key>[^~]+)~(?<value>.*)" | table key value | eval dummy="" | xyseries dummy key value | fields - dummy