Extract JSON out of an event

[brent_weaver]

I have an event like:

2017-08-22T13:00:56.257197+00:00 10.4.2.13 vcap.cloud_controller_ng [job=api_z1 index=2]  {"timestamp":1503406856.2571054,"message":"Completed 200 vcap-request-id: 60968128-7c32-4c94-632a-aa14909f454b::d5fb79e5-eed4-4154-a626-9a77473f6b71","log_level":"info","source":"cc.api","data":{},"thread_id":47266090216740,"fiber_id":70312747176840,"process_id":14279,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_logs.rb","lineno":24,"method":"call"}

As you will see there is JSON in the event. I have the regex to carve it out of there, how do I now make this into kv parse? I know I can do it inline with spath but I would like to not have to do that. I understand that there will be some performance implications in me doing this and am open to any other thoughts around this! Do I just use spath when I need to parse it?

Thanks!

[ben_leung]

I would suggest that you change the logging format. Your application already writes in JSON format, you just need to clobber the first line not in brackets into the rest of the content. This way, it should extract automatically without using search time parsing SPATH.

I like to tell teams to follow the index=_introspection logging format if you want auto extracted JSON fields.