Split the content of a JSON string that is inside a JSON data log

[rafamss]

Hi everyone,

I have logs like the line below. I want to split the content of the request_headers field during search time. I tried to use spath, mvexpand, and split command, but without success. Could you help me?

This log was anonymized using the scrub command.

{"account_id":"000000000aaaa","audience":"qhau812n","caller":"christopher/logging.az:272","duration_seconds":5.1q-07,"forwarded_for":"43.021.022.16","host":"blue_car.n1l-prod.com","cassaundra":{"annotations":{"irmgard.p2z.coy/wxcmfcoKsdhourfZjjnnoe":"disabled","checksum/margarita":"721637kh01d24w0ww0kum20552n0vuuo0dih06y01061pmbq06223004f30230b1","cluster-marquerite.cassaundra.ws/safe-to-evict":"true","nga.christene.com/role":"aqsmvuyjwazaff-role-zlm912-0a","denisha.cassaundra.ws/guillermiNa":"1010-04-00I03:18:07-04:00","cassaundra.ws/bee":"ena.privileged","reloader.valentin.com/auto":"true"},"container_hash":"205509154532.blue.john.zzabc.christene.com/aqsmvuyjwazaff@rzv013:02vn2k3o301sql10le02ysn1jb42050gt2100k1b112lq72f1243v0601ao8j34x","container_image":"205509154532.eva.ken.zlm912-0a.christene.com/aqsmvuyjwazaff:10123u1","container_name":"aqsmvuyjwazaff","tressa_id":"2w0z641dn0155r0jn0e126ozu0be001uvb1ebz0470g55u11401x0zj1dt000djq","host":"ip-00-116-16-002.js1.internal","labels":{"app":"aqsmvuyjwazaff","app.cassaundra.ws/component":"yolando","app.cassaundra.ws/instance":"aqsmvuyjwazaff","app.cassaundra.ws/managed-by":"Helm","app.cassaundra.ws/name":"aqsmvuyjwazaff","app.cassaundra.ws/version":"10123u1","chart":"yolando-0.11.5","helm.sh/chart":"yolando-0.11.5","helm.sh/timestamp":"10200111020100","heritage":"Helm","pod-template-hash":"543312340d","n1l-architecture":"az-kit","n1l-cluster":"ena-svc","n1l-peter":"true","n1l-https":"true","n1l-yq":"false","n1l-hildegarde":"true","n1l-service-version":"10123u1","n1l-splunk":"true","n1l-tier":"yolando","grocerystore.ami/architecture":"az-kit","grocerystore.ami/cluster":"ena-svc","grocerystore.ami/peter":"true","grocerystore.ami/https":"true","grocerystore.ami/yq":"false","grocerystore.ami/hildegarde":"true","grocerystore.ami/selector-label":"aqsmvuyjwazaff","grocerystore.ami/splunk":"true"},"alexandra_name":"n1l","pod_id":"z50s6323-x5zm-435e-p1m2-tdqr6383m030","pod_name":"aqsmvuyjwazaff-deployment-543312340d-gc0is"},"level":"error","method":"GET","method_name":"AlrYcgwqpofDngmftgihrqpZzmhcWbXnsbhmpl","msg":"completed request","proto":"HTTP/1.1","remote_address":"12.10.21.0:25004","request_headers":"{\"Accept-Encoding\":[\"gzip\"],\"Origin\":[\"www.grocerystore.ami\"],\"User-Agent\":[\"Evelia AufroSuwbg\"],\"Via\":[\"1.1 0010z4437f3bd4031q6cfn0051l0h10b.aufrosuwbg.net (AufroSuwbg)\"],\"X-Bob-Mk-Id\":[\"MlxVFb3Yiob0Qh5l_wPlZf_kmtQiRAkvLSBvxIYQR1NhgR46KuE1ct==\"],\"X-Forwarded-For\":[\"43.021.022.16\",\"12.10.20.14\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Oe-Fingerprint\":[\"y-lue-db-l-x-x-013wj11b-00000000-471uga2k-00000000-p2wcw222-n-x-1.1-u-x-x-n-n\"]}","request_path":"/tad/x-mens/availability/bear/113202/month?start_date=1010-07-00J00:00:00.000H","response_code":400,"response_size":0,"felicitas_body_strings":[],"stacktrace":"*errors.charlEsetta client:query not encoded\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/blue_car/shared/helper_functions.az:226 (0n01g02n7)\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/blue_car-server/rest-blue_car/transport_availability.az:005 (0d02p000c)\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.cgmwskbczdos.com/N1L/H0IFrylh/christopher/server_options_http.az:021 (0tkg64ou)\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.com/az-kit/kit/transport/http/server.az:102 (0jo23322)\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.com/azrilla/lan/lan.az:100 (0m0332102)\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.com/azrilla/handlers/russ.az:106 (0m602mx7)\n/usr/local/az/src/net/http/server.az:1830 (0y10q110)\n/usr/local/az/src/net/http/server.az:0620 (0e70433a)\n/usr/local/az/src/runtime/rae_qmd51.s:1310 (0t150fa0)","stream":"marlon","time":"1010-04-02B00:59:08.056537201B","transaction_id":"00v0002j-1n7s-3m23-5kw4-264164hj13p0","ts":"1010-04-02B00:59:08.083V","type":"HTTP"}

 Thanks.

[ITWhisperer · Accepted solution]

The issue appears to be that the duration_seconds has q instead of e separating the mantissa and exponent. If you substitute the q for an e, the json can be parsed by spath.

| makeresults 
| eval _raw="{\"account_id\":\"000000000aaaa\",\"audience\":\"qhau812n\",\"caller\":\"christopher/logging.az:272\",\"duration_seconds\":5.1q-07,\"forwarded_for\":\"43.021.022.16\",\"host\":\"blue_car.n1l-prod.com\",\"cassaundra\":{\"annotations\":{\"irmgard.p2z.coy/wxcmfcoKsdhourfZjjnnoe\":\"disabled\",\"checksum/margarita\":\"721637kh01d24w0ww0kum20552n0vuuo0dih06y01061pmbq06223004f30230b1\",\"cluster-marquerite.cassaundra.ws/safe-to-evict\":\"true\",\"nga.christene.com/role\":\"aqsmvuyjwazaff-role-zlm912-0a\",\"denisha.cassaundra.ws/guillermiNa\":\"1010-04-00I03:18:07-04:00\",\"cassaundra.ws/bee\":\"ena.privileged\",\"reloader.valentin.com/auto\":\"true\"},\"container_hash\":\"205509154532.blue.john.zzabc.christene.com/aqsmvuyjwazaff@rzv013:02vn2k3o301sql10le02ysn1jb42050gt2100k1b112lq72f1243v0601ao8j34x\",\"container_image\":\"205509154532.eva.ken.zlm912-0a.christene.com/aqsmvuyjwazaff:10123u1\",\"container_name\":\"aqsmvuyjwazaff\",\"tressa_id\":\"2w0z641dn0155r0jn0e126ozu0be001uvb1ebz0470g55u11401x0zj1dt000djq\",\"host\":\"ip-00-116-16-002.js1.internal\",\"labels\":{\"app\":\"aqsmvuyjwazaff\",\"app.cassaundra.ws/component\":\"yolando\",\"app.cassaundra.ws/instance\":\"aqsmvuyjwazaff\",\"app.cassaundra.ws/managed-by\":\"Helm\",\"app.cassaundra.ws/name\":\"aqsmvuyjwazaff\",\"app.cassaundra.ws/version\":\"10123u1\",\"chart\":\"yolando-0.11.5\",\"helm.sh/chart\":\"yolando-0.11.5\",\"helm.sh/timestamp\":\"10200111020100\",\"heritage\":\"Helm\",\"pod-template-hash\":\"543312340d\",\"n1l-architecture\":\"az-kit\",\"n1l-cluster\":\"ena-svc\",\"n1l-peter\":\"true\",\"n1l-https\":\"true\",\"n1l-yq\":\"false\",\"n1l-hildegarde\":\"true\",\"n1l-service-version\":\"10123u1\",\"n1l-splunk\":\"true\",\"n1l-tier\":\"yolando\",\"grocerystore.ami/architecture\":\"az-kit\",\"grocerystore.ami/cluster\":\"ena-svc\",\"grocerystore.ami/peter\":\"true\",\"grocerystore.ami/https\":\"true\",\"grocerystore.ami/yq\":\"false\",\"grocerystore.ami/hildegarde\":\"true\",\"grocerystore.ami/selector-label\":\"aqsmvuyjwazaff\",\"grocerystore.ami/splunk\":\"true\"},\"alexandra_name\":\"n1l\",\"pod_id\":\"z50s6323-x5zm-435e-p1m2-tdqr6383m030\",\"pod_name\":\"aqsmvuyjwazaff-deployment-543312340d-gc0is\"},\"level\":\"error\",\"method\":\"GET\",\"method_name\":\"AlrYcgwqpofDngmftgihrqpZzmhcWbXnsbhmpl\",\"msg\":\"completed request\",\"proto\":\"HTTP/1.1\",\"remote_address\":\"12.10.21.0:25004\",\"request_headers\":\"{\\\"Accept-Encoding\\\":[\\\"gzip\\\"],\\\"Origin\\\":[\\\"www.grocerystore.ami\\\"],\\\"User-Agent\\\":[\\\"Evelia AufroSuwbg\\\"],\\\"Via\\\":[\\\"1.1 0010z4437f3bd4031q6cfn0051l0h10b.aufrosuwbg.net (AufroSuwbg)\\\"],\\\"X-Bob-Mk-Id\\\":[\\\"MlxVFb3Yiob0Qh5l_wPlZf_kmtQiRAkvLSBvxIYQR1NhgR46KuE1ct==\\\"],\\\"X-Forwarded-For\\\":[\\\"43.021.022.16\\\",\\\"12.10.20.14\\\"],\\\"X-Forwarded-Proto\\\":[\\\"https\\\"],\\\"X-Oe-Fingerprint\\\":[\\\"y-lue-db-l-x-x-013wj11b-00000000-471uga2k-00000000-p2wcw222-n-x-1.1-u-x-x-n-n\\\"]}\",\"request_path\":\"/tad/x-mens/availability/bear/113202/month?start_date=1010-07-00J00:00:00.000H\",\"response_code\":400,\"response_size\":0,\"felicitas_body_strings\":[],\"stacktrace\":\"*errors.charlEsetta client:query not encoded\\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/blue_car/shared/helper_functions.az:226 (0n01g02n7)\\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/blue_car-server/rest-blue_car/transport_availability.az:005 (0d02p000c)\\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.cgmwskbczdos.com/N1L/H0IFrylh/christopher/server_options_http.az:021 (0tkg64ou)\\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.com/az-kit/kit/transport/http/server.az:102 (0jo23322)\\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.com/azrilla/lan/lan.az:100 (0m0332102)\\n/az/src/barbie.cgmwskbczdos.com/N1L/AqsmvuyJwazaff/vendor/barbie.com/azrilla/handlers/russ.az:106 (0m602mx7)\\n/usr/local/az/src/net/http/server.az:1830 (0y10q110)\\n/usr/local/az/src/net/http/server.az:0620 (0e70433a)\\n/usr/local/az/src/runtime/rae_qmd51.s:1310 (0t150fa0)\",\"stream\":\"marlon\",\"time\":\"1010-04-02B00:59:08.056537201B\",\"transaction_id\":\"00v0002j-1n7s-3m23-5kw4-264164hj13p0\",\"ts\":\"1010-04-02B00:59:08.083V\",\"type\":\"HTTP\"}"
| rex mode=sed "s/(?<d>duration_seconds\":\d\.\d+)q/\1e/1"
| spath path=request_headers output=request_headers
| spath input=request_headers

I am assuming that this still represents the same value, but since you are looking for the request_header field, this may not be important.

[rafamss]

Thank you, @ITWhisperer. It worked fine.