Regex search help?

[monicateja]

log: {“timeMillis”:“1667091964927",“timestamp”:“2022-10-30T01:06:04.927Z”,“thread”:“reactor-http-epoll-3",“level”:“INFO”,“logger”:“com.status”,“message”:“Business Key=null, Publishing  status : [ client_req_id fd6e03ea-99ed-4f0c-b52f-db7ceca0adc8, event_date 2022-10-30T01:06:04.927Z, event_name STORE_EXIT_AUDIT_RESULT, request_source RECEIPT_AUDIT_SERVICE ] message {“receipts”:[{“tc_id”:“20420249182009622534”,“tc_date”:“2022-10-30T01:06:04.927Z”,“tc_store_id”:“92”,“tc_operator_number”:“11”,“tc_register_number”:“11”,“tc_audit_status”:“SUCCESS”,“tc_previously_audited”:false,“tc_previously_audited_date”:null}],“audit_result”:{“audit_date”:“2022-10-30T01:06:03.846Z”,“audit_store_id”:“92",“audit_associate_id”:“103956635",“audit_result”:“Pass”,“audit_failure_reason”:null,“scanned_items_number”:3,“items_found”:[],“items_not_found”:[]}}” ,“traceId”:“”, “spanId”:“”, “parentSpanId”:“”, “traceparent”:“”, “tracestate”:“”, “message_code”:“”, “flag”:“”, “tenant”: “”, “businessKey”:“”, “ringId”:“”, “locationNumber”:“”, “dc”:“”}

I want to see Store exit result with operator number, store id, register number , audit result , scanned items number , items found , items not found and also audit date , tc_id, timestamp in a table with those values.

can anyone please 

[yuanliu]

As I always tell people, do not attempt regex on structured data like JSON.  So, my first question is: How faithful is that sample event in terms of format?  I ask because the quotation marks used are all over places. This inconsistency makes me doubt the fidelity of the rest of format.  After correcting for inconsistent quotation marks, the part that belongs to "log" is still not conformant JSON.

If the log is truly this messy, you can take consolation that a small, but key part of the log is still conformant.  Instead of starting with "log" field, I have to start from that second "message".

| rex "message: (?<in_message>{.+}})"
| spath input=in_message path=receipts{}
| mvexpand receipts{}
| spath input=receipts{}
| spath input=in_message path=audit_result
| spath input=audit_result
| fields - data log in_message _time audit_result receipts{}

After this, you can expect something like

audit_associate_id

audit_date

audit_failure_reason

audit_store_id

scanned_items_number

tc_audit_status

tc_date

tc_id

tc_operator_number

tc_previously_audited

tc_previously_audited_date

tc_register_number

tc_store_id

103956635

2022-10-30T01:06:03.846Z

null

92

3

SUCCESS

2022-10-30T01:06:04.927Z

20420249182009622534

11

false

null

11

92

I think these are the fields that you wanted.