topic Re: need help on REX command in Reporting

need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:02:21 GMT

Hi team , from above screenshot can you suggest how can i extract SEP013 against "errorCode":
I need count of events for SEP013

Note : am planning to put dedup on order id , so i will get correct count

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:06:29 GMT

adding command which i have tried but not working , need help correct rex patternt

index = app_events_sdda_core_de_prod source="/home/sdda/apps/logs/sep-app/app-json.log"
|fields message
|rex field=_raw "errorCode=(?<Error>[^,]*)"
|rex field=_raw "orderId":(?<Order>\w+)"
|table Error, Order

Re: need help on REX command

gcusello — Tue, 16 Jul 2024 06:07:23 GMT

Hi @bhaskar5428,

this seems to be a json format, so if you use INDEXED_EXTRACTIONS = JSON in the props.conf or if you use the spath command (https://docs.splunk.com/Documentation/Splunk/9.2.2/SearchReference/Spath) in the search you should be able to extract your fields.

It's also possible to use the rex command to extract the field, but to do this I need an example of your data in text format (not a screenshot), using the Inser/Edit code sample button (the one with "</>").

Ciao.

Giuseppe

Re: need help on REX command

manjunathmeti — Tue, 16 Jul 2024 06:12:22 GMT

| rex \"orderId\":\"(<?orderId>[\w\-]+)\" | rex \"errorCode\":\"(<?errorCode>\w+)\"

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:27:04 GMT

Please find the text message

message: Send jms message [queue=SEP.TO.PEIN, statusCode=200, idempotencyId=b95d0d10-9709-4299-9d3e-8c65dd5a539d, processId=PE2400000582026, delivApp=null, message={"transactionItems":[{"itemId":"4f2170cd-35f6-4d03-b0fe-6ebbca6e00cb","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"adea3dff-8e02-433e-a2ad-94bac828989b","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"bf96b534-2150-4228-843f-9fb920a1f44f","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"cb6e42ac-ee83-48b6-8213-7faf0311c6d0","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"bdd8a76f-ddb4-4616-a793-68ddd72aad0e","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"cb339f2a-ad85-410c-9043-275aa1e4fe17","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595011","currencyCode":"INR"}}}],"orderStatusResponse":{"orderStatus":"ORDER_FAILURE","orderId":"b95d0d10-9709-4299-9d3e-8c65dd5a539d"},"error":{"errorCode":"SEP013","errorDescription":"Cannot find IDMS-0784 account by accNumber: 0190595009"}}]

Above highlighted am trying to extract in two fields, Please help

and also below query am using

index = app_events_sdda_core_de_prod source="/home/sdda/apps/logs/sep-app/app-json.log" SEP013

But i can not use above , because eventually i need all errorcodes , and that is available on below search query.

index = app_events_sdda_core_de_prod source="/home/sdda/apps/logs/sep-app/app-json.log" level=TRACE ---> so please suggest how can i find error code and order id by using this search

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:27:28 GMT

Not working

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:30:00 GMT

message: Send jms message [queue=SEP.TO.PEIN, statusCode=200, idempotencyId=b95d0d10-9709-4299-9d3e-8c65dd5a539d, processId=PE2400000582026, delivApp=null, message={"transactionItems":[{"itemId":"4f2170cd-35f6-4d03-b0fe-6ebbca6e00cb","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"adea3dff-8e02-433e-a2ad-94bac828989b","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"bf96b534-2150-4228-843f-9fb920a1f44f","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"cb6e42ac-ee83-48b6-8213-7faf0311c6d0","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"bdd8a76f-ddb4-4616-a793-68ddd72aad0e","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595009","currencyCode":"INR"}}},{"itemId":"cb339f2a-ad85-410c-9043-275aa1e4fe17","status":"FAILED","accountIdentification":{"contractAccountNumber":{"branchCode":"0784","accountNumber":"0190595011","currencyCode":"INR"}}}],"orderStatusResponse":{"orderStatus":"ORDER_FAILURE","orderId":"b95d0d10-9709-4299-9d3e-8c65dd5a539d"},"error":{"errorCode":"SEP013","errorDescription":"Cannot find IDMS-0784 account by accNumber: 0190595009"}}]

Re: need help on REX command

gcusello — Tue, 16 Jul 2024 06:35:02 GMT

Hi @bhaskar5428 ,

this is the regex to extract the highlighted fields:

| rex "\"orderId\":\"(?<orderId>[^\"]+)\"},\"error\":\{\"errorCode\":\"(?<errorCode>[^\"]+)\""

that you can test at https://regex101.com/r/XkBntG/1

but I hint to try with INDEXED_EXTRACTIONS = JSON or spath.

Ciao.

Giuseppe

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:40:52 GMT

so basically i would like to find all error code with their order id
i want to use below main search

index = app_events_sdda_core_de_prod source="/home/sdda/apps/logs/sep-app/app-json.log" level=TRACE

so what i posted earlier part of above search
i want to extract all errorcodes and their order id

Later i am planning to put bar chart for the same based on range
hope you got requirement

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:47:03 GMT

can you please suggest how to use INDEXED_EXTRACTIONS = JSON or spath.
i mean any example and what are the benefits.

Re: need help on REX command

bhaskar5428 — Tue, 16 Jul 2024 06:58:08 GMT

btw, i tried below rex , any idea why its not showing any records

Re: need help on REX command

gcusello — Tue, 16 Jul 2024 08:39:58 GMT

Hi @bhaskar5428 ,

here you can find all the information https://hurricanelabs.com/splunk-tutorials/the-indexed-extractions-vs-search-time-extractions-splunk-case-study/

Anyway, you can add INDEXED_EXTRACTIONS on the props.conf on the UFs and or the SHs

or use something like this:

[your_sourcetype] INDEXED_EXTRACTIONS = JSON

or in search

index = app_events_sdda_core_de_prod source="/home/sdda/apps/logs/sep-app/app-json.log" | spath

Ciao.

Giuseppe

Re: need help on REX command

PickleRick — Tue, 16 Jul 2024 09:09:11 GMT

Honestly (yes, I know it is not helping in the immediate problem at hand but might save you some time in the future), this is a very ugly data format. I suspect that someone just receives data with a third-party solution which adds its own headers and forwards it to Splunk. The result is that you have some structure which Splunk is able to parse (the "outer json") and within that you have completely unparsed message field. And this field is "kinda like a json but not quite" so it doesn't parse properly (and I suspect there can be other types of events in that message field so no single parsing schema would work here).

In the long run it would be best if you could force admins of your source solution to provide data in a more sane format.

Re: need help on REX command

jotne — Tue, 16 Jul 2024 09:20:13 GMT

It would help if you post the data as text instead of a photo of it.