Show all possible values

[weird_guy]

Hello.

I have a lot of events. Each event contains similar string \"errorDetail\":\"possible_value\" 

Please specify how to create new field \"errorDetail\" and  stats all possible values? (There are more than 50 kinds of errorDetail)

For example:

\"errorDetail\":\"acctNumber\" 
\"errorDetail\":\"Message Version higher"\
\"errorDetail\":\"email\"

Thank you.

[weird_guy]

Hello @ITWhisperer 

Ive entered 

INTERNAL_VALIDATION_FAILED| spath
| rex field=statusMessage "\[(?<ds_message>[^\]]+)"
| spath input=ds_message
| stats count by errorDetail

And there is only "errorDetail\":  + count of events without values.

[ITWhisperer]

Here is a runanywhere example using your original event data showing the solution working. If it is not working with your real data, this means that the sample you shared is not an accurate representation of your real data. Please share an updated accurate representation of your data.

[weird_guy]

Hello

Here is an updated accurate data.

Thank you.

3DS2 Server ARes Response: {"messageType":"ARes","status":"INTERNAL_VALIDATION_FAILED","statusMessage":"invalid message fields, wrong message from ds:[{\"threeDSServerTransID\":\"123\",\"messageType\":\"Erro\",\"messageVersion\":\"2.2.0\",\"acsTransID\":\"345\",\"dsTransID\":\"567\",\"errorCode\":\"305\",\"errorComponent\":\"A\",\"errorDescription\":\"Cardholder Account Number is not in a range belonging to Issuer\",\"errorDetail\":\"acctNumber\",\"errorMessageType\":\"AReq\"}]; type[Erro] code[101] component[SERVER]"}

[ITWhisperer]

Try this

| rex "Response: (?<response>\{.+\})"
| spath input=response
| rex field=statusMessage "ds:\[(?<ds_message>[^\]]+)"
| spath input=ds_message
| stats count by errorDetail

[weird_guy]

Magic. It works.

But small issue here. It shows \"errorDetail\":

Hmmm

[ITWhisperer]

Again, here is a runanywhere example with your sample data

| makeresults
| eval _raw="3DS2 Server ARes Response: {\"messageType\":\"ARes\",\"status\":\"INTERNAL_VALIDATION_FAILED\",\"statusMessage\":\"invalid message fields, wrong message from ds:[{\\\"threeDSServerTransID\\\":\\\"123\\\",\\\"messageType\\\":\\\"Erro\\\",\\\"messageVersion\\\":\\\"2.2.0\\\",\\\"acsTransID\\\":\\\"345\\\",\\\"dsTransID\\\":\\\"567\\\",\\\"errorCode\\\":\\\"305\\\",\\\"errorComponent\\\":\\\"A\\\",\\\"errorDescription\\\":\\\"Cardholder Account Number is not in a range belonging to Issuer\\\",\\\"errorDetail\\\":\\\"acctNumber\\\",\\\"errorMessageType\\\":\\\"AReq\\\"}]; type[Erro] code[101] component[SERVER]\"}"
| rex "Response: (?<response>\{.+\})"
| spath input=response
| rex field=statusMessage "ds:\[(?<ds_message>[^\]]+)"
| spath input=ds_message
| stats count by errorDetail

If it is not working for some of your real data, then your sample is not an accurate representation of said (failing) data.

[yuanliu]

There can be a small problem: the error message, or "invalid message fields, wrong message from ds" as prefaced in the raw message, is a JSON array.  You want to handle that as an entity.

| rex "^[^{]+(?<response>.+)"
| spath input=response
| rename messageType as topMessageType ``` handle namespace conflict ```
| rex field=statusMessage "^[^\[]+(?<message_from_ds>[^\]]+\])"
| spath input=message_from_ds path={}
| mvexpand {}
| spath input={}
| dedup errorDetail
| table errorDetail

[weird_guy]

Hello @ITWhisperer 

Thank you for your response.

Here is the raw data:

{

"messageType": "Data",
"status": "Error",
"statusMessage": "invalid message fields, wrong message from ds:[{\"threeDSServerTransID\":\"123\",\"messageType\":\"Erro\",\"messageVersion\":\"2.2.0\",\"acsTransID\":\"123\",\"dsTransID\":\"123\",\"errorCode\":\"305\",\"errorComponent\":\"A\",\"errorDescription\":\"Transaction data not valid\",\"errorDetail\":\"No issuer found\",\"errorMessageType\":\"AReq\"}]; type[Erro] code[101] component[SERVER]"

}

[PickleRick]

Ugh. That's a pretty example of ugly data. Technically your data is a json structure with a field containing a string. That string describes another json structure but from splunk's point of view it's just a string. That makes it very inconvenient and possibly inefficient to manipulate. It would be much better if you got this from your source as some more sane format.

[ITWhisperer]

From your raw event you could do this

| spath
| rex field=statusMessage "\[(?<ds_message>[^\]]+)"
| spath input=ds_message
| stats count by errorDetail

if you have already extracted statusMessage when the event was ingested, you can skip the first spath command

[ITWhisperer]

This looks like JSON format data - if so, you should be extracting as JSON and using the JSON functions to manipulate the data. Please share your full event in raw format in a code block, anonymise your data as appropriate. This will enable volunteers to better guide you on a way forward.