topic Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value? in Splunk Search

How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

pradjswl — Tue, 29 Sep 2020 10:33:02 GMT

I have the following events.

event 1)

[08-09-2016_08:00:40.567_PDT] [ERROR] - [ePdv0XVRu2] [xxx@yyy.com] [] [auth] [ResourceAuthenticationFilter] - TATS_SS_TOKEN_ID TOKEN IN SESSION GVtghrUaE%2FIU5H8Lpa%2FcfAhIZvdT7Q1Q%2F4UL3zgnngrOrL97eUYn5e0j8sXk5eN6%2FSQEsVAz066qk%2F1KanQjxreAL%2F4qAbPs5C6K9ZVKWAPENBF%2BC3k0nSDcXFTYw4Ep%2BvAt9HwFbCN9eg1Xj8qG9KfLa0Is%2B9YeGmEiYAH4MQoBmH6Zx6Y%2FStxOMNwsvySruKmdlnMpXeFLrPWbd6iVrCmCvOzIZZaNtyq9trGUAxHaTGbQxTkE8clMWcvUhenkhWxijr2%2F%2FnASvxU9rIrfgkV%2Bnirw2kLKZWf%2BW1e5nNpZ6OE9aZsaSXTYSaIno4RHG8qzwNMtvdykNJLIFCGFAj6Fdt7k8A3%2BSTYY5aircTcONh0u8GOPNuVWCFFc3WUQ DID NOT MATCH WITH COOKIE GVtghrUaE%2FIU5H8Lpa%2FcfAhIZvdT7Q1Q%2F4UL3zgnngrOrL97eUYn5e0j8sXk5eN6%2FSQEsVAz066qk%2F1KanQjxreAL%2F4qAbPs5C6K9ZVKWAPENBF%2BC3k0nSDcXFTYw4Ep%2BvAt9HwFbCN9eg1Xj8qG9KfLa0Is%2B9YeGmEiYAH4MQoBmH6Zx6Y%2FStxOMNwsvySruKmdlnMpXeFLrPWbd6iVrCmCvOzIZZaNtyq9trGUAxHaTGbQxTkE8clMWcvUhenkhWxijr2%2F%2FnASvxU9rIrfgkV%2Bnirw2kLKZWf%2BW1e5nNrTdaX1vVAhzrXBszldYtE5cEm9yffwuivWl6DpoobEqpZnTtfrVa3CEJ7uHqPv%2B1aj9K%2BaJz%2B%2Bc376kG5%2FJcNn PRESEN

event 2)

[08-09-2016_08:00:41.451_PDT] [ERROR] - [ePdv0XVRu2] [xxx@yyy.com [] [unauth] [ResourceReqValidationFilter] - Not Authorized TO Access this URI https:zzz.com

I am using this regular expression, and extracted the Error description.

(?:\].*?){7}\s-\s(?P.*)

The field a_xf_ErrorDescription returns a very large value for the1st event as you can see that it has cookie related information. In reality, there can't be readable format of English words continuing to more than 10-15 character(except space).

As per current regular expression

a_xf_ErrorDescription=TATS_SS_TOKEN_ID TOKEN IN SESSION GVtghrUaE%2FIU5H8Lpa%2FcfAhIZvdT7Q1Q%2F4UL3zgnngrOrL97eUYn5e0j8sXk5eN6%2FSQEsVAz066qk%2F1KanQjxreAL%2F4qAbPs5C6K9ZVKWAPENBF%2BC3k0nSDcXFTYw4Ep%2BvAt9HwFbCN9eg1Xj8qG9KfLa0Is%2B9YeGmEiYAH4MQoBmH6Zx6Y%2FStxOMNwsvySruKmdlnMpXeFLrPWbd6iVrCmCvOzIZZaNtyq9trGUAxHaTGbQxTkE8clMWcvUhenkhWxijr2%2F%2FnASvxU9rIrfgkV%2Bnirw2kLKZWf%2BW1e5nNpZ6OE9aZsaSXTYSaIno4RHG8qzwNMtvdykNJLIFCGFAj6Fdt7k8A3%2BSTYY5aircTcONh0u8GOPNuVWCFFc3WUQ DID NOT MATCH WITH COOKIE GVtghrUaE%2FIU5H8Lpa%2FcfAhIZvdT7Q1Q%2F4UL3zgnngrOrL97eUYn5e0j8sXk5eN6%2FSQEsVAz066qk%2F1KanQjxreAL%2F4qAbPs5C6K9ZVKWAPENBF%2BC3k0nSDcXFTYw4Ep%2BvAt9HwFbCN9eg1Xj8qG9KfLa0Is%2B9YeGmEiYAH4MQoBmH6Zx6Y%2FStxOMNwsvySruKmdlnMpXeFLrPWbd6iVrCmCvOzIZZaNtyq9trGUAxHaTGbQxTkE8clMWcvUhenkhWxijr2%2F%2FnASvxU9rIrfgkV%2Bnirw2kLKZWf%2BW1e5nNrTdaX1vVAhzrXBszldYtE5cEm9yffwuivWl6DpoobEqpZnTtfrVa3CEJ7uHqPv%2B1aj9K%2BaJz%2B%2Bc376kG5%2FJcNn PRESEN

Question 1) Is there a way for a field extraction to STOP & IGNORE a word which has more than 15 (or 20) characters ? So that the extracted field for event 1 would just have the value as:

a_xf_ErrorDescription=TATS_SS_TOKEN_ID TOKEN IN SESSION

Question 2) Is there a way for the field extraction to CONTINUE & IGNORE the word which has more than 15 (or 20) characters so that the extracted field for event 1 would have the value as:

a_xf_ErrorDescription=TATS_SS_TOKEN_ID TOKEN IN SESSION DID NOT MATCH WITH COOKIE PRESEN

The reason I want to Trim the extracted field to meaningful name so that it's easier to create a timechart with the field having common error.

Thanks for your feedback.

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

sundareshr — Tue, 09 Aug 2016 20:14:14 GMT

Try this run anywhere example

| makeresults | eval a_xf_ErrorDescription="TATS_SS_TOKEN_ID TOKEN IN SESSION GVtghrUaE%2FIU5H8Lpa%2FcfAhIZvdT7Q1Q%2F4UL3zgnngrOrL97eUYn5e0j8sXk5eN6%2FSQEsVAz066qk%2F1KanQjxreAL%2F4qAbPs5C6K9ZVKWAPENBF%2BC3k0nSDcXFTYw4Ep%2BvAt9HwFbCN9eg1Xj8qG9KfLa0Is%2B9YeGmEiYAH4MQoBmH6Zx6Y%2FStxOMNwsvySruKmdlnMpXeFLrPWbd6iVrCmCvOzIZZaNtyq9trGUAxHaTGbQxTkE8clMWcvUhenkhWxijr2%2F%2FnASvxU9rIrfgkV%2Bnirw2kLKZWf%2BW1e5nNpZ6OE9aZsaSXTYSaIno4RHG8qzwNMtvdykNJLIFCGFAj6Fdt7k8A3%2BSTYY5aircTcONh0u8GOPNuVWCFFc3WUQ DID NOT MATCH WITH COOKIE GVtghrUaE%2FIU5H8Lpa%2FcfAhIZvdT7Q1Q%2F4UL3zgnngrOrL97eUYn5e0j8sXk5eN6%2FSQEsVAz066qk%2F1KanQjxreAL%2F4qAbPs5C6K9ZVKWAPENBF%2BC3k0nSDcXFTYw4Ep%2BvAt9HwFbCN9eg1Xj8qG9KfLa0Is%2B9YeGmEiYAH4MQoBmH6Zx6Y%2FStxOMNwsvySruKmdlnMpXeFLrPWbd6iVrCmCvOzIZZaNtyq9trGUAxHaTGbQxTkE8clMWcvUhenkhWxijr2%2F%2FnASvxU9rIrfgkV%2Bnirw2kLKZWf%2BW1e5nNrTdaX1vVAhzrXBszldYtE5cEm9yffwuivWl6DpoobEqpZnTtfrVa3CEJ7uHqPv%2B1aj9K%2BaJz%2B%2Bc376kG5%2FJcNn PRESEN" | rex field=a_xf_ErrorDescription max_match=0 "\s(?<words>\w{1,10})\s?" | table words | nomv words

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

pradjswl — Tue, 09 Aug 2016 20:26:34 GMT

Thank you @ sundareshr . If I pipe my orignal query with the makeresults , I am getting following error.
Error in 'makeresults' command: This command must be the first command of a search.

Where do i specify the sourcetype and other part of the search criteria ?

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

somesoni2 — Tue, 09 Aug 2016 20:42:08 GMT

Remove everything before | rex field=a... and replace it with your original query

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

sundareshr — Tue, 09 Aug 2016 20:46:13 GMT

Like this

your base search | rex field=a_xf_ErrorDescription max_match=0 "\s(?<words>\w{1,10})\s?" | table words | nomv words

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

pradjswl — Tue, 09 Aug 2016 21:30:23 GMT

great ty @somesoni2 & @sundareshr

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

pradjswl — Tue, 09 Aug 2016 21:33:12 GMT

what does \w{1,10} do? Does it ignore any word of minimum 1 to maximum 10 characters?

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

sundareshr — Tue, 09 Aug 2016 21:38:03 GMT

It captures between 1 & 10 characters. I assume the longest word will be 10 characters and cookie wil be greater than that. You can increase/reduce the 10. Keep the 1

@pradjswl if this works, please accept the answer to close it out.

Re: How to edit my regular expression to extract a field and trim out strings with more than X characters (except space) from the value?

pradjswl — Wed, 10 Aug 2016 13:43:35 GMT

@sundareshr - Done. Sorry I didnt knew about Answer accepting. Just being new to site 🙂