Splunk Search

Reduce the regex steps

dpatiladobe
Explorer

Trying to extract Dimensions out of Query, but it is taking 1500 plus steps due to which I am getting limits.conf error.

 

 

[{(, ](?<Dimensions>[a-z0-9A-Z\[\.\]+[\-\ \_]*)[\.&\[]

SELECT { [Measures].[IMS Org Count] } ON COLUMNS, NONEMPTY ( { [End User].[End User ID].[End User ID].MEMBERS * [End User].[End User Name].[End User Name].MEMBERS * [Product].[PMBU Short Desc].[PMBU Short Desc].MEMBERS * [Product].[PMBU Medium Desc].[PMBU Medium Desc].MEMBERS * [IMS Org].[Unique Id].[Unique Id].MEMBERS * [IMS Org].[IMS Org Id].[IMS Org Id].MEMBERS * [IMS Org].[MC Org Name].[MC Org Name].MEMBERS * [Is Active Account].[Is Active Account].[Is Active Account].MEMBERS* [Billing End User].[End User ID].[End User ID].MEMBERS } , [Measures].[IMS Org Count] ) ON ROWS FROM ( SELECT CASE '1' WHEN "5" THEN [Account Manager].[AM Org Lead Ldap].[AM Org Lead Ldap].[xxxx] WHEN "4" THEN [Account Manager].[Regional Manager Ldap].[Regional Manager Ldap].[xxx] WHEN "3" THEN [Account Manager].[AM Manager Ldap].[AM Manager Ldap].[xxx] WHEN "2" THEN [Account Manager].[AM Lead Ldap].[AM Lead Ldap].[xxx] WHEN "1" THEN [Account Manager].[AM Ldap].[AM Ldap].[xxxx] END ON 0 FROM XX )

 

 

https://regex101.com/r/HEdUhy/1/

Labels (2)
0 Karma
1 Solution

Birbal
Engager

 

[{(, ](?<Dimensions>(\[[^\]]+\]\.)+)

 

gives 755 steps for the data provided in your regex101.com my solution at  https://regex101.com/r/HEdUhy/2  with your exact prefix.

If you remove prefix it turns to be 357 steps for your given data. 

(?<Dimensions>(\[[^\]]+\]\.)+)

solution at https://regex101.com/r/HEdUhy/3 

View solution in original post

cmerriman
Super Champion

Is the regex101 what you want extracted? all those groups? or is it not exactly what you want? what is the limit problem? is this a regex you're doing in props or transforms or are you doing it with |rex on the search line? can you provide the entire stanza or the entire search string?

0 Karma

dpatiladobe
Explorer

The issue is regex is taking almost ~1300 steps due to which i am getting limit error's.

The query is  as below and regex is `[{(, ](?<Dimensions>[a-z0-9A-Z\[\.\]+[\-\ \_]*)[\.&\[]`

SELECT { [Measures].[IMS Org Count] } ON COLUMNS, NONEMPTY ( { [End User].[End User ID].[End User ID].MEMBERS * [End User].[End User Name].[End User Name].MEMBERS * [Product].[PMBU Short Desc].[PMBU Short Desc].MEMBERS * [Product].[PMBU Medium Desc].[PMBU Medium Desc].MEMBERS * [IMS Org].[Unique Id].[Unique Id].MEMBERS * [IMS Org].[IMS Org Id].[IMS Org Id].MEMBERS * [IMS Org].[MC Org Name].[MC Org Name].MEMBERS * [Is Active Account].[Is Active Account].[Is Active Account].MEMBERS* [Billing End User].[End User ID].[End User ID].MEMBERS } , [Measures].[IMS Org Count] ) ON ROWS FROM ( SELECT CASE '1' WHEN "5" THEN [Account Manager].[AM Org Lead Ldap].[AM Org Lead Ldap].[xxxx] WHEN "4" THEN [Account Manager].[Regional Manager Ldap].[Regional Manager Ldap].[xxx] WHEN "3" THEN [Account Manager].[AM Manager Ldap].[AM Manager Ldap].[xxx] WHEN "2" THEN [Account Manager].[AM Lead Ldap].[AM Lead Ldap].[xxx] WHEN "1" THEN [Account Manager].[AM Ldap].[AM Ldap].[xxxx] END ON 0 FROM XX )

  

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I think that @cmerriman has a valid point about your data. Is that the exact set of matches that you want to get? If it is, then that makes it so that we know the actual constraints. If you are saying only that you want to get that data exactly as it is with that data, but you just want to reduce the steps so that you don't have to make changes to limits.conf, then that is one thing.

Looking at the results you got in regex101, I'm not seeing any kind of correlation to useful data that you could use from the regex you are using. I think that is your biggest problem in getting a good answer. If you want to get different results, then that would make finding a better regex a whole lot easier. As it is, I'm finding this very difficult to help find you a solution because if the data changes slightly but still be valid SQL, then you won't get a similar resulting set of Dimensions from your regex. Hopefully you can see the point I'm trying to make.

0 Karma

dpatiladobe
Explorer

Yes these what i wanted "If you are saying only that you want to get that data exactly as it is with that data, but you just want to reduce the steps so that you don't have to make changes to limits.conf, then that is one thing."

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

This regex will reduce your steps from 1322 to 1228, but I have no idea if it will prevent your limits errors:

[{(, ](?<Dimensions>[-.\w[\]+ ]+)[.&[]

 The data that you are working with and the result set that you want is rather abstract and hard to make better, since it isn't very comprehendible. The actual goal of what a Dimensions is is not clear. 

0 Karma

Birbal
Engager

 

[{(, ](?<Dimensions>(\[[^\]]+\]\.)+)

 

gives 755 steps for the data provided in your regex101.com my solution at  https://regex101.com/r/HEdUhy/2  with your exact prefix.

If you remove prefix it turns to be 357 steps for your given data. 

(?<Dimensions>(\[[^\]]+\]\.)+)

solution at https://regex101.com/r/HEdUhy/3 

dpatiladobe
Explorer

These is what something i wanted. Thank you Appreciated.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...