Splunk Search

Regex Help; exceeded configured match_limit

JoshuaJohn
Contributor

I have this query that works in all regex assist sites but is too greedy for my Splunk Environment. I am unable to add it to props, and it must be in the query itself.

| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+).+?(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+).+?(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+).+?(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+).+?(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+).+?(?:\"isProduction\\\" : )(?<Prod>[^,]+).+?(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+).+?(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+).+?(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+).+?(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+).+?(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+).+?(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"

Here is what a log looks like

{"publish_time": 1548432475.596, "data": {"insertId": "xueeeeees", "receiveTimestamp": "2019-01-21T15:51:31.311111132Z", "logName": "projects/nitro-mobile/logs/boot", "labels": {"compute.awsapis.com/resource_name": "-------", "container.awsapis.com/pod_name": "-----------", "container.awsapis.com/stream": "----", "container.awsapis.com/namespace_name": "---------"}, "jsonPayload": {"thread": "http-nitro", "timestamp": "1548432475.596", "message": "payload=2019/01/01  StartEvent {\n  \"appDetails\" : {\n    \"version\" : \"1\",\n    \"number\" : \"1\",\n    \"ver\" : \"11.1.0\",\n    \"release\" : \"11.1.1\",\n    \"storeId\" : \"0120\",\n    \"terminalId\" : \"41\"\n  },\n  \"capTime\" : \"2019-01-01 10:59:48\",\n  \"MSR\" : {\n    \"MSRReach\" : \"Scanner not connected\",\n    \"version\" : \"Not available\",\n    \"isConnected\" : true,\n    \"SB\" : \"\"\n  },\n  \"isProduction\" : true,\n  \"pinpad\" : {\n    \"serialNumber\" : \"111-112-111-111\",\n    \"OSVersion\" : \"AWS12343\",\n    \"model\" : \"910X5\"\n  },\n  \"MDM\" : {\n    \"lastSeen\" : \"Not Available\"\n  },\n  \"device\" : {\n    \"Serial\" : \"BV12AXAJ013J\",\n    \"battery\" : \"100.0\",\n    \"network\" : \"ReachableViaWiFi\",\n    \"osVersion\" : \"11.0.1\",\n    \"memory\" : \"100\"\n  }\n} EndEvent\n"

Any ideas to improve efficiency?

Tags (2)
0 Karma
1 Solution

JoshuaJohn
Contributor

Well this is ugly, but it works.

| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)" 
| rex "(?:\\\"ver\\\\\\\" : \\\\\\\")(?<Ver>[^\\\\]+)" 
| rex "(?:\\\"storeId\\\\\\\" : \\\\\\\")(?P<StoreNum>[^\\\\]+)" 
| rex "(?:\\\"MSRReach\\\\\\\" : \\\\\\\")(?<ScannerConnection>[^\\\\]+)" 
| rex "(?:\\\"isConnected\\\\\\\" : )(?<ConnectedStatus>[^,]+)" 
| rex "(?:\\\"isProduction\\\\\\\" : )(?<Prod>[^,]+)" 
| rex "(?:\\\"serialNumber\\\\\\\" : \\\\\\\")(?<ScannerSerialNum>[^\\\\]+)" 
| rex "(?:\\\"OSVersion\\\\\\\" : \\\\\\\")(?<ScannerOSVersion>[^\\\\]+)" 
| rex "(?:\\\"model\\\\\\\" : \\\\\\\")(?<ScannerModel>[^\\\\]+)" 
| rex "(?:\\\"Serial\\\\\\\" : \\\\\\\")(?<iPadSerial>[^\\\\]+)" 
| rex "(?:\\\"battery\\\\\\\" : \\\\\\\")(?<iPadBattery>[^\\\\]+)" 
| rex "(?:\\\"osVersion\\\\\\\" : \\\\\\\")(?<iPadOSVersion>[^\\\\]+)"

View solution in original post

woodcock
Esteemed Legend

Have you tried piping your events through | kv or | spath? You will be surprised at how much gets done well enough!

JoshuaJohn
Contributor

I haven't, but I will look into it. Looks promising especially for my data.

0 Karma

JoshuaJohn
Contributor

Well this is ugly, but it works.

| rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)" 
| rex "(?:\\\"ver\\\\\\\" : \\\\\\\")(?<Ver>[^\\\\]+)" 
| rex "(?:\\\"storeId\\\\\\\" : \\\\\\\")(?P<StoreNum>[^\\\\]+)" 
| rex "(?:\\\"MSRReach\\\\\\\" : \\\\\\\")(?<ScannerConnection>[^\\\\]+)" 
| rex "(?:\\\"isConnected\\\\\\\" : )(?<ConnectedStatus>[^,]+)" 
| rex "(?:\\\"isProduction\\\\\\\" : )(?<Prod>[^,]+)" 
| rex "(?:\\\"serialNumber\\\\\\\" : \\\\\\\")(?<ScannerSerialNum>[^\\\\]+)" 
| rex "(?:\\\"OSVersion\\\\\\\" : \\\\\\\")(?<ScannerOSVersion>[^\\\\]+)" 
| rex "(?:\\\"model\\\\\\\" : \\\\\\\")(?<ScannerModel>[^\\\\]+)" 
| rex "(?:\\\"Serial\\\\\\\" : \\\\\\\")(?<iPadSerial>[^\\\\]+)" 
| rex "(?:\\\"battery\\\\\\\" : \\\\\\\")(?<iPadBattery>[^\\\\]+)" 
| rex "(?:\\\"osVersion\\\\\\\" : \\\\\\\")(?<iPadOSVersion>[^\\\\]+)"
Get Updates on the Splunk Community!

Maximize the Value from Microsoft Defender with Splunk

<P style=" text-align: center; "><span class="lia-inline-image-display-wrapper lia-image-align-center" ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

<FONT size="5"><FONT size="5" color="#FF00FF">Get the latest news and updates from the Splunk Community ...