topic Can you help me with the following query using regex: missing terminating ] for character class in Splunk Search

Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn — Mon, 21 Jan 2019 19:57:50 GMT

So I tested this regex with regex101 and it seems to be working but Splunk doesn't seem to like it. Any ideas?

| rex field=_raw "^(?:{\"publish_time\": )(?P<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?P<Ver>[^\\]+)|(?:\"storeId\\\" : \\\")(?P<StoreNum>[^\\]+)|(?:\"MSRReach\\\" : \\\")(?P<ScannerConnection>[^\\]+)|(?:\"isConnected\\\" : )(?P<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?P<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?P<ScannerSerialNum>[^\\]+)|(?:\"OSVersion\\\" : \\\")(?P<ScannerOSVersion>[^\\]+)|(?:\"model\\\" : \\\")(?P<ScannerModel>[^\\]+)|(?:\"Serial\\\" : \\\")(?P<iPadSerial>[^\\]+)|(?:\"battery\\\" : \\\")(?P<iPadBattery>[^\\]+)|(?:\"osVersion\\\" : \\\")(?P<iPadOSVersion>[^\\]+)$"

Here is what the log looks like

{"publish_time": 1548432475.596, "data": {"insertId": "xueeeeees", "receiveTimestamp": "2019-01-21T15:51:31.311111132Z", "logName": "projects/nitro-mobile/logs/boot", "labels": {"compute.awsapis.com/resource_name": "-------", "container.awsapis.com/pod_name": "-----------", "container.awsapis.com/stream": "----", "container.awsapis.com/namespace_name": "---------"}, "jsonPayload": {"thread": "http-nitro", "timestamp": "1548432475.596", "message": "payload=2019/01/01  StartEvent {\n  \"appDetails\" : {\n    \"version\" : \"1\",\n    \"number\" : \"1\",\n    \"ver\" : \"11.1.0\",\n    \"release\" : \"11.1.1\",\n    \"storeId\" : \"0120\",\n    \"terminalId\" : \"41\"\n  },\n  \"capTime\" : \"2019-01-01 10:59:48\",\n  \"MSR\" : {\n    \"MSRReach\" : \"Scanner not connected\",\n    \"version\" : \"Not available\",\n    \"isConnected\" : true,\n    \"SB\" : \"\"\n  },\n  \"isProduction\" : true,\n  \"pinpad\" : {\n    \"serialNumber\" : \"111-112-111-111\",\n    \"OSVersion\" : \"AWS12343\",\n    \"model\" : \"910X5\"\n  },\n  \"MDM\" : {\n    \"lastSeen\" : \"Not Available\"\n  },\n  \"device\" : {\n    \"Serial\" : \"BV12AXAJ013J\",\n    \"battery\" : \"100.0\",\n    \"network\" : \"ReachableViaWiFi\",\n    \"osVersion\" : \"11.0.1\",\n    \"memory\" : \"100\"\n  }\n} EndEvent\n",

Thanks

Re: Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn — Mon, 21 Jan 2019 19:59:28 GMT

This is the error I keep getting: missing terminating ] for character class

Re: Can you help me with the following query using regex: missing terminating ] for character class

dkeck — Mon, 21 Jan 2019 20:35:00 GMT

HI,

normaly this error will tell you that slashes haven`t bin escaped correctly.

Please see the post of landen99 here https://answers.splunk.com/answers/29739/should-i-be-escaping-each-backslash-with-a-second-backslash-in-my-regex.html#answer-214500

in rex you have to escaped differently than in fieldextraction.

https://answers.splunk.com/answers/594708/what-is-wrong-with-my-regex-regex-missing-terminat.html

Hope that makes sense.

Re: Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn — Mon, 21 Jan 2019 20:49:43 GMT

Thats what I thought was the issue but whenever I add more backslashes I only get the results from my first capture

Re: Can you help me with the following query using regex: missing terminating ] for character class

richgalloway — Mon, 21 Jan 2019 22:29:21 GMT

If you get results by adding backslashes then you've fixed the error. The next step is to fix the regex to return the expected results. The reason you get only your first capture group back is because have put an OR operator (|) between each keyword/value pair. That tells regex to return the first matching pair.

If your field always appear in the same order, try this regular expression. It's much more efficient.

(?:{\"publish_time\": )(?P<PublishTime>[^,]+).*?(?:\"ver\\\" : \\\")(?P<Ver>[^\\]+).*?(?:\"storeId\\\" : \\\")(?P<StoreNum>[^\\]+).*?(?:\"MSRReach\\\" : \\\")(?P<ScannerConnection>[^\\]+).*?(?:\"isConnected\\\" : )(?P<ConnectedStatus>[^,]+).*?(?:\"isProduction\\\" : )(?P<Prod>[^,]+).*?(?:\"serialNumber\\\" : \\\")(?P<ScannerSerialNum>[^\\]+).*?(?:\"OSVersion\\\" : \\\")(?P<ScannerOSVersion>[^\\]+).*?(?:\"model\\\" : \\\")(?P<ScannerModel>[^\\]+).*?(?:\"Serial\\\" : \\\")(?P<iPadSerial>[^\\]+).*?(?:\"battery\\\" : \\\")(?P<iPadBattery>[^\\]+).*?(?:\"osVersion\\\" : \\\")(?P<iPadOSVersion>[^\\]+)

Re: Can you help me with the following query using regex: missing terminating ] for character class

woodcock — Tue, 22 Jan 2019 00:49:26 GMT

Try this:

... | rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+)|(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+)|(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+)|(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+)|(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+)|(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+)|(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+)|(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+)|(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"

Re: Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn — Tue, 22 Jan 2019 14:45:59 GMT

Unfortunately still getting the same issue with your regex statement

Re: Can you help me with the following query using regex: missing terminating ] for character class

woodcock — Tue, 22 Jan 2019 15:25:03 GMT

Try mine; I tested it in splunk.

Re: Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn — Tue, 22 Jan 2019 15:44:34 GMT

Sorry somehow missed it! The only issue is I need all the fields, and the pipe is only giving me the first field.

Re: Can you help me with the following query using regex: missing terminating ] for character class

woodcock — Tue, 22 Jan 2019 15:55:18 GMT

You are using | which means OR. In any case, the questions was asking about your syntax error and that seems to be fixed, right? So you should Accept and answer, UpVote anything helpful, and if you have a different problem now, post another question.

Re: Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn — Tue, 22 Jan 2019 15:58:06 GMT

Agreed. Thanks