Can you help me with the following query using regex: missing terminating ] for character class

[JoshuaJohn]

So I tested this regex with regex101 and it seems to be working but Splunk doesn't seem to like it. Any ideas?

| rex field=_raw "^(?:{\"publish_time\": )(?P<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?P<Ver>[^\\]+)|(?:\"storeId\\\" : \\\")(?P<StoreNum>[^\\]+)|(?:\"MSRReach\\\" : \\\")(?P<ScannerConnection>[^\\]+)|(?:\"isConnected\\\" : )(?P<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?P<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?P<ScannerSerialNum>[^\\]+)|(?:\"OSVersion\\\" : \\\")(?P<ScannerOSVersion>[^\\]+)|(?:\"model\\\" : \\\")(?P<ScannerModel>[^\\]+)|(?:\"Serial\\\" : \\\")(?P<iPadSerial>[^\\]+)|(?:\"battery\\\" : \\\")(?P<iPadBattery>[^\\]+)|(?:\"osVersion\\\" : \\\")(?P<iPadOSVersion>[^\\]+)$"

Here is what the log looks like

{"publish_time": 1548432475.596, "data": {"insertId": "xueeeeees", "receiveTimestamp": "2019-01-21T15:51:31.311111132Z", "logName": "projects/nitro-mobile/logs/boot", "labels": {"compute.awsapis.com/resource_name": "-------", "container.awsapis.com/pod_name": "-----------", "container.awsapis.com/stream": "----", "container.awsapis.com/namespace_name": "---------"}, "jsonPayload": {"thread": "http-nitro", "timestamp": "1548432475.596", "message": "payload=2019/01/01  StartEvent {\n  \"appDetails\" : {\n    \"version\" : \"1\",\n    \"number\" : \"1\",\n    \"ver\" : \"11.1.0\",\n    \"release\" : \"11.1.1\",\n    \"storeId\" : \"0120\",\n    \"terminalId\" : \"41\"\n  },\n  \"capTime\" : \"2019-01-01 10:59:48\",\n  \"MSR\" : {\n    \"MSRReach\" : \"Scanner not connected\",\n    \"version\" : \"Not available\",\n    \"isConnected\" : true,\n    \"SB\" : \"\"\n  },\n  \"isProduction\" : true,\n  \"pinpad\" : {\n    \"serialNumber\" : \"111-112-111-111\",\n    \"OSVersion\" : \"AWS12343\",\n    \"model\" : \"910X5\"\n  },\n  \"MDM\" : {\n    \"lastSeen\" : \"Not Available\"\n  },\n  \"device\" : {\n    \"Serial\" : \"BV12AXAJ013J\",\n    \"battery\" : \"100.0\",\n    \"network\" : \"ReachableViaWiFi\",\n    \"osVersion\" : \"11.0.1\",\n    \"memory\" : \"100\"\n  }\n} EndEvent\n", 

Thanks

[woodcock · Accepted solution]

Try this:

... | rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+)|(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+)|(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+)|(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+)|(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+)|(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+)|(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+)|(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+)|(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"

[JoshuaJohn]

Sorry somehow missed it! The only issue is I need all the fields, and the pipe is only giving me the first field.

[woodcock]

You are using | which means OR. In any case, the questions was asking about your syntax error and that seems to be fixed, right? So you should Accept and answer, UpVote anything helpful, and if you have a different problem now, post another question.

[JoshuaJohn]

Agreed. Thanks

[richgalloway]

If you get results by adding backslashes then you've fixed the error. The next step is to fix the regex to return the expected results. The reason you get only your first capture group back is because have put an OR operator (|) between each keyword/value pair. That tells regex to return the first matching pair.

If your field always appear in the same order, try this regular expression. It's much more efficient.

(?:{\"publish_time\": )(?P<PublishTime>[^,]+).*?(?:\"ver\\\" : \\\")(?P<Ver>[^\\]+).*?(?:\"storeId\\\" : \\\")(?P<StoreNum>[^\\]+).*?(?:\"MSRReach\\\" : \\\")(?P<ScannerConnection>[^\\]+).*?(?:\"isConnected\\\" : )(?P<ConnectedStatus>[^,]+).*?(?:\"isProduction\\\" : )(?P<Prod>[^,]+).*?(?:\"serialNumber\\\" : \\\")(?P<ScannerSerialNum>[^\\]+).*?(?:\"OSVersion\\\" : \\\")(?P<ScannerOSVersion>[^\\]+).*?(?:\"model\\\" : \\\")(?P<ScannerModel>[^\\]+).*?(?:\"Serial\\\" : \\\")(?P<iPadSerial>[^\\]+).*?(?:\"battery\\\" : \\\")(?P<iPadBattery>[^\\]+).*?(?:\"osVersion\\\" : \\\")(?P<iPadOSVersion>[^\\]+)

[JoshuaJohn]

Unfortunately still getting the same issue with your regex statement

[woodcock]

Try mine; I tested it in splunk.

[dkeck]

HI,

normaly this error will tell you that slashes haven`t bin escaped correctly.

Please see the post of landen99 here https://answers.splunk.com/answers/29739/should-i-be-escaping-each-backslash-with-a-second-backslash...

in rex you have to escaped differently than in fieldextraction.

https://answers.splunk.com/answers/594708/what-is-wrong-with-my-regex-regex-missing-terminat.html

Hope that makes sense.

[JoshuaJohn]

Thats what I thought was the issue but whenever I add more backslashes I only get the results from my first capture

[JoshuaJohn]

This is the error I keep getting: missing terminating ] for character class