Splunk Search

Can you help me with the following query using regex: missing terminating ] for character class

JoshuaJohn
Contributor

So I tested this regex with regex101 and it seems to be working but Splunk doesn't seem to like it. Any ideas?

| rex field=_raw "^(?:{\"publish_time\": )(?P<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?P<Ver>[^\\]+)|(?:\"storeId\\\" : \\\")(?P<StoreNum>[^\\]+)|(?:\"MSRReach\\\" : \\\")(?P<ScannerConnection>[^\\]+)|(?:\"isConnected\\\" : )(?P<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?P<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?P<ScannerSerialNum>[^\\]+)|(?:\"OSVersion\\\" : \\\")(?P<ScannerOSVersion>[^\\]+)|(?:\"model\\\" : \\\")(?P<ScannerModel>[^\\]+)|(?:\"Serial\\\" : \\\")(?P<iPadSerial>[^\\]+)|(?:\"battery\\\" : \\\")(?P<iPadBattery>[^\\]+)|(?:\"osVersion\\\" : \\\")(?P<iPadOSVersion>[^\\]+)$"

Here is what the log looks like

{"publish_time": 1548432475.596, "data": {"insertId": "xueeeeees", "receiveTimestamp": "2019-01-21T15:51:31.311111132Z", "logName": "projects/nitro-mobile/logs/boot", "labels": {"compute.awsapis.com/resource_name": "-------", "container.awsapis.com/pod_name": "-----------", "container.awsapis.com/stream": "----", "container.awsapis.com/namespace_name": "---------"}, "jsonPayload": {"thread": "http-nitro", "timestamp": "1548432475.596", "message": "payload=2019/01/01  StartEvent {\n  \"appDetails\" : {\n    \"version\" : \"1\",\n    \"number\" : \"1\",\n    \"ver\" : \"11.1.0\",\n    \"release\" : \"11.1.1\",\n    \"storeId\" : \"0120\",\n    \"terminalId\" : \"41\"\n  },\n  \"capTime\" : \"2019-01-01 10:59:48\",\n  \"MSR\" : {\n    \"MSRReach\" : \"Scanner not connected\",\n    \"version\" : \"Not available\",\n    \"isConnected\" : true,\n    \"SB\" : \"\"\n  },\n  \"isProduction\" : true,\n  \"pinpad\" : {\n    \"serialNumber\" : \"111-112-111-111\",\n    \"OSVersion\" : \"AWS12343\",\n    \"model\" : \"910X5\"\n  },\n  \"MDM\" : {\n    \"lastSeen\" : \"Not Available\"\n  },\n  \"device\" : {\n    \"Serial\" : \"BV12AXAJ013J\",\n    \"battery\" : \"100.0\",\n    \"network\" : \"ReachableViaWiFi\",\n    \"osVersion\" : \"11.0.1\",\n    \"memory\" : \"100\"\n  }\n} EndEvent\n", 

Thanks

Tags (2)
0 Karma
1 Solution

woodcock
Esteemed Legend

Try this:

... | rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+)|(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+)|(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+)|(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+)|(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+)|(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+)|(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+)|(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+)|(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"

View solution in original post

0 Karma

woodcock
Esteemed Legend

Try this:

... | rex "^(?:{\"publish_time\": )(?<PublishTime>[^,]+)?|(?:\"ver\\\" : \\\")(?<Ver>[^\\\]+)|(?:\"storeId\\\" : \\\")(?<StoreNum>[^\\\]+)|(?:\"MSRReach\\\" : \\\")(?<ScannerConnection>[^\\\]+)|(?:\"isConnected\\\" : )(?<ConnectedStatus>[^,]+)|(?:\"isProduction\\\" : )(?<Prod>[^,]+)|(?:\"serialNumber\\\" : \\\")(?<ScannerSerialNum>[^\\\]+)|(?:\"OSVersion\\\" : \\\")(?<ScannerOSVersion>[^\\\]+)|(?:\"model\\\" : \\\")(?<ScannerModel>[^\\\]+)|(?:\"Serial\\\" : \\\")(?<iPadSerial>[^\\\]+)|(?:\"battery\\\" : \\\")(?<iPadBattery>[^\\\]+)|(?:\"osVersion\\\" : \\\")(?<iPadOSVersion>[^\\\]+)$"
0 Karma

JoshuaJohn
Contributor

Sorry somehow missed it! The only issue is I need all the fields, and the pipe is only giving me the first field.

0 Karma

woodcock
Esteemed Legend

You are using | which means OR. In any case, the questions was asking about your syntax error and that seems to be fixed, right? So you should Accept and answer, UpVote anything helpful, and if you have a different problem now, post another question.

JoshuaJohn
Contributor

Agreed. Thanks

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you get results by adding backslashes then you've fixed the error. The next step is to fix the regex to return the expected results. The reason you get only your first capture group back is because have put an OR operator (|) between each keyword/value pair. That tells regex to return the first matching pair.

If your field always appear in the same order, try this regular expression. It's much more efficient.

(?:{\"publish_time\": )(?P<PublishTime>[^,]+).*?(?:\"ver\\\" : \\\")(?P<Ver>[^\\]+).*?(?:\"storeId\\\" : \\\")(?P<StoreNum>[^\\]+).*?(?:\"MSRReach\\\" : \\\")(?P<ScannerConnection>[^\\]+).*?(?:\"isConnected\\\" : )(?P<ConnectedStatus>[^,]+).*?(?:\"isProduction\\\" : )(?P<Prod>[^,]+).*?(?:\"serialNumber\\\" : \\\")(?P<ScannerSerialNum>[^\\]+).*?(?:\"OSVersion\\\" : \\\")(?P<ScannerOSVersion>[^\\]+).*?(?:\"model\\\" : \\\")(?P<ScannerModel>[^\\]+).*?(?:\"Serial\\\" : \\\")(?P<iPadSerial>[^\\]+).*?(?:\"battery\\\" : \\\")(?P<iPadBattery>[^\\]+).*?(?:\"osVersion\\\" : \\\")(?P<iPadOSVersion>[^\\]+)
---
If this reply helps you, an upvote would be appreciated.

JoshuaJohn
Contributor

Unfortunately still getting the same issue with your regex statement

0 Karma

woodcock
Esteemed Legend

Try mine; I tested it in splunk.

0 Karma

dkeck
Influencer

HI,

normaly this error will tell you that slashes haven`t bin escaped correctly.

Please see the post of landen99 here https://answers.splunk.com/answers/29739/should-i-be-escaping-each-backslash-with-a-second-backslash...

in rex you have to escaped differently than in fieldextraction.

https://answers.splunk.com/answers/594708/what-is-wrong-with-my-regex-regex-missing-terminat.html

Hope that makes sense.

JoshuaJohn
Contributor

Thats what I thought was the issue but whenever I add more backslashes I only get the results from my first capture

0 Karma

JoshuaJohn
Contributor

This is the error I keep getting: missing terminating ] for character class

0 Karma