topic Re: how to identify malformed regex ? in Splunk Search

how to identify malformed regex ?

tarunmalhotra79 — Mon, 31 May 2021 09:32:49 GMT

Dear Splunker,

I have a lookup, which contains all the regex patterns. I would like to identify which of them are malformed.

When i say malformed, I simply mean missing parenthesis or may be some mismatch in basic regular expressions.

For ex- ((hostA.*,hostB.*) // This will be malformed as its missing last parenthesis.

If i run this command in splunk

| regex "((hostA.*,hostB.*)"

It will give me the error , missing parenthesis and will immediately stop the execution , I am trying to find such kind of error proactively and correct them.

Any help would be really appreciated.

Thanks in advance

Re: how to identify malformed regex ?

richgalloway — Mon, 31 May 2021 15:58:13 GMT

Splunk doesn't have a command to preprocess regular expressions. Consider writing an external command that does so. Or just upload your lookup to an on-line regex checker. Or write a Python script to check the regexes.

Re: how to identify malformed regex ?

venkatasri — Tue, 01 Jun 2021 00:34:51 GMT

Hi @tarunmalhotra79

Found some log events related to rex, you can give a try if it helps.

index=_internal sourcetype=splunkd rex* OR regex* (ERROR OR WARN)

-----------------------------------------------------

An upvote would be appreciated if it helps!

Re: how to identify malformed regex ?

tarunmalhotra79 — Tue, 01 Jun 2021 06:26:11 GMT

Thanks Venkatasri for you answer.

I simply have a lookup which contains list of patterns some are malformed and some are good, the idea is to process then using regex and identify if some of then are malformed or not.

In Order to check from the _internal I need to first run it

Like - regex "My Pattern" and then i can see there is an error.

But in my case I have a lookup which contains around 300 pattern or so and I need to check if any of those are malformed or not ?I am not sure how to run 300 pattern present in some lookup in one go?

Again, thanks for your response and let me know if you have any queries around my question.

Re: how to identify malformed regex ?

tarunmalhotra79 — Tue, 01 Jun 2021 06:33:21 GMT

Thanks richgalloway for you response.

As of now, it looks like I have to go through the external option (using script) if it is not much time taking or else may be I have to do it manually then.

Re: how to identify malformed regex ?

venkatasri — Tue, 01 Jun 2021 07:14:16 GMT

Hi @tarunmalhotra79

You can automate it using Splunk search API's tried as described here got following error.

At a high level you have to iterate through the CSV regex, and execute Splunk search/job API which returns a SID, further passing on SID to results API will provide error/success output.

Alternatively you can reuse py script already written here and improvise as per your need - Creating searches using the REST API - Splunk Documentation

splunk@splunkbox-3:17:03:36:~/bin $ curl -u admin:p*****wrd -k https://localhost:8089/services/search/jobs -d search="| regex \"((hostA.*,hostB.*)\"" <?xml version="1.0" encoding="UTF-8"?> <response> <sid>1622531169.5</sid> </response> splunk@splunkbox-3:17:06:09:~/bin $ curl -u admin:p*****wrd -k https://localhost:8089/services/search/jobs/1622531169.5/results/ --get -d output_mode=csv <?xml version="1.0" encoding="UTF-8"?> <response> <messages> <msg type="FATAL">Error in 'SearchOperator:regex': The regex '((hostA.*,hostB.*)' is invalid. Regex: missing closing parenthesis.</msg> </messages> </response>

----------

An upvote would be appreciated if it helps!