topic How to fetch values with regex which is inside double backslash in Splunk Search

How to fetch values with regex which is inside double backslash

rkishoreqa — Fri, 17 Sep 2021 17:07:33 GMT

Can anyone please help me to create the regex expression for the below log.

> {\\n \\\"process\\\": \\\"get_input\\\",\\n \\\"totalProcessed\\\": \\\"0\\\",\\n \\\"SuccessfullyProcessed\\\": \\\"0\\\",\\n \\\"FailedToProcess\\\": \\\"0\\\",\\n \\\"FileName\\\": \\\"\\\"\\n}

I created the regex for this as below, but for the 'FileName' I am getting '\n'.

> | rex field=_raw "process\W+(?<process>[\w\s]+)" | rex field=_raw "totalProcessed\W+(?<totalProcessed>[\w\s]+)"| rex field=_raw "SuccessfullyProcessed\W+(?<SuccessfullyProcessed>[\w\s]+)" | rex field=_raw "FileName\W+(?<FileName>[\w\s]+)" | rex field=_raw "FailedToProcess\W+(?<FailedToProcess>[\w\s]+)"

It seems some modification/rebuild the regex is needed. Please help me on this.

Thanks in advance.

Re: How to fetch values with regex which is inside double backslash

richgalloway — Fri, 17 Sep 2021 18:28:52 GMT

What exactly are you trying to extract? Do you want to keep or exclude the backslashes?

Re: How to fetch values with regex which is inside double backslash

ashvinpandey — Fri, 17 Sep 2021 18:40:32 GMT

@rkishoreqa Please use the below regex and let me know if it worked.

| regex field=_raw "process.*?\".*?\"(?P<process>\w+)" | regex field=_raw "totalProcessed.*?\".*?\"(?P<totalProcessed>\d+)" | regex field=_raw "SuccessfullyProcessed.*?\".*?\"(?P<SuccessfullyProcessed>\d+)" | regex field=_raw "FailedToProcess.*?\".*?\"(?P<FailedToProcess>\d+)" | regex field=_raw "FileName.*?\".*?\"(?P<FileName>.*?)\\"

Also, If this reply helps you, an upvote would be appreciated.

Re: How to fetch values with regex which is inside double backslash

rkishoreqa — Mon, 20 Sep 2021 12:23:31 GMT

@richgalloway I need the below highlighted values. The values from the key pair, but need to escape \\\.

> {\\n \\\"process\\\": \\\"get_input\\\",\\n \\\"totalProcessed\\\": \\\"0\\\",\\n \\\"SuccessfullyProcessed\\\": \\\"0\\\",\\n \\\"FailedToProcess\\\": \\\"0\\\",\\n \\\"FileName\\\": \\\"<<>>\\\"\\n}

Re: How to fetch values with regex which is inside double backslash

ro_mc — Mon, 20 Sep 2021 14:38:08 GMT

Instead of modifying the regex, you need to fix the source data.

This could be done using a main script and a helper script.

I.e. each script could read in the input, and naturally strip out the escape characters in the output.

It looks like the current source data has been double-escaped, so " has become \" and then \\\". As such, you may need a second script to read in output from the first script, which can then be used as the source for ingest into Splunk.

Re: How to fetch values with regex which is inside double backslash

rkishoreqa — Wed, 22 Sep 2021 16:18:56 GMT

can anyone help me on this, waiting for resolution.

Re: How to fetch values with regex which is inside double backslash

richgalloway — Wed, 22 Sep 2021 16:45:04 GMT

Did you try the rex commands offered by @ashvinpandey ? If so, please reply to his answer with the results so we can try to resolve the issue.

Re: How to fetch values with regex which is inside double backslash

rkishoreqa — Wed, 22 Sep 2021 17:47:42 GMT

I am getting the below error when I tried with rex.

> Error in 'rex' command: Encountered the following error while compiling the regex 'FileName:*?".*?"(?P<FileName>.*?)\': Regex: \ at end of pattern.

After removing the prefix '\\' as below, then I am getting the 'FileName' as blank.

> | rex field=_raw "FileName.*?\".*?\"(?P<FileName>.*?)"

Re: How to fetch values with regex which is inside double backslash

ro_mc — Wed, 22 Sep 2021 18:09:24 GMT

This is the expected result, as the filename is blank in the example provided.

| regex field=_raw "FileName.*?\".*?\"(?P<FileName>.*?)\\"

To overcome the issue of the trailing backslashes, you can try either escaping the resolved backslash, so the last portion would read as

(?P<FileName>.*?)\\\" or (?P<FileName>.*?)\\\\"

An alternative is to avoid wildcard matches, such as matching on characters that aren't backslashes. The following should work, but it depends on how Splunk interprets the backslash character in this context:

(?P<FileName>[^\]*)"

Finally, note that if the filenames themselves contain backslashes, this will break the regex, which is why it's better to remove them at the time of data ingest, allowing you to capture legitimate data without errors, as well as process any escaping that Splunk might (probably will) apply to the file path backslashes.