topic Re: Issue with regex in Splunk Search

Issue with regex

bharpur183 — Mon, 02 Oct 2017 22:48:47 GMT

This is the event :

02OCT2017_16:46:47.212 130880:140149567481600 INFO event.py:177 root event = {"hopTrace": {"hops": [{"machine": {"nodeId": 569}, "application": {"processId": 19295, "processName": "udrqssvc.tsk", "appName": "DRQS"}, "authenticatedUser": {"uuid": 10095155}}]}, "event": {"eventType": "DRQS UPDATED", "drqsNumber": 107516809(FIELD5), "newHeader": {"status": "Q", "function": "N539", "billToId": 5028, "yellowKey": "", "billToType": "HIER", "lastUpdateTime": "2017-10-02T20:46:47.000+00:00", "type": "IW", "creatorUuid": 1603009, "slaCategory": -1, "summary": "MM/DD n539 hardware failure IBM PMR: 24465.L6Q.000", "queue": "", "timeClosed": "1899-12-31T05:00:00.000+00:00", "ouTypeCode": 0, "routeToGroup": 270(FIELD4), "ouTypeDescription": "", "tsCustomerNumber": 0, "closedUuid": 0, "lastUpdateUuid": 10095155, "createTime": "2017-09-29T12:00:48.000+00:00", "ownerUuid": 2984495}, "logNotes": [{"logNoteId": "1049598095", "timestamp": "2017-10-02T20:46:47.141+00:00", "authorUuid": 10095155, "logText": [{"text": "Note added from offline, remote machine 208\n", "textType": "DEEMPHASIZED"}, {"text": "{FIFW PRQS 160269881(FIELD6)} submitted to take N539(FIELD1) (N539) offline on Tue Oct 03 2017 19:00:00 GMT-0400 (EDT)(FIELD2) for HARDWARE REPAIRS(FIELD3)\n", "textType": "NORMAL"}], "isAutomated": true}]}, "metadata": {"publishId": "121785005", "publishTime": "2017-10-02T16:46:47.189-04:00"}}

From the above event I want to create a statistics table with Field1-Fileds 6
I have highlighted the needed fields as bold . I get some of them but not all 6 fields

Re: Issue with regex

sduff_splunk — Tue, 03 Oct 2017 00:22:23 GMT

| rex field=_raw "(?<field6>\S+) submitted to take (?<field1>\S+) .* offline on (?<field2>.*) for (?<field3>[^\"]+)\""

| rex field=_raw "\"drqsNumber\": (?<field5>\S+),"

| rex field=_raw "\"routeToGroup\": (?<field4>\S+),"

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 00:34:00 GMT

get an error :

Error in 'rex' command: Encountered the following error while compiling the regex '(?.*) for (?[^"]+)"': Regex: syntax error in subpattern name (missing terminator)

Re: Issue with regex

sduff_splunk — Tue, 03 Oct 2017 00:37:45 GMT

I've updated the query

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 00:39:47 GMT

Please check this rex query... (if the event got different words, then rex needs to be adjusted)
Updated - the base query

contextName=olliebot service_name=olliebot source="/bb/logs/vcon/olliebot.log.2017*" AND (PRQS AND submitted)
    | rex field=_raw ".*\"drqsNumber\"\:\s(?<Field5>\d+)\(FIELD5\).*\"routeToGroup\"\:\s(?<Field4>\d+)\(FIELD4\).*PRQS\s(?<Field6>\d+)\(FIELD6\).*take\s(?<Field1>\S+)\(FIELD1\).*offline\son\s(?<Field2>.*)\(FIELD2\)\sfor\s(?<Field3>.*)\(FIELD3\)"
    | table Field5 Field4 Field3 Field6 Field1 Field2 Field3 _raw

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 00:46:25 GMT

keep getting an error :

Unknown search command 'sourcetype'.

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 00:55:09 GMT

i uploaded the sample event on my splunk with the sourcetype as rexField.
for your environment, you have to write the base splunk query which will get the right events.

similar to this -
index=main source=fieldlogs sourcetype=logs host=hostname | remaining rex query

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 00:55:09 GMT

Thanks . That works . can you please provide the same for fields 1 and field 5 ?

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 01:03:40 GMT

yep. I have my regular query before the above rex you provided , but still gives me the unknown source type error

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 01:06:15 GMT

may i know your splunk search query please, when you run it, do you get the events similar to the one updated on the question

Re: Issue with regex

sduff_splunk — Tue, 03 Oct 2017 01:09:03 GMT

Updated. Please look at the rex command to learn how this command works.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Rex

http://docs.splunk.com/Documentation/Splunk/latest/Search/SPLandregularexpressions

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 01:16:18 GMT

this is my query

contextName=olliebot service_name=olliebot source="/bb/logs/vcon/olliebot.log.2017*" AND (PRQS AND submitted)

and yes the question above is the result of the above query

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 01:17:42 GMT

Please try this query and update me what happens -
contextName=olliebot service_name=olliebot source="/bb/logs/vcon/olliebot.log.2017*" AND (PRQS AND submitted) | rex field=_raw ".*\"drqsNumber\"\:\s(?\d+)\(FIELD5\).*\"routeToGroup\"\:\s(?\d+)\(FIELD4\).*PRQS\s(?\d+)\(FIELD6\).*take\s(?\S+)\(FIELD1\).*offline\son\s(?.*)\(FIELD2\)\sfor\s(?.*)\(FIELD3\)" | table Field5 Field4 Field3 Field6 Field1 Field2 Field3 _raw

Re: Issue with regex

bharpur183 — Tue, 29 Sep 2020 16:04:19 GMT

got this error now :

Error in 'rex' command: Encountered the following error while compiling the regex '."drqsNumber":\s(?\d+)(FIELD5)."routeToGroup":\s(?\d+)(FIELD4).PRQS\s(?\d+)(FIELD6).*take\s(?\S+)(FIELD1).*offline\son\s(?.)(FIELD2)\sfor\s(?.*)(FIELD3)': Regex: unrecognized character after (? or (?-

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 01:30:51 GMT

i have updated the Splunk Query on this answer post.. can you please try it and update me -

Re: Issue with regex

bharpur183 — Tue, 29 Sep 2020 16:04:22 GMT

same error :

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 01:36:56 GMT

ok, lets do step by step.. when you run this, do you get the Field5 results?

contextName=olliebot service_name=olliebot source="/bb/logs/vcon/olliebot.log.2017*" AND (PRQS AND submitted) | rex field=_raw ".*\"drqsNumber\"\:\s(?<Field5>\d+) | table _raw Field5

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 01:59:25 GMT

the above query is going on for the last 20 mins and its still going and producing no results

Re: Issue with regex

inventsekar — Tue, 03 Oct 2017 02:03:24 GMT

when you run this, how many events you get -
contextName=olliebot service_name=olliebot source="/bb/logs/vcon/olliebot.log.2017*" AND (PRQS AND submitted)

Re: Issue with regex

bharpur183 — Tue, 03 Oct 2017 02:07:26 GMT

ok the query finished and did yield field 5 correctly ( there are empty rows in between the results in the statistics table) but it did yield field 5