topic Re: Regex : Extract text after occurrence of a specific recurring character in Splunk Search

Regex : Extract text after occurrence of a specific recurring character

Sukisen1981 — Tue, 19 Jun 2018 13:14:04 GMT

I have events like this -
[2018-03-30 13:45:51,515] [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] [INFO] [CustomerImpl] [6fbfb1ab-c947-49e9-964d-761390208a3b] Returning getCustomer for customer
and
[2018-03-30 13:45:51,515] [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] [INFO] [CometHeaderInboundRef] [] Read from inbound Header - UUID=6fbfb1ab-c947-49e9-964d-761390208a3b Username=maxuser

Now, ask is to get everything using regex after the last square bracket till end of event, so first event should return- Returning getCustomer for customer and second event should return - Read from inbound Header - UUID=6fbfb1ab-c947-49e9-964d-761390208a3b Username=maxuser

Re: Regex : Extract text after occurrence of a specific recurring character

mayurr98 — Tue, 19 Jun 2018 13:25:24 GMT

Try this

<your base search> | rex field=_raw ".*\]\s+(?<field>.*)"

Re: Regex : Extract text after occurrence of a specific recurring character

Sukisen1981 — Tue, 19 Jun 2018 13:34:57 GMT

Hi - Yes I tried this before. It looked simple enough to me at the beginning , just escape everything before the last square bracket and fetch till event end. Ideally this should have worked in a jiffy, but I am receiving empty results.
Point to note - When I manually copied these lines into a notepad (to check if my rgex is correct) and ran our same query above , I got the results. This event set however is a live log sent by a forwarder and it does not work on the live data.
If our regex is correct, how come it works as a notepad source and not live logging? Is data source/way it is being indexed through forwarder an issue ? I am slightly worried

Re: Regex : Extract text after occurrence of a specific recurring character

soumyasaha25 — Tue, 19 Jun 2018 13:38:14 GMT

try this
<your base search> | rex field=_raw ".*\]\s*(?<some_field>[^\n\r]+)"

Re: Regex : Extract text after occurrence of a specific recurring character

Sukisen1981 — Tue, 19 Jun 2018 13:53:28 GMT

Hi - once again, same result as above, meaning If i manually use this as a notepad data source , i receive the correct results but not from the live logs....
I am not that good in rgex, as per me our first regex above should have worked..what is the difference between that regex and yours?

Re: Regex : Extract text after occurrence of a specific recurring character

soumyasaha25 — Tue, 19 Jun 2018 15:25:25 GMT

can you post the query that you are running in Splunk (if required please mask sensitive data).

Re: Regex : Extract text after occurrence of a specific recurring character

Sukisen1981 — Sat, 23 Jun 2018 08:44:42 GMT

Hi, there was an issue with the data indexing. I don't know what it was but there were multiple complaints from many users , some were not able to view the auto extracted fields as well. Yesterday, we re-indexed the whole data. Now, both the regexes work. I am going to accept @mayurr98 's answer and up vote @soumyasaha25 's comment.
I do not think there was anything wrong in our approach, meanwhile - the curious case of the data indexing is being investigated by the splunk admin team 🙂