Splunk Search

How do I write the regex to extract these values from my sample data?

splunker1981
Path Finder

Evening Splunk experts,

I am stuck trying to perform an extraction. I am using the built-in tool, but it keeps generating an incorrect regex. When I try to generate one myself, which works just fine using only regex tester, I get a message that no matches were found.

Below are sample logs from an in-house app and I am trying to create a field called gen_id and store the value to the right of the GENDID in the logs.

Nov 16 20:34:39 dtcSource01 dtc_data: detailStatus: Message finished GENID 399488 done
Nov 16 20:34:39 dtcSource01 dtc_data: detailStatus: MMR Message done RGENID 4 GENID 399488
Nov 16 20:34:39 dtcSource01 dtc_data: detailStatus: DSTC: shield GENID 399488
Nov 16 20:34:39 dtcSource01 dtc_data: detailStatus: MMR processing Begin RGENID 4 GENID 399488 to local shield
Nov 16 20:34:36 dtcSource01 dtc_data: detailStatus: GENID 399488 queued for processing
Nov 16 20:34:36 dtcSource01 dtc_data: detailStatus: DSTC: Delivering GENID 399488 to DSTC (skipping local)
Nov 16 20:34:36 dtcSource01 dtc_data: detailStatus: DSTC: Tagging GENID 399488 for shield from 192.x.x.x
Nov 16 20:19:14 dtcSource01 dtc_data: detailStatus: Begin GENID 568833 GID 289575
Nov 16 20:19:12 dtcSource01 dtc_data: detailStatus: GENID 568938 queued for local processing
Nov 16 20:19:12 dtcSource01 dtc_data: detailStatus: DSTC: Delivering GENID 568938 to DSTC (skipping work queue)
Nov 16 20:21:05 dtcSource01 dtc_data: detailStatus: GENID 398845 SubStatus 'working'
Nov 16 20:21:05 dtcSource01 dtc_data: detailStatus: DSTC: Delivering GENID 398845 to DSTC (skipping work queue)
Nov 16 20:21:05 dtcSource01 dtc_data: detailStatus: GENID 398845 queued for processing
Nov 16 20:19:17 dtcSource01 dtc_data: detailStatus: MMR processing Begin RGENID 4 GENID 568938 to local
Nov 16 20:19:14 dtcSource01 dtc_data: detailStatus: DSTC: Delivering GENID 568833 to DSTC (skipping local)
Nov 16 20:19:11 dtcSource01 dtc_data: detailStatus: DSTC: Delivering GENID 568899 to DSTC (skipping local)  
Nov 16 20:11:33 dtcSource01 dtc_data: detailStatus: DSTC: Tagging GENID 703728 for shield from 192.x.x.x

I tried the following commands, but they either don't capture what I need or capture data before or after the ID - not sure what I am doing wrong. A simple GENID\s\d+ should do it.

  | rex field=_raw "GENID (?<GENID>.*)"
  | rex field=_raw "GENID (?<GENID>\s\d+)"
  | rex field=_raw "GENID (?<GENID> \d+).*?"
  | (?P<FIELDNAME>\w+\s+\d+)

What I would like to have extracted are the following values - I want to be able to create a field called gen_id and store the values below.

    Fieldname         ID
    GENID            399488
    GENID            568833
    GENID            398845
    GENID            568833

Thanks for helping out.

0 Karma
1 Solution

sundareshr
Legend

Here's one way

GENID\s(?<gen_id>\d+)

View solution in original post

0 Karma

yulianaif
New Member

I hope this work. I use \s before GENID because there is value "RGENID" in your data.

| rex field=_raw "\sGENID\s(?<gen_id>\d+)"
0 Karma

sundareshr
Legend

Here's one way

GENID\s(?<gen_id>\d+)
0 Karma

splunker1981
Path Finder

Thanks for the reply, but I don't follow. I tried adding that to the regex generator to test and I get an error "Invalid regex: unexpected end of pattern"

0 Karma

sundareshr
Legend

That's strange. It missing the field name. Try this GENID\s(?\d+)

0 Karma

sundareshr
Legend

It's missing again. For some reason, the field name is getting dropped here. In the online documentation (http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/rex) for rex. look at the first example. There a "from" after the ? within angle brackets. That's what is missing. You need to add that, you may change "from" to whatever you would like to name the field. Hope this explains it

0 Karma

ppablo
Retired

Hi @sundareshr

I just edited your original answer to get the field name within angle brackets to show up properly. Looks like it has to be within a Code Sample box on its own line. I've filed a bug with angle brackets not being rendered properly on this site when only surrounded by backticks.

Cheers!

Patrick

0 Karma

sundareshr
Legend

@ppablo Thanks. I tried both backticks as well as code sample box on its own line. I have not been able to get those angle brackets to show these past couple days.

0 Karma
Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...