How to extract all values for a single field using rex?

[harshal_chakran]

Hi,
I have a log file from which I am trying to extract a value of the specific term "Security ID".
My data is divided in two events, as follows:

EVENT 1:

waterfall:
    Security ID:     NULL SID
    Data Language:   -
    Data Syntax:     -
    Data ID:         0x0

Data Type:         3

New Interface:
    Security ID:     QWERTY\ts123654
    Data Language:   ts123654
    Data Syntax:     QWERTY
    Data ID:         0x17r2627u8
    Data GUID:     {00000000-0000-0000-0000-000000000000}

EVENT 2 :

waterfall:
    Security ID:      ASDFGH\ts654321
    Data Language:  ts654321
    Data Syntax:      ASDFGH
    Data ID:          0x17r2612323
    Data GUID:      {00000000-0000-0000-0000-000000000000}

I want to extract the values of the term "Security ID" from the logs. To display the data in following manner:

NULL SID
QWERTY\ts123654
ASDFGH\ts654321

I have used the field extractor utility of Splunk, but not able to capture all the Security ID's.
Please Help...!!!

[wpreston]

Try this to see if it works:

... search terms here ... | rex "Security\sID:\s(?<Security_ID>.*)\sData\sLanguage"

If so, you can add the regular expression into your props.conf file to extract the field automatically.

[wpreston]

No problem, happy to help!

[KindaWorking]

There are a couple of things that will not work for this. I believe the regular expression you are looking for is something like:

Security\sID:\s+(?<SecurityID>.*)\n

There is quite a bit of whitespace between Security ID: and the data he is hoping to grab. The thing that I do not know how to do (and am super keen to know how it can be done) is how to extract multiple values of the same field from a single event.

[wpreston]

Getting past the extra white space is easy enough with a slightly modified regex (the extra white space and current formatting of the events with line breaks was not in the original post).

To extract multiple values of the same field from a single event, you need to add your extraction to transforms.conf and add MV_ADD = True, then either create a new report stanza or add to an existing report stanza in props.conf for the host, source, or sourcetype that the field is associated with. For this example, I'll use a sourcetype of 'waterfall':

transforms.conf

[Security_ID_Extraction]
REGEX = Security\sID:\s+(?<SecurityID>.*)\n
MV_ADD = True

props.conf

[waterfall]
REPORT-waterfall_fields = Security_ID_Extraction

[KindaWorking]

Cool, thanks for that wpreston. I know I did not ask the question but I had the exact same question I was going to ask.