topic Re: How to extract from multiline events using regular expressions with variables? in Splunk Search

How to extract from multiline events using regular expressions with variables?

arisat — Tue, 24 Sep 2019 14:30:50 GMT

Hi,

I have a rather large multiline event which I am trying to extract data from. The problem is that the format is along the lines of:

key0 = "bob"
key1 = "foo"
key2 = "bar"
bob = blah $value0 blah
foo = $value1 blah
bar = $value2 blah blah

It's all rather annoyingly unstructured and arbitrary, so in order to find the values, I have to first extract the key1, key2 and key3 values, to know where to look for the values. I do this with rex, and it goes well. Then I go searching through the text for the values, again using regex.

The problem is that I somehow have to feed the results from the first search into the subsequent regex searches, and at least REX does not seem to like it if I type 'rex field=_raw "somethingsomething" + $key1 + "something" '
Apparently it's static text only, no variables welcome.
I did play around with the replace function in eval, but it does not seem to want to look beyond a single line.

Any ideas how I can solve this?

Best regards,

Arild

Re: How to extract from multiline events using regular expressions with variables?

jpolvino — Tue, 24 Sep 2019 14:52:55 GMT

Can you please add a little more detail? Do you have 1 large multi-line event that has both the initial search value and then other text near that value? Are there always exactly 3 "key" values?

A couple real (sanitized) examples would help a lot.

Re: How to extract from multiline events using regular expressions with variables?

Anantha123 — Tue, 24 Sep 2019 15:00:20 GMT

try this format

| rex field=key0 "blah (?<value0>[^/s]+)" | rex field=key1 "(?<value1>[^/s]+)" | rex field=key3 "(?<value2>[^/s]+)"

Re: How to extract from multiline events using regular expressions with variables?

arisat — Wed, 25 Sep 2019 07:15:35 GMT

Sure. I have a few pretty huge multi-line events, and it contains both the initial and subsequent search values. It's an SNMP file, so there might be good techniques for reading this that I am unaware of.

Here's a sanitized and reduced (for clarity) example:

.1.3.6.1.2.1.2.2.1.2.10044 = STRING: "FastEthernet0/12"
.1.3.6.1.2.1.2.2.1.2.10045 = STRING: "FastEthernet0/13"
.1.3.6.1.2.1.2.2.1.2.10046 = STRING: "FastEthernet0/14"
.1.3.6.1.2.1.2.2.1.2.10047 = STRING: "FastEthernet0/15"
.1.3.6.1.2.1.2.2.1.2.10048 = STRING: "FastEthernet0/16"
.1.3.6.1.2.1.2.2.1.2.10101 = STRING: "GigabitEthernet0/1"
.1.3.6.1.2.1.2.2.1.2.10102 = STRING: "GigabitEthernet0/2"
.1.3.6.1.2.1.2.2.1.2.10501 = STRING: "Null0"
[...]
.1.3.6.1.4.1.9.9.23.1.2.1.1.3.10101.1 = INTEGER: 1
.1.3.6.1.4.1.9.9.23.1.2.1.1.4.10101.1 = STRING: "
.1.3.6.1.4.1.9.9.23.1.2.1.1.5.10101.1 = STRING: "Cisco IOS Software"
.1.3.6.1.4.1.9.9.23.1.2.1.1.6.10101.1 = STRING: "cisco-switch-01.example.com"
.1.3.6.1.4.1.9.9.23.1.2.1.1.7.10101.1 = STRING: "GigabitEthernet0/3"
.1.3.6.1.4.1.9.9.23.1.2.1.1.8.10101.1 = STRING: "cisco WS-C2950D"

So, in this case, I can see from the lower section that this switch is connected to another switch called "cisco-switch-01.example.com". And the OID string tells me that the "reference" is "10101". So what I want to do is go back to the event and search for the ".1.3.6.1.2.1.2.2.1.2.10101"-string because I have found "10101". But I am struggling to do just that.

Is there a way to do:

| rex field=_raw ".1.3.6.1.2.1.2.2.1.2." + myReference  + ".*STRING: \"(?<myInterface>.*\""

Or something similar?