topic Re: REX with a variable rather than a string in Splunk Search

REX with a variable rather than a string

PhilAndreotti — Fri, 22 Aug 2014 11:51:39 GMT

I have a large chunk of raw data from one of my servers and am trying to filter the data down using a multiple REX statements.

From the data, I run a REX to return my first value, for example:

mysearch | rex"ProcessID:(?<PROCESS>)"

This works fine and I can see that the value is correct. I would then like to use the value of PROCESS in a further REX statement:

searchAsAbovve | rexAsAbove | rex "<PROCESS>(?<CPU>)"

I have tried multiple variations but I believe the PROCESS value is being input as a string rather than taking the value extracted from the previous REX.

Can anyone advise if there is a way to tell Splunk that the second REX is using a variable rather than a string? I have also tried using EVAL, for example eval "ProcID"=PROCESS then using the eval field in my REX statement but appear to get the same result.

Perhaps there is a better way of achieving this.

Thanks in advance

Re: REX with a variable rather than a string

martin_mueller — Fri, 22 Aug 2014 11:53:21 GMT

Answering regular-expression-based questions without a look at the (anonymized?) raw data is nigh-on impossible, even for us 😛

Using a field value probably isn't going to work, but there might be an entirely different way to solve the issue.

Re: REX with a variable rather than a string

PhilAndreotti — Fri, 22 Aug 2014 12:18:12 GMT

Thanks Martin, good point and I'll try to elaborate with the extract below:

SNMPv2-SMI::mib-2."25.4.2.1.2.1127" = "upstart-socket-" SNMPv2-SMI::mib-2."25.5.1.1.2.1127" = "388" SNMPv2-SMI::mib-2."25.4.2.1.2.1134" = "rsyslogd" SNMPv2-SMI::mib-2."25.5.1.1.2.1134" = "1576" SNMPv2-SMI::mib-2."25.4.2.1.2.1173" = "dbus-daemon" SNMPv2-SMI::mib-2."25.5.1.1.2.1173" = "988" SNMPv2-SMI::mib-2."25.4.2.1.2.1271" = "kworker/9:2" SNMPv2-SMI::mib-2."25.5.1.1.2.1271" = "0" SNMPv2-SMI::mib-2."25.4.2.1.2.1320" = "sshd" SNMPv2-SMI::mib-2."25.5.1.1.2.1320" = "2928"
Next comment will have details due to char limit

Re: REX with a variable rather than a string

PhilAndreotti — Fri, 22 Aug 2014 12:22:47 GMT

The extract is an SNMP query from a server. It returns the process ID's and the RAM used for each. I am using an initial REX to identify the PID, then I need to use the PID to identify the RAM used. i.e.

REX to identify the PID of rsyslogd would = 1134.

I then use EVAL or direct in to another REX to add the PID to the following SNMP query to determine the number that is returned after SNMPv2-SMI::mib-2."25.5.1.1.2.1134" = " which is 1576. This is the RAM used for this particular process.

As the PID will change on each boot, I need to search for the unique PID every time for accuracy.

Re: REX with a variable rather than a string

martin_mueller — Fri, 22 Aug 2014 12:31:45 GMT

Here's a thought:

... | rex "(?<PID>\d+)\" = \"rsyslogd\"" | eval RAM = replace(_raw, ".*" + PID + "\" = \"(\d+)\".*", "\1") | ...

That'll extract the 1134 before "rsyslogd" and use that in the replace() to locate the number after the equals sign, replacing the entire string with just that number - effectively extracting the field.

Re: REX with a variable rather than a string

PhilAndreotti — Fri, 22 Aug 2014 13:06:57 GMT

This worked perfectly thank you. Now I'm just trying to understand exactly what is going on rather than just use the answer but I've done a few tests and it seems to be doing exactly what I need so thank you very much

Re: REX with a variable rather than a string

martin_mueller — Fri, 22 Aug 2014 14:37:02 GMT

The key here is to work around the apparent shortcoming of rex - static regex strings with no field value replacement - and achieve the same thing with eval that obviously can use field values.

The replace()'s regex matches the entire _raw event, replacing everything with the content of the first capturing group - the RAM usage.