topic Re: Using Rex to extract string from event for table in Splunk Search

Using Rex to extract string from event for table

vil505 — Mon, 06 Jun 2016 17:24:53 GMT

Hi,

I'm sure this is very simple, but I'm fairly new to regex and rex.

I'm trying to use rex to extract a string from the event logs, and then show that sring in a table.

Here is a sample event:

"2016-06-06 12:14:11,114 [RMI TCP Connection(453679)-10.128.110.184]- Remote invocation of " and here would be specifics.

Using the field extractor, I came up with the following:

rex field=_raw "(?ms)^(?:[^ \\n]* ){5}(?P<Remote_Invocation>\\w+\\s+\\w+\\s+)"

This allowed me to use the following search to table it:

index=qp_mds source="/app/logdata/logs/marketdata/performance.log"| rex field=_raw "(?ms)^(?:[^ \\n]* ){5}(?P<Remote_Invocation>\\w+\\s+\\w+\\s+)"| Table Remote_Invocation

However, all my table shows is Remote Invocation where the event should be. I can get the string if I table _raw, but it give me the whole string, whereas I only want what is after Remote invocation. Any help would be greatly appreciated.

Thank you.

Re: Using Rex to extract string from event for table

richgalloway — Mon, 06 Jun 2016 17:31:47 GMT

The field extractor can be hit-or-miss. Sometimes it's too specific. Try this:

index=qp_mds source="/app/logdata/logs/marketdata/performance.log"| rex "Remote invocation of (?<Remote_Invocation>.*)" | Table Remote_Invocation

Re: Using Rex to extract string from event for table

vil505 — Mon, 06 Jun 2016 17:38:27 GMT

No, that didn't work. Now the just shows up blank.

Re: Using Rex to extract string from event for table

Richfez — Mon, 06 Jun 2016 17:42:52 GMT

Can you paste in an event without the obfuscation (or at least with far far less obfuscation)?

You could try a minor variant of richgalloway's answer in case there are differences in spacing:

index=qp_mds source="/app/logdata/logs/marketdata/performance.log"| rex "Remote\s+invocation\s+of\s+(?<Remote_Invocation>.*)" | Table Remote_Invocation

How many events contain the "Remote invocation" string? You could try limited your search to only those events up front, in case it's just that it's in such a small percentage it doesn't show up all over:

index=qp_mds source="/app/logdata/logs/marketdata/performance.log" "Remote invocation"| rex "Remote\s+invocation\s+of\s+(?<Remote_Invocation>.*)" | Table Remote_Invocation

Re: Using Rex to extract string from event for table

richgalloway — Mon, 06 Jun 2016 17:45:15 GMT

This run-anywhere example works in my environment.

| metadata type=sourcetypes | head 1 | eval string="2016-06-06 12:14:11,114 [RMI TCP Connection(453679)-10.128.110.184]- Remote invocation of and here would be specifics." | rex field=string "Remote invocation of (?<Remote_Invocation>.*)" | Table Remote_Invocation

Re: Using Rex to extract string from event for table

woodcock — Mon, 06 Jun 2016 17:46:06 GMT

Like this:

... | rex "Remote\s+Invocation\s+of\s+(?<Remote_Invocation>.+)" | stats count by Remote_Invocation