topic Re: Extracting individual lines from a single field in Splunk Search

Extracting individual lines from a single field

erstexas — Tue, 25 Jun 2013 19:20:55 GMT

Hello,

I am working with Nessus data and I am trying to pull a software list from the results. Nessus exports this data in a single field:

The following software are installed on the remote host :
Cisco WebEx Meetings
OpenSSL-0.9.7c Binaries (GnuWin32)  [version 0.9.7c]
Python 2.5 elementtree-1.2.6-20050316
Python 2.5 pycrypto-2.0.1
Python 2.5 pywin32-211
McAfee VirusScan Enterprise  [version 8.7.0]  [installed on 2010/09/08]
Microsoft Visual C++ 2008 Redistributable - x86 9.0.30729.4148  [version 9.0.30729.4148] [installed on 2010/09/14]
Microsoft SQL Server 2008 Management Objects  [version 10.1.2531.0]  [installed on 2012/10/18]
ActiveWatchIse  [version 1.0.0]  [installed on 2010/09/14]
AWMonitorSvcSetup  [version 1.0.0]  [installed on 2010/09/14]
Python 2.5.2  [version 2.5.2150]  [installed on 2010/09/14]
Microsoft Visual C++ 2005 Redistributable  [version 8.0.61001]  [installed on 2012/12/16]
Microsoft Visual C++ 2008 Redistributable - x86 9.0.30729.6161  [version 9.0.30729.6161]  [installed on 2012/12/16]
Xiotech ISE Manager   [version 3.1.5.4]  [installed on 2012/10/18]
McAfee Agent  [version 4.5.0.1270]  [installed on 2011/05/02]
SQL Server System CLR Types  [version 10.1.2531.0]  [installed on 2012/10/18]
VMware Tools  [version 8.3.18.20074]  [installed on 2013/02/24]

I need to "table" each of the above lines with a matching Host/Software pair. I cannot figure out how to parse this field correctly.

I have tried:

rex field=Plugin_Output "(?< software>.*\n)"  (extra space only for display)
makemv delim="\n", Plugin_Output

Any assistance with this one?

Re: Extracting individual lines from a single field

kristian_kolb — Tue, 25 Jun 2013 20:38:36 GMT

Hi, I used some stacktrace to make this work. The first rex is just to create the multiline field, so you can skip that. The second rex pulls it apart into a multivalued field.

...| rex "(?m)\s+(?<stacktrace>(\s*at\s+.*\))+)" 
| rex field=stacktrace max_match=10 "(?<lines>.*)\n"

Hope this helps,

Kristian

Re: Extracting individual lines from a single field

adrianathome — Thu, 27 Jun 2013 02:48:35 GMT

Some ideas. Replace [ with | and ] with blank space using the rex command. Then create a multi-value field delimiting on the |.

Re: Extracting individual lines from a single field

erstexas — Thu, 27 Jun 2013 14:19:52 GMT

This pointed me in the right direction and I think I got it figured out now. Thanks!

Re: Extracting individual lines from a single field

erstexas — Thu, 27 Jun 2013 14:21:23 GMT

This looked promising but it was not working for me. "lines" (in your example) would not populate.

I did figure it out though this morning and will post my solution. Thanks!

Re: Extracting individual lines from a single field

erstexas — Thu, 27 Jun 2013 14:35:41 GMT

I got this working with the following query:

... | rex field=Plugin_Output mode=sed "s/\n/|/g" | makemv delim="|" Plugin_Output | mvexpand Plugin_Output | table Host, Plugin_Output

Looking back now, I could have been probably got it working just with the newline once I got the chain of commands correct, but the above works and I can fine-tune it from here.

Re: Extracting individual lines from a single field

erstexas — Thu, 27 Jun 2013 14:38:17 GMT

Solution below.

Re: Extracting individual lines from a single field

eashwar — Fri, 05 Jul 2013 09:43:59 GMT

hello, can you please explain me what (?m) is. thanks in advance

Re: Extracting individual lines from a single field

kristian_kolb — Thu, 11 Jul 2013 14:18:38 GMT

multi-line