Splunk Search

Extracting fields in payload xml multiline log

Builder

Hi all.

I need help setting an input and extracting multiline fields with one entry like this:

####<May 2, 2015 23:37:26 PM PCT> <Warning> <TGFG Logging> <host> <source> <[ACTIVE] ExecuteThread: '26' for queue: 'jboss.kernel.Default (self-tuning)'> <<user>> <> <y7433edf553abf7c3:-12a453ef2:148c05609dd:-5000-000000000000342> <54564444> <JB-000000> < [FILE34, null, null, REQUEST] <!--- Input BD Profiles -->: <soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <ns:RunService xmlns:ns="http://mysite.com/s3" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <ns:data>
      <con:java-content ref="jcid:-12ac44f2:35t54ed33:-4ea8" xmlns:con="http://www.jobss.com/wli/sb/context"/>
    </ns:data>
  </ns:RunService>
</soapenv:Body>>

Useful info:

  • All the events begin with the same:

  • I need to extract the field "Status", in the example is the single word between the < > after the timestamp (<Warning>).

  • Also, i need to extract the <host>

  • I need to extract the <source>

  • Everything left must be called "Payload".

Anyone can help me?

Thanks!

0 Karma
1 Solution

Esteemed Legend

Like I said, fix your linebreaking and timestampinginprops.conf` first or you'll never get anything off the ground but once you do, this will work:

| rex "^####<[^>]+>\s*<(?<Status>[^>]+)>\s*<[^>]+>\s*<(?<Host>[^>]+)>\s*<(?<Source>[^>]+)>\s*(?<Payload>.*)$"

View solution in original post

Esteemed Legend

Like I said, fix your linebreaking and timestampinginprops.conf` first or you'll never get anything off the ground but once you do, this will work:

| rex "^####<[^>]+>\s*<(?<Status>[^>]+)>\s*<[^>]+>\s*<(?<Host>[^>]+)>\s*<(?<Source>[^>]+)>\s*(?<Payload>.*)$"

View solution in original post

Builder

You rock! thanks!

0 Karma

Builder

Works very nice. How i can extract only the "payload"?

Thank you so much!

0 Karma

Esteemed Legend
| rex "^####<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*(?<Payload>.*)$"

Don't forget to click "Accept".

0 Karma

Splunk Employee
Splunk Employee

If they're already editing props.conf, why would you suggest they use rex instead of adding an EXTRACT ?

0 Karma

Communicator

| rex field=_raw "####<[^>]+>\s<(?[^>]+)>\s<[^>]+>\s<(?[^>]+)>\s<(?[^>]+)>\s(?.*)"

That'll do the extracting you want but like the other said, you'll need to fix your linebreaking and timestamping via your props.conf.

0 Karma

Esteemed Legend

All that you are asking is possible and not too difficult but you are WAY ahead of yourself. You need to get timestamping and linebreaking working first. Are you working on that?

Builder

Yes, but doesn't work properly.

0 Karma

Esteemed Legend

Do you have anything at all coming into Splunk yet? In other words, do you have inputs.conf working and timestamp recognition working?

0 Karma

Builder

Not for now. My test extracts fields with the symbols ( <> ), not only the content 😞

0 Karma