Splunk Search

Extracting fields in payload xml multiline log

changux
Builder

Hi all.

I need help setting an input and extracting multiline fields with one entry like this:

####<May 2, 2015 23:37:26 PM PCT> <Warning> <TGFG Logging> <host> <source> <[ACTIVE] ExecuteThread: '26' for queue: 'jboss.kernel.Default (self-tuning)'> <<user>> <> <y7433edf553abf7c3:-12a453ef2:148c05609dd:-5000-000000000000342> <54564444> <JB-000000> < [FILE34, null, null, REQUEST] <!--- Input BD Profiles -->: <soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <ns:RunService xmlns:ns="http://mysite.com/s3" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <ns:data>
      <con:java-content ref="jcid:-12ac44f2:35t54ed33:-4ea8" xmlns:con="http://www.jobss.com/wli/sb/context"/>
    </ns:data>
  </ns:RunService>
</soapenv:Body>>

Useful info:

  • All the events begin with the same:

  • I need to extract the field "Status", in the example is the single word between the < > after the timestamp (<Warning>).

  • Also, i need to extract the <host>

  • I need to extract the <source>

  • Everything left must be called "Payload".

Anyone can help me?

Thanks!

0 Karma
1 Solution

woodcock
Esteemed Legend

Like I said, fix your linebreaking and timestampinginprops.conf` first or you'll never get anything off the ground but once you do, this will work:

| rex "^####<[^>]+>\s*<(?<Status>[^>]+)>\s*<[^>]+>\s*<(?<Host>[^>]+)>\s*<(?<Source>[^>]+)>\s*(?<Payload>.*)$"

View solution in original post

woodcock
Esteemed Legend

Like I said, fix your linebreaking and timestampinginprops.conf` first or you'll never get anything off the ground but once you do, this will work:

| rex "^####<[^>]+>\s*<(?<Status>[^>]+)>\s*<[^>]+>\s*<(?<Host>[^>]+)>\s*<(?<Source>[^>]+)>\s*(?<Payload>.*)$"

changux
Builder

You rock! thanks!

0 Karma

changux
Builder

Works very nice. How i can extract only the "payload"?

Thank you so much!

0 Karma

woodcock
Esteemed Legend
| rex "^####<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*(?<Payload>.*)$"

Don't forget to click "Accept".

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

If they're already editing props.conf, why would you suggest they use rex instead of adding an EXTRACT ?

0 Karma

lcrielaa
Communicator

| rex field=_raw "####<[^>]+>\s<(?[^>]+)>\s<[^>]+>\s<(?[^>]+)>\s<(?[^>]+)>\s(?.*)"

That'll do the extracting you want but like the other said, you'll need to fix your linebreaking and timestamping via your props.conf.

0 Karma

woodcock
Esteemed Legend

All that you are asking is possible and not too difficult but you are WAY ahead of yourself. You need to get timestamping and linebreaking working first. Are you working on that?

changux
Builder

Yes, but doesn't work properly.

0 Karma

woodcock
Esteemed Legend

Do you have anything at all coming into Splunk yet? In other words, do you have inputs.conf working and timestamp recognition working?

0 Karma

changux
Builder

Not for now. My test extracts fields with the symbols ( <> ), not only the content 😞

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...