topic Re: Separate Fields in Splunk Search

Separate Fields

jaykay — Mon, 17 Dec 2012 21:45:08 GMT

Hi,

I'm new to splunk, so please excuse the basic question. I have some data in the following format:
Field1=abcdefg;Field2=12345;field3=98373
Field1=abcdefg&Field2=12345;field3=98373

Note the different separators. I can quite easily extract one of these using the following command:

... | rex field=_raw "Field1=(?<Field1>.*)Field2=(?<Field2>.*)"Field3=(?<Field3>.*)

I thought I could expand on this, in order to extract both of them at the same time, so I tried this, but it does not seem to work:

... | rex field=_raw "Field1[;&](?<Field1>.*)Field2[;&](?<Field2>.*)"Field3[;&](?<Field3>.*)

Could someone please help with this? I imagine there probably is a better way to do this, but I am still trying different ways. Is there a way to just give my delimiters and have it extract everything in it's own field?

Thanks

Re: Separate Fields

lguinn2 — Mon, 17 Dec 2012 22:12:13 GMT

Field1=abcdefg&Field2=12345;field3=98373

does not match any of your regular expressions, as field3 is lower case in the above example. I will assume that you meant:

Field1=abcdefg&Field2=12345;Field3=98373

instead. Now your first regular expression will almost work, although it has a spurious " in it. The second regular expression has lost the = and put the field separator in the wrong places.
You could do this:

yoursearchhere
| rex  "Field1=(?<Field1>.*)[;&]Field2=(?<Field2>.*)[;&]Field3=(?<Field3>.*)"

Here is another answer that may be helpful:

http://splunk-base.splunk.com/answers/30409/splunk-field-seperators

Here is an example of using DELIMS in transforms.conf. It applies to an event where field/value pairs are separated by ';' symbols, and the field names are separated from their corresponding values by '=' symbols:

[pipe_eq]
DELIMS = ";", "="

You can find out more about DELIMS here. Also look at the documentation for transforms.conf

Re: Separate Fields

jaykay — Tue, 18 Dec 2012 10:16:32 GMT

Sorry, the errors in that were due to me trying to remember the data off the top of my head. Turns out the code you posted and the code I used are the same, and that didn't quite work. Here is the actual format of the data:

field1=field1value=&field2value=0&field3value=0&field4=DataNotRequired
field1=field1value;+Field5=DataNotRequired=djkhkdjkhdkjash

Re: Separate Fields

jaykay — Tue, 18 Dec 2012 10:16:50 GMT

Extracting field1 works, but then a lot of the other fields (2 onwards) are all left in there also. What if I wanted to just extract that one field and just search the various delimiters at the end of it? Could I then later pick another field and extract that (whether that data is before or after the already extracted data)? Unfoortunately, as the data which follows is not always consistent, I cannot input that data into a new field by naming the field.

Thanks

Re: Separate Fields

jaykay — Tue, 18 Dec 2012 11:54:34 GMT

Ok so I have been testing this further, and the following works better for me, as it seems to extract mroe fields. It is still not perfect for the most important fields though.

search term | extract pairdelim=";+&", kvdelim="=", auto=f

Using this, the following do not get caught still:

Field72=&Field1=Field1Value&Field52=Field52Value

Note that Field72 is blank, which may be the reason for it not being extracted.