topic Re: I need help finding a substring and setting it to display in a new field. in Splunk Search

I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 16:30:24 GMT

I am sure this is probably a noob question, but I am a noob and I have been researching this for a while this morning and am not having any luck. Maybe you can help!

Ok, I am pulling a query from a log file that returns a random string of text such as:

xxxxxxxxxxxxxxxxxxxxxxxxxx11=123456xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

What I would like to do is take that 11= value and set it to a new field called OrderID, whereas the 123456 which just show up in the order ID field. Any help or insight on this would be great!

Thanks again!

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 16:37:06 GMT

Are the xxx's for real? Is the value always preceded by 11=? Is the value always a number? What is the name of the field that contains the random string of text?

Re: I need help finding a substring and setting it to display in a new field.

sbsbb — Wed, 08 Jan 2014 16:38:36 GMT

Hi,

I'm not sure what you wan't

If you wan't a field OrderID with a value "11=123456"
I would make something like
| rex "(?P\d*\=\d*)"

I you need a field 11, that has a value 123456, I don't know

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 16:41:46 GMT

sorry! ok the xxxx's are just to represent the random text that may precede or follow the 11= value. yes the value is always preceded by 11=. For the sake of simplicity and my understanding lets just say it is always a number. This text is just being pulled from the _raw field.

Thanks!

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 16:45:33 GMT

Try

yoursearch |rex ".*[1][1]=(?P<orderID>\d+)\D"

This should create a field from _raw named orderID.

Explaination:

rex used without a field= will extract from _raw

The expression needs to be enclosed in quotes.

.* means any sequence of characters or symbols.

[1][1] means exactly the number 11

= is not a regular expression, so it is not escaped and means exactly the symbol =.

(?P<orderid>\d+) is the capture group.

? means not lazy, or grab the first occurance.

P stands for pull the value and name it orderid.

<orderid> sets field name for the extracted value, in this case we're going to call it orderid.

\d+ means one or more digit. So, what we have so far is, grab one or more digits after 11= as value for the field orderid.

\D means not a digit. So, it says stop grabbing digits for the value when a non-digit is encountered.

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 16:50:39 GMT

thanks, I am ok on the understanding of creating a new field, what I am having trouble with is taking the 11=123465 and then just returning 123456 in that OrderID field.

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 16:52:48 GMT

sorry! to clarify, I would like to find the "11=123456" and then just return the 123456 in the OrderID field. The 123456 is just for example, this could be any random number following the "11="

Thanks!

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 16:54:53 GMT

The above code should do it. It grabs the value after the = and puts it in a new field called orderID.

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 17:07:25 GMT

doesnt seem to work, it is creating a new field called OrderID, but it is coming up empty! 😞

This is my search: search ErrorMessage | rex ".*\d\d=(?P\d+)\D" | table OrderID

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 17:07:57 GMT

I updated the code so it specifically looks for "11=" and grabs the value after that. It will stop grabbing numbers when it hits a non-digit, and it assumes there are no spaces between the = and the value you're after.

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 17:16:32 GMT

hmm, I tried again with updated code and still doesnt seem to pull the number, still just seeing an empty OrderID field. I am fairly new to using splunk, so it's probably user error, but I just tried with: search ErrorMessage | rex ".*[1][1]=(?P\d+)\D" | table OrderID

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 17:44:47 GMT

thaks so much for your help, any other thoughts on why its not working for me? Again, this is my query: search ErrorMessage | rex ".*[1][1]=(?Pd+)D" | table OrderID

but the OrderID field is just showing as blank...

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 17:46:01 GMT

Is the 11= showing up anywhere else?
Can you post a the `_raw' output from one of the events?

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 17:55:47 GMT

this is piece of the raw output where the 11= shows, and no other 11= are in here

\x150=ABCD\x1128=ABCD\x11=73563269\x155=ABCD\x154=1

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 18:06:48 GMT

Your problem might be a case issue. orderid does not equal OrderID Fields are case sensitive.
I've retested and it works fine for me...

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Wed, 08 Jan 2014 18:07:42 GMT

Try:
search ErrorMessage | rex ".*[1][1]=(?P<orderid>d+)D" | table orderid

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Wed, 08 Jan 2014 18:48:15 GMT

still no luck, but I am working on it, thanks for your help!

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Thu, 09 Jan 2014 16:21:07 GMT

I feel like I am just missing something simple, can you break down this part ".*[1][1]=(?Pd+)D" and tell me exactly what each part is doing? I have a general understanding but maybe your explanation might clear things up for me!

Thanks again!

Re: I need help finding a substring and setting it to display in a new field.

lukejadamec — Thu, 09 Jan 2014 16:34:18 GMT

Some how an error was introduced from the original formula. It is missing slashes and should be:
search ErrorMessage | rex ".*[1][1]=(?P<orderid>\d+)\D" | table orderid

Re: I need help finding a substring and setting it to display in a new field.

dlespron — Thu, 09 Jan 2014 16:55:38 GMT

THAT WORKED! Thanks so much you are awesome! Also, I greatly appreciate you taking the time to provide explanation, this is all very helpful to a Splunk noob like myslef!