topic Re: How to create a field with varying lengths of field values? in Splunk Search

How to create a field with varying lengths of field values?

gilbxrtx_7 — Mon, 23 Apr 2018 09:55:12 GMT

I want to create a field which extract values, however I have some field values that I want to extract which contain more than one word.
For instance:
(image file: field-error.png)

I want the 'auth' and 'SECURE AUDIT' values to be considered as part of the same field that I am going to create. However I got the field extraction error message: "The extraction failed. If you are extracting multiple fields, try removing one or more fields. Start with extractions that are embedded within longer text strings."

Basically I want to extract the text between the yellow boxed text and the value '0' to be added as a field.
How can I add the field such that I don't get the error?

Splunk only highlighted the first word 'SECURE' instead of 'SECURE AUDIT', which is not what I want.

Output with the following regex suggested:

(?:[^\s]+\s){2}(?P[^\s]+(?:\s\w+)?)\s\d+\s+<

(image file: error-3.png)

I tried to modify the regex by removing the 'w', and it highlighted only the 'auth' texts and 'AUDIT' without the 'SECURE'

To display the result output I have removed my second image as Splunk only allows two image files uploaded.

Regular expression generated by Splunk:
^(?:[^ \n]* ){3}(?P\w+\s+\w+)

I realized the regex suggested by one user extracted wrongly as he/she did not include < 37 > 1 at the front of the lines as indicated in my text line. The wording got misaligned and hence the wrong extraction.
https://regex101.com/r/biHi9a/4
The link above is the correct sample text that I want to work with. Would appreciate any help to modify it to achieve my desired results.

Re: How to create a field with varying lengths of field values?

FrankVl — Mon, 23 Apr 2018 13:39:26 GMT

Can you share the regular expression that Splunk generates for you? Then we can help adjust it to achieve your desired result.

Re: How to create a field with varying lengths of field values?

ssadanala1 — Mon, 23 Apr 2018 14:14:42 GMT

Try this regex

rex field = _raw "\s+\w+\s+(?P.*)\s+\d+"

Re: How to create a field with varying lengths of field values?

p_gurav — Mon, 23 Apr 2018 14:18:35 GMT

Can you try:

| rex field=_raw "(?P<field1>[^\s]+)\s\d+\<"

Re: How to create a field with varying lengths of field values?

cafissimo — Mon, 23 Apr 2018 14:38:47 GMT

Try this regex:

(?:[^\s]+\s){2}(?P[^\s]+(?:\s\w+)?)\s\d+\s+<

Test it here:
https://regex101.com/r/biHi9a/1

Best regards

Re: How to create a field with varying lengths of field values?

gilbxrtx_7 — Tue, 24 Apr 2018 00:52:17 GMT

Hi, your answer seems correct in the regex test link. However when I copied and paste the exact regex into splunk, it somehow highlighted the text wrongly.

It highlighted the text before the auth as part of the field extraction as well.

I have edited my question to display the output when I added the regex you suggested into splunk.
@cafissimo

https://regex101.com/r/biHi9a/4
I realized that your regex sampled worked because you did not include < 37 > 1 at the front of each line. After I added < 37 > 1 the alignment shifted and the text were not highlighted correctly anymore
I have attached the updated link with the correct sample text

Re: How to create a field with varying lengths of field values?

gilbxrtx_7 — Tue, 24 Apr 2018 01:02:31 GMT

I realized what you meant, but I do not want to extract fields while searching, I want to create a fixed field.

Re: How to create a field with varying lengths of field values?

gilbxrtx_7 — Tue, 24 Apr 2018 01:02:47 GMT

I realized what you meant, but I do not want to extract fields while searching, I want to create a fixed field.

Re: How to create a field with varying lengths of field values?

gilbxrtx_7 — Tue, 24 Apr 2018 01:05:15 GMT

This is the regular expression generated by Splunk which extracted wrongly.
^(?:[^ \n]* ){3}(?P\w+\s+\w+)

Re: How to create a field with varying lengths of field values?

kthammireddygar — Tue, 24 Apr 2018 04:30:09 GMT

Try this to create a field with all texts within the square brackets.

| rex "\s*\[(?P<Fieldname>.*?)\]\s*"

Re: How to create a field with varying lengths of field values?

gilbxrtx_7 — Tue, 24 Apr 2018 06:47:19 GMT

this is not what I'm asking for in this question
@kthammireddygari

Re: How to create a field with varying lengths of field values?

cafissimo — Tue, 24 Apr 2018 12:38:23 GMT

Here it is a working complete example with the extraction of "field1"

| makeresults count=2
| streamstats count
| eval _raw=case(count==1,"<37> 1 2018-04-23T08:08:08Z E876876876 SECURE AUDIT 0 <secure audit error> [cdkncdckne dckmdkcmd]",count==2,"<37> 1 2018-04-23T08:08:08Z E876876876 auth 0 <login> [kdnckn rrgrr]
<37> 1 2018-04-23T08:08:08Z E876876876 auth 0 <login> [kdnckn rrgrr]")
| rex field=_raw "<\d+>\s+\d+\s+(?:[^\s]+\s){2}(?P<field1>[^\s]+(?:\s+\w+)*)\s\d+\s+<"

Re: How to create a field with varying lengths of field values?

woodcock — Thu, 10 May 2018 05:12:47 GMT

Go to http://regex101.com and build it yourself. You should not be using Splunk's regex builder.