Splunk Search

Erex example

amarish_vlabs
New Member

Hi, I have the below log and values for "days" field are 4, 10 , 15, 30. Could you please extract the "days" field using the "erex" command.

Log :

2017-11-21 04:55:34,060 tn="[14347886-7337]" ll=INFO cn="s.c.m" - id="57ef4442-aa9f-444d-bd80-e4fd3018f82", action="execute", operation="put", collection="Messages", "days" : "4", "Code" : "491833", useSecondary="false", retries="0"

Tags (1)
0 Karma
1 Solution

niketn
Legend

Following is a run anywhere search example for erex command to extract the field you are interested in (based on the sample provided). Please try out and confirm.

|  makeresults
|  eval _raw="2017-11-21 04:55:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f82\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"4\", \"Code\" : \"491833\", useSecondary=\"false\", retries=\"0\""
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:56:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f83\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"13\", \"Code\" : \"491834\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:57:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f84\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"24\", \"Code\" : \"491835\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:58:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f85\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"14\", \"Code\" : \"491836\", useSecondary=\"false\", retries=\"0\""]
|  erex days examples="4,13"

Ideally you should use rex command and once you have tested the same save your regular expression as Field Extraction for reusability and maintenance. Following is the regular expression:

|  makeresults
|  eval _raw="2017-11-21 04:55:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f82\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"4\", \"Code\" : \"491833\", useSecondary=\"false\", retries=\"0\""
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:56:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f83\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"13\", \"Code\" : \"491834\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:57:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f84\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"24\", \"Code\" : \"491835\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:58:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f85\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"14\", \"Code\" : \"491836\", useSecondary=\"false\", retries=\"0\""]
|  rex "\"days\"\s+:\s+\"(?<days>[^\"]+)\""

While the above examples use makeresults and append to mock some sample events as per question. You can try out the final pipe with erex or rex in your base search returning data as per your question:

Using rex command

<YourBaseSearch>
|  rex "\"days\"\s+:\s+\"(?<days>[^\"]+)\""

Using erex command

<YourBaseSearch>
|  erex days examples="4,13"

PS: erex might not be robust for field extraction in production data or else you will have to use a lot of sample examples and counterexamples.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

saurabh_tek11
Communicator

@amarish_vlabs, Here is your sample Event :

2017-11-21 04:55:34,060 tn="[14347886-7337]" ll=INFO cn="s.c.m" - id="57ef4442-aa9f-444d-bd80-e4fd3018f82", action="execute", operation="put", collection="Messages", "days" : "4", "Code" : "491833", useSecondary="false", retries="0"

using erex:

index=* source="amarish_vlabs.txt"  | erex newfield_days examples=4

Job says : Successfully learned regex. Consider using: | rex "(?i)\-e(?P<newfield_days>\d+)"

Now we can see new Search-time field "newfield_days" has been extracted on-the-fly (not persistent field) and moving forward better we use the rex command as that's efficient & faster.
Using rex :

index=* source="amarish_vlabs.txt"  | rex "(?i)\-e(?P<newfield_days>\d+)"

This answer is to start rolling the ball based on one given event. To generalize this on larger set of data and generate (possibly) precise regular expression using erex command, use the optional arguments like counterexamples, fromfield & maxtrainers.

here is syntax of erex :

erex [<field>] examples=<string> [counterexamples=<string>] [fromfield=<field>] [maxtrainers=<int>]

You may see more examples here.

0 Karma

saurabh_tek11
Communicator

Hi @amarish_vlabs, does this answers your question or you have some query? Please feel free to ask.
If no query, please accept the answer so as to close this open question. 🙂 Thank you - Saurabh

0 Karma

niketn
Legend

Following is a run anywhere search example for erex command to extract the field you are interested in (based on the sample provided). Please try out and confirm.

|  makeresults
|  eval _raw="2017-11-21 04:55:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f82\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"4\", \"Code\" : \"491833\", useSecondary=\"false\", retries=\"0\""
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:56:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f83\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"13\", \"Code\" : \"491834\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:57:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f84\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"24\", \"Code\" : \"491835\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:58:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f85\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"14\", \"Code\" : \"491836\", useSecondary=\"false\", retries=\"0\""]
|  erex days examples="4,13"

Ideally you should use rex command and once you have tested the same save your regular expression as Field Extraction for reusability and maintenance. Following is the regular expression:

|  makeresults
|  eval _raw="2017-11-21 04:55:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f82\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"4\", \"Code\" : \"491833\", useSecondary=\"false\", retries=\"0\""
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:56:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f83\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"13\", \"Code\" : \"491834\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:57:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f84\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"24\", \"Code\" : \"491835\", useSecondary=\"false\", retries=\"0\""]
|  append
    [|  makeresults
    |  eval _raw="2017-11-21 04:58:34,060 tn=\"[14347886-7337]\" ll=INFO cn=\"s.c.m\" - id=\"57ef4442-aa9f-444d-bd80-e4fd3018f85\", action=\"execute\", operation=\"put\", collection=\"Messages\", \"days\" : \"14\", \"Code\" : \"491836\", useSecondary=\"false\", retries=\"0\""]
|  rex "\"days\"\s+:\s+\"(?<days>[^\"]+)\""

While the above examples use makeresults and append to mock some sample events as per question. You can try out the final pipe with erex or rex in your base search returning data as per your question:

Using rex command

<YourBaseSearch>
|  rex "\"days\"\s+:\s+\"(?<days>[^\"]+)\""

Using erex command

<YourBaseSearch>
|  erex days examples="4,13"

PS: erex might not be robust for field extraction in production data or else you will have to use a lot of sample examples and counterexamples.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

amarish_vlabs
New Member

Thanks for your clear explanation. It is very useful.

0 Karma

niketn
Legend

@amarish_vlabs, glad you found it useful. Please accept/up vote the answer if it helped 🙂

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@amarish_vlabs, any reason why you want to rely on erex and not rex or regular expression?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

amarish_vlabs
New Member

@niketnilay, I extracted the days field by using "rex" command. I just want to know how to extract with "erex". If this is possible with "erex", Please let me know. Otherwise just ignore. Thank you sir for your reply.

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

Hi

Can you please try this configuration?

props.conf

[my_sourcetype]
REPORT-mydays = mydays

transforms.conf

[mydays]
REGEX = \"days\"\s:\s\"(?<days>.+)\",\s\"
FORMAT = $1

Thanks

0 Karma

amarish_vlabs
New Member

Thank you so much for your reply.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...