Splunk Search

How do you populate a field if the search time extracted field (using regular expression) is not presented in the logs?

raj_mpl
Path Finder

Hi All,

I am trying to populate a custom field value if my search time extracted field is not present in the raw log by using the below two methods . Here refield is my search time extracted field

1)mysearch | eval Myfield=if(isnotnull(refield),refield,Custom_field)

2)|eval Myfield=coalesce(refield,Custom_field)

But, in the output, I am getting the result in Myfield as values of both refield and Custom_field .

Thank you !

Tags (2)
0 Karma
1 Solution

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl ,

The duplicated values are because of transaction command which brings together all the values matches the transaction.

Since since the transaction doesn't depend on the Error_Description , do the coalesce after the transaction and before the table command or just fillnull value="check the log" Error_Description at the end of the search

e.g.

  index=myindex |rex field=_raw "(?ms)TEXT\.\s+(?P<refield>.+?)at\s(web|org)\."|eval Custom_field=”check the log”
  |transaction  id,host  startswith="started" endswith="completed"
  |eval  Error_Description=coalesce(refield,Custom_field)
  |table  id host Error_Description
Happy Splunking!

View solution in original post

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl ,

The duplicated values are because of transaction command which brings together all the values matches the transaction.

Since since the transaction doesn't depend on the Error_Description , do the coalesce after the transaction and before the table command or just fillnull value="check the log" Error_Description at the end of the search

e.g.

  index=myindex |rex field=_raw "(?ms)TEXT\.\s+(?P<refield>.+?)at\s(web|org)\."|eval Custom_field=”check the log”
  |transaction  id,host  startswith="started" endswith="completed"
  |eval  Error_Description=coalesce(refield,Custom_field)
  |table  id host Error_Description
Happy Splunking!
0 Karma

raj_mpl
Path Finder

Thank you @renjith.nair . I just changed the placement of the Error_Description filed
And it worked perfectly . Thank you

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl , for incomplete transaction as mentioned in the comment ,

try

 |transaction  id,host  startswith="started" endswith="completed" keepevicted=true
 |where closed_txn=0|eval runTime=round((now()-_time)/3600,2)
|where runTime>2
Happy Splunking!
0 Karma

raj_mpl
Path Finder

Can you please explain a bit , What actually it will perform ?

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl ,
Myfield=coalesce(refield,Custom_field) should give you the first non-null value. What you mean by both values are assigned to Myfield ? Is it a list or concatenated?

Happy Splunking!
0 Karma

raj_mpl
Path Finder

Hi @renjith.nair , Thanks for your reply on this
Yes I am getting the two results by concatenation in a separate line for Myfield

Myfield
Value1 (regex extracted string (captured group))
Value2( Custom_field value)

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Do you mind sharing your search ?

Happy Splunking!
0 Karma

raj_mpl
Path Finder

Sure , Below is my search

 index=myindex |rex field=_raw "(?ms)TEXT\.\s+(?P<refield>.+?)at\s(web|org)\."|eval Custom_field=”check the log”| eval  Error_Description=if(isnotnull(refield),refield,Custom_field)|transaction  id,host  startswith="started" endswith="completed" |table  id host Error_Description

And also tried |eval Error_description=coalesce(refield,Custom_field)

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl ,
Thats because your one transaction has more than one values for Error_Description. You can verify by removing the table command and look at the events directly

Happy Splunking!
0 Karma

raj_mpl
Path Finder

Hi @renjith.nair .., Yes my regex will extract the field value for Error_Description at search time , My requirement is if the regex provided will not able to pick anything as per the condition , I have to populate an new field

But what actually happening is The Error_Descriptin field is having field a value also in it as you said 2 values

Then what is the resolution for this?

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl ,since the transaction doesn't depend on the Error_Description , do the coalesce after the transaction and before the table command or just fillnull value="check the log" Error_Description

Happy Splunking!
0 Karma

raj_mpl
Path Finder

Thank you @renjith.nair , I just changed the placement of the Error_Description filed
And it worked perfectly .

On an other note can you give some ideas to me to find the Long running jobs using transaction command.
I want to create an alert for long running transactions .
Consider events will start with "start" and completes with "closed" string. In this with a Customer_Id common in them .

Note : My focus is not on completed transactions , I have to identify the ongoing jobs which are running from past 2 hours and not closed yet (still running) .

Thank you .
Rajesh

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

@raj_mpl ,alright. I will move the comment to the answer section.
For your next question, is the customer id unique for each transaction ? and if not how do you identify the transactions - especially if another transaction starts and end before the first transaction (overlapping) ? If there is a uniq id for each transaction/job, we might be able to find it without using a transaction command.

Happy Splunking!
0 Karma

raj_mpl
Path Finder

Hi @renjith.nair , Yes the Job name(customer ID) and the id is unique for each transaction
Below is the Query I am using for still running Jobs .
index=myindex "] Agent" "load plan instance" | rex field=_raw "instance\s(?[^)]+)\s((?[^)]+)" |transaction Job_Name,id startswith="started plan instance" endswith="successfully completed Plan" keepevicted=true | where closed_txn=0 | search NOT stopped | table _time, Job_Name, userid

My requirement is to get an alert for the transaction which is still in progress (from past 2hours) , I will schedule the alert to run every 20 min using cron notation

0 Karma

raj_mpl
Path Finder

You can reply me in the thread Can you help me create an alert involving the transaction command

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...