Splunk Search

help with outputcsv and map commands needed

damucka
Builder

Hello,

I desperately search the way to overcome the issue with the map command overwriting the variable values.
I came up with the idea that I would dump them before the map into the csv and then read back again, but ... maps seems to overwrite also the csv file in the var/run/splunk/csv directory.

My code looks as follows:

 | outputtext usexml=false | fields decision host_to_trigger triggertime| fields - _raw  | outputcsv rtetriggering_ICP.txt

 | where isnotnull(host_to_trigger) and isnotnull(decision)
 | map maxsearches=20 search="dbxquery query=\"call SYS.MANAGEMENT_CONSOLE_PROC('runtimedump dump -f /usr/sap/ICP/HDB02/$host_to_trigger$/trace/DB_ICP/iAlerting_rtedump_ANOMALY_$triggertime$.trc','$host_to_trigger$:30240',?)\" connection=\"HANA_MLBSO_ICP\" | eval decision=$decision$ "

 |rename comment AS " *** Restore decision var as it was vleared by the map  ***  "
 | appendcols
 [
 | inputcsv rtetriggering_ICP.txt | eval decision=decision | eval host_to_trigger=host_to_trigger | eval triggertime=triggertime
 ]

As soon as map becomes any parameters, the rtetriggering_ICP.txt gets wiped out as well.
Please help.

Kind Regards,
Kamil

0 Karma
1 Solution

somesoni2
Revered Legend

The map command would fail if trying to use null tokens. If possible, set the null value to some indicative string value and then use the map command, something like this

| table host_to_trigger decision triggertime   
| eval host_to_trigger=coalesce(host_to_trigger,"NA") 
| eval decision=coalesce(decision,"NA")
| eval triggertime=coalesce(triggertime,"NA")
  | where isnotnull(host_to_trigger) and isnotnull(decision)
  | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

View solution in original post

0 Karma

somesoni2
Revered Legend

The map command would fail if trying to use null tokens. If possible, set the null value to some indicative string value and then use the map command, something like this

| table host_to_trigger decision triggertime   
| eval host_to_trigger=coalesce(host_to_trigger,"NA") 
| eval decision=coalesce(decision,"NA")
| eval triggertime=coalesce(triggertime,"NA")
  | where isnotnull(host_to_trigger) and isnotnull(decision)
  | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "
0 Karma

damucka
Builder

Hello @somesoni2

Unfortunately it is not that easy. The point is that with the "NA" values the map will still be executed, in my case (original, not the simplified above) going to the database and trying to trigger dumps there, taking 120 sec. And I execute it in the alert each minute.
The question would be also why the "where" does not prevent it.
I tried/errored a bit and came up with the below, which kind of works, but still I do not know why:

 | outputtext usexml=false | fields decision host_to_trigger triggertime| fields - _raw  | outputcsv rtetriggering_ICP_test.txt
 | table host_to_trigger decision triggertime   
 | where isnotnull(host_to_trigger) and isnotnull(decision) 
 | map maxsearches=20 search="dbxquery query=\"call SYS.MANAGEMENT_CONSOLE_PROC('runtimedump dump -f /usr/sap/ICP/HDB02/$host_to_trigger$/trace/DB_ICP/indexserver_$host_to_trigger$.30240.rtedump.iAlerting_ANOMALY_$triggertime$.trc','$host_to_trigger$:30240',?)\" connection=\"HANA_MLBSO_ICP\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

 |appendcols
 [
 | inputcsv rtetriggering_ICP_test.txt | eval decision=decision | eval host_to_trigger=host_to_trigger
 ] 
 | table host_to_trigger decision triggertime  
 | where isnotnull(host_to_trigger) and isnotnull(decision) 
 | map maxsearches=20 search="dbxquery query=\"call SYS.MANAGEMENT_CONSOLE_PROC('profiler clear','$host_to_trigger$:30240',?)\" connection=\"HANA_MLBSO_ICP\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

Kind Regards,
Kamil

0 Karma

somesoni2
Revered Legend

What's your requirement here? Do you want to show the fields decision host_to_trigger triggertime from first query to result of map command?? Is | eval decision=$decision$ not working for you?

0 Karma

damucka
Builder

After you hint I started trying and overcome the issue, partially. In the below code (I changed the db query to make it simplier) all works fine as long as the host_to_trigger, decision and triggertime are set before.
When they are empty however, the second map command reports and error:
Error in 'map': Did not find value for required attribute 'decision'.

| table host_to_trigger decision triggertime   
 | where isnotnull(host_to_trigger) and isnotnull(decision)
 | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "


 | where isnotnull(host_to_trigger) and isnotnull(decision)
 | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

This I do not understand however - I would expect that the where isnotnull will catch the emtpy values and not let the map being executed.

0 Karma

damucka
Builder

One more thing:
- I do not like how the map command works at all, honestly, it creates lots of issues on my side with clearing the variables. But as per my understanding it is the only command where I can pass the arguments/variables "forward". So I have to use it.
In case someone could suggest any other way to execute the query with the parameters set before, I would be happy to give up on the map.

Kind regards,
Kamil

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...