Splunk Search

help with outputcsv and map commands needed

damucka
Builder

Hello,

I desperately search the way to overcome the issue with the map command overwriting the variable values.
I came up with the idea that I would dump them before the map into the csv and then read back again, but ... maps seems to overwrite also the csv file in the var/run/splunk/csv directory.

My code looks as follows:

 | outputtext usexml=false | fields decision host_to_trigger triggertime| fields - _raw  | outputcsv rtetriggering_ICP.txt

 | where isnotnull(host_to_trigger) and isnotnull(decision)
 | map maxsearches=20 search="dbxquery query=\"call SYS.MANAGEMENT_CONSOLE_PROC('runtimedump dump -f /usr/sap/ICP/HDB02/$host_to_trigger$/trace/DB_ICP/iAlerting_rtedump_ANOMALY_$triggertime$.trc','$host_to_trigger$:30240',?)\" connection=\"HANA_MLBSO_ICP\" | eval decision=$decision$ "

 |rename comment AS " *** Restore decision var as it was vleared by the map  ***  "
 | appendcols
 [
 | inputcsv rtetriggering_ICP.txt | eval decision=decision | eval host_to_trigger=host_to_trigger | eval triggertime=triggertime
 ]

As soon as map becomes any parameters, the rtetriggering_ICP.txt gets wiped out as well.
Please help.

Kind Regards,
Kamil

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

The map command would fail if trying to use null tokens. If possible, set the null value to some indicative string value and then use the map command, something like this

| table host_to_trigger decision triggertime   
| eval host_to_trigger=coalesce(host_to_trigger,"NA") 
| eval decision=coalesce(decision,"NA")
| eval triggertime=coalesce(triggertime,"NA")
  | where isnotnull(host_to_trigger) and isnotnull(decision)
  | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The map command would fail if trying to use null tokens. If possible, set the null value to some indicative string value and then use the map command, something like this

| table host_to_trigger decision triggertime   
| eval host_to_trigger=coalesce(host_to_trigger,"NA") 
| eval decision=coalesce(decision,"NA")
| eval triggertime=coalesce(triggertime,"NA")
  | where isnotnull(host_to_trigger) and isnotnull(decision)
  | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "
0 Karma

damucka
Builder

Hello @somesoni2

Unfortunately it is not that easy. The point is that with the "NA" values the map will still be executed, in my case (original, not the simplified above) going to the database and trying to trigger dumps there, taking 120 sec. And I execute it in the alert each minute.
The question would be also why the "where" does not prevent it.
I tried/errored a bit and came up with the below, which kind of works, but still I do not know why:

 | outputtext usexml=false | fields decision host_to_trigger triggertime| fields - _raw  | outputcsv rtetriggering_ICP_test.txt
 | table host_to_trigger decision triggertime   
 | where isnotnull(host_to_trigger) and isnotnull(decision) 
 | map maxsearches=20 search="dbxquery query=\"call SYS.MANAGEMENT_CONSOLE_PROC('runtimedump dump -f /usr/sap/ICP/HDB02/$host_to_trigger$/trace/DB_ICP/indexserver_$host_to_trigger$.30240.rtedump.iAlerting_ANOMALY_$triggertime$.trc','$host_to_trigger$:30240',?)\" connection=\"HANA_MLBSO_ICP\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

 |appendcols
 [
 | inputcsv rtetriggering_ICP_test.txt | eval decision=decision | eval host_to_trigger=host_to_trigger
 ] 
 | table host_to_trigger decision triggertime  
 | where isnotnull(host_to_trigger) and isnotnull(decision) 
 | map maxsearches=20 search="dbxquery query=\"call SYS.MANAGEMENT_CONSOLE_PROC('profiler clear','$host_to_trigger$:30240',?)\" connection=\"HANA_MLBSO_ICP\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

Kind Regards,
Kamil

0 Karma

somesoni2
SplunkTrust
SplunkTrust

What's your requirement here? Do you want to show the fields decision host_to_trigger triggertime from first query to result of map command?? Is | eval decision=$decision$ not working for you?

0 Karma

damucka
Builder

After you hint I started trying and overcome the issue, partially. In the below code (I changed the db query to make it simplier) all works fine as long as the host_to_trigger, decision and triggertime are set before.
When they are empty however, the second map command reports and error:
Error in 'map': Did not find value for required attribute 'decision'.

| table host_to_trigger decision triggertime   
 | where isnotnull(host_to_trigger) and isnotnull(decision)
 | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "


 | where isnotnull(host_to_trigger) and isnotnull(decision)
 | map maxsearches=20 search="dbxquery query=\"select 'foo' from dummy\" connection=\"HANA_MLBSO\"  |appendcols[|makeresults| eval decision=\"$decision$\" | eval triggertime=\"$triggertime$\" | eval host_to_trigger=\"$host_to_trigger$\"] "

This I do not understand however - I would expect that the where isnotnull will catch the emtpy values and not let the map being executed.

0 Karma

damucka
Builder

One more thing:
- I do not like how the map command works at all, honestly, it creates lots of issues on my side with clearing the variables. But as per my understanding it is the only command where I can pass the arguments/variables "forward". So I have to use it.
In case someone could suggest any other way to execute the query with the parameters set before, I would be happy to give up on the map.

Kind regards,
Kamil

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...