Knowledge Management

External Lookup not passing correct arguments to Perl script

raynold_peterso
Path Finder

I have an external perl script which pulls enrichment data for events. I need to pass the lookup a string for the perl script to pull the correct data. Below are my config files:

transform.conf
[RcpArgTest]
allow_caching = 0
case_sensitive_match = 1
external_cmd = RcpArgTest.cmd Notification
external_type = executable
fields_list = Notification, Notes, Summary, Severity, ClassName, InstanceName, EventName, EventText

This config calls external_cmd of RcpArgTest.cmd:

RcpArgTest.pl %1 %2 %3 %4

The perl script at this point is very simple just to log actions and show passed variables.

#!D:\InCharge\CONSOLE\smarts\bin\sm_perl.exe
use warnings;
use strict;

# Open the log file
openLog();

# Command line arguments - file to process.
debugLog("Argument Count: ".@ARGV);
debugLog(" Program Input: ".$ARGV[0]);
debugLog(" Program Input: ".$ARGV[1]);
debugLog(" Program Input: ".$ARGV[2]);
debugLog(" Program Input: ".$ARGV[3]);

print "Notification, Notes, Summary, Severity, ClassName, InstanceName, EventName, EventText";
closeLog();

sub getLoggingTime {

    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localtime(time);
    my $nice_timestamp = sprintf ( "%04d/%02d/%02d %02d:%02d:%02d",
                                   $year+1900,$mon+1,$mday,$hour,$min,$sec);
    return $nice_timestamp;
}

sub debugLog {
    my ($message) = @_;
    print LOG getLoggingTime().": $message\n";
}

sub openLog {
    # Open the log file
    open LOG,">> D:\\Program Files\\Splunk\\etc\\apps\\search\\logs\\notif_call.log" or die "Could not open notif_call.log $!\n";
    debugLog("Starting Program....");
}

sub closeLog {
    debugLog("Work done - Ending program");
    close LOG;
}

Now, when this lookup is ran with the following.....

| stats count
| eval Notification="NOTIFICATION-VoltageSensor__Fault__PowerSupply210__CiscoUCSChassis_I-VoltageSensor__Fault__PowerSupply210__CiscoUCSChassis-VOLT-BEM-10.7.15.5[ALBISC02CR-N4R3-FAB]/2/3356137/Chassis-2/Psu-4/210V_OutOfRange"
| table Notification
| lookup RcpArgTest Notification

I receive no errors with this search. Also the log file defined shows the following:

2019/03/13 09:58:57: Starting Program....
2019/03/13 09:58:57: Argument Count: 1
2019/03/13 09:58:57:  Program Input: Notification
2019/03/13 09:58:57:  Program Input: 
2019/03/13 09:58:57:  Program Input: 
2019/03/13 09:58:57:  Program Input: 
2019/03/13 09:58:57: Work done - Ending program

As you can see, my argument which is passed is the column name of my data "Notification". I don't this is correct. I am expecting the value for Notification and not just the name.

Something is amiss. Can anyone show me the error in my ways. I am sure this is a simple fix. At least that is what I am hoping.

Thanks in advance,
Rcp

1 Solution

raynold_peterso
Path Finder

Well, once again I had no takers on this issue at all. I did open a case with Splunk and they have been working on it without any success.

I guess its up to me to find my own resolution.

I did find a solution! It took long enough.

During my call with Splunk yesterday, we talked through my current configuration and the flow of the data between Splunk and my external lookup script. A light went off in my head. Let me explain.

My transforms.conf is as follows:

[NotificationDataPerl]
external_cmd = NotifEnrich.cmd Notification
external_type = executable
fields_list = Notification, ClassName, EventName, InstanceName, Notes, Severity, Summary, EventText

The problem thus far has been in sending the filled in CSV file from my lookup script back into Splunk. The documentation stays its a CSV file supplied to standard input. I have a windows cmd file which calls my perl scirpt which does the heavy lifting. Once it has gathered my enriching data, it would pass it to the standard output. *I realized the breakdown was between my perl script and the windows cmd, and not Splunk! *

So, I found a cmd wrapper script for perl which allows you to have a cmd which contains the perl script inside the cmd. All in one file. This was the solution I was looking for.

My enrichment perl script:

@rem --*-Perl-*--
@perl.exe -x "%~f0" %*
goto :eof
#!perl

"Your Perl code..."


open OUT, ">&STDOUT";
print OUT "Enrichment data as CSV";

This caused the data to be sent to Splunk as expected and the enrichment data is now presented to splunk after the external lookup.

This is working very well not for me and I have no issues, well at least not any involving Splunk.

I hope this helps you in the future.

Rcp

View solution in original post

0 Karma

raynold_peterso
Path Finder

Well, once again I had no takers on this issue at all. I did open a case with Splunk and they have been working on it without any success.

I guess its up to me to find my own resolution.

I did find a solution! It took long enough.

During my call with Splunk yesterday, we talked through my current configuration and the flow of the data between Splunk and my external lookup script. A light went off in my head. Let me explain.

My transforms.conf is as follows:

[NotificationDataPerl]
external_cmd = NotifEnrich.cmd Notification
external_type = executable
fields_list = Notification, ClassName, EventName, InstanceName, Notes, Severity, Summary, EventText

The problem thus far has been in sending the filled in CSV file from my lookup script back into Splunk. The documentation stays its a CSV file supplied to standard input. I have a windows cmd file which calls my perl scirpt which does the heavy lifting. Once it has gathered my enriching data, it would pass it to the standard output. *I realized the breakdown was between my perl script and the windows cmd, and not Splunk! *

So, I found a cmd wrapper script for perl which allows you to have a cmd which contains the perl script inside the cmd. All in one file. This was the solution I was looking for.

My enrichment perl script:

@rem --*-Perl-*--
@perl.exe -x "%~f0" %*
goto :eof
#!perl

"Your Perl code..."


open OUT, ">&STDOUT";
print OUT "Enrichment data as CSV";

This caused the data to be sent to Splunk as expected and the enrichment data is now presented to splunk after the external lookup.

This is working very well not for me and I have no issues, well at least not any involving Splunk.

I hope this helps you in the future.

Rcp

0 Karma

raynold_peterso
Path Finder

Since I saw no takers on answering this question, I thought I might need to find my own.

I read the fine print of the documentation and kind of read between the lines.

The doc says that splunk will "Pass the values" to your script. I was assuming that the "Pass" was in the form of a CSV as a command line argument. No so!!! The "Pass" is at the STDIN layer. So, I made my changes and found the data I was looking for as a CSV. I tested and changed my code around to process that STDIN and I am getting my data that I am expecting.

#!D:\InCharge\CONSOLE\smarts\bin\sm_perl.exe
use strict;
use warnings;
use Text::CSV qw( csv );

my $aoh = csv (in => *STDIN,
               headers => "auto");   # as array of hash

my $Notification = $$aoh[0]->{Notification};

All works as expected, until I want to send the updated CSV back to Splunk.

I am using a standard module to do the CSV handling in Perl, Text::CSV. It will input and output the data in a CSV where ever you want it to go.

I am outputting my updated CSV as follows:

csv (in => $aoh, out => *STDOUT );

I also sent the CSV to a file...

csv (in => $aoh, out => "test.csv" );

And the outcome is:

InstanceName,ClassName,Notes,EventText,Notification,Summary,Severity,EventName
I-VoltageSensor_Fault_PowerSupply210_CiscoUCSChassis-VOLT-BEM-10.7.15.5[ALBISC02CR-N4R3-FAB]/2/3356137/Chassis-2/Psu-4/210V,VoltageSensor_Fault_PowerSupply210_CiscoUCSChassis,"Initial Alert","Indicates that the voltage for this device is outside of the normal operating range and exceeds RelativeVoltageThreshold.This event is generated when the CurrentValue of the component is greater than the HighThreshold.",NOTIFICATION-VoltageSensor__Fault__PowerSupply210__CiscoUCSChassis_I-VoltageSensor__Fault__PowerSupply210__CiscoUCSChassis-VOLT-BEM-10.7.15.5[ALBISC02CR-N4R3-FAB]/2/3356137/Chassis-2/Psu-4/210V_OutOfRange,"VoltageSensor OutOfRange 100%: VOLT-10.7.15.5[ALBISC02CR-N4R3-FAB]/sys/chassis-2/psu-4/stats-210V",2,OutOfRange

All as I expect, but the STDOUT portion is not sending back the fields to the Splunk lookup and its output. I get just nothing!

alt text

What possibly could I be missing?

Thanks again in advance,
Rcp

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...