What's the fastest way to inject data to HTTP Event Collector, from a PERL script?

[mdonnelly_splun]

I've been searching for a generic example of how to bring data from a perl script, into Splunk using HEC, including HTTPS.

Use dedicated HEC libraries for PERL?

Have the perl script call the curl commands that are found in the HEC docs?

Or use something perl-native, really lightweight and readily available?

[mdonnelly_splun · Accepted solution]

The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.

After testing functionality and performance - one good option is REST::Client

The example file below is useful for the use case of, testing performance of injection from perl. On my test VM, the code below inserted 20,000 events per minute into Splunk 7.3.1.

Of course, this example's real purpose is to show you how to establish the connection, disable SSL verification if needed, and how to inject using POST routines. In a real script, you'd set the JSON content in the $client->POST commands. (Fancy coders might even use additional perl libraries to build up the JSON.)

#!/usr/bin/perl -w
use strict;
use v5.16.3;
use REST::Client;
my $token = '7a49c676-22b8-40e3-8e37-9dad19a9bdb2';
my $hec_uri = 'https://127.0.0.1:8088';
my $n=3600;  # 3600 is one insert every 1 second for 1 hour

my $client = REST::Client->new();
$client->getUseragent()->ssl_opts(verify_hostname => 0);
$client->getUseragent()->ssl_opts(SSL_verify_mode => 'SSL_VERIFY_NONE');
$client->setHost($hec_uri);
$client->addHeader("Authorization", "Splunk $token");

$client->POST('/services/collector/event', '{"sourcetype": "mysourcetype", "event":"my payload here"}');  ## initial test
print $client->responseContent();

print "\n==========================";
print "\nTiming $n runs using direct perl libs: ";
my $start=time();
my $i=0;
while ($i++ <=$n)
{
        $client->POST('/services/collector', "{\"sourcetype\": \"mysourcetype\", \"event\":\"testrun=$$ speed TEST #$i\"}");  
}
my $end=time();
my $elapsed=$end-$start;
print "took $elapsed seconds\n";