Getting Data In

Why logstream field from Cloudwatch is being rejected by Kinesis Firehose when using HEC with event data?

David1
Loves-to-Learn

I've setup Kinesis Firehose to push to Splunk HEC which is ingesting fine, however, I would like to add the logstream field from Cloudwatch to Splunk.   The code used in the Lambda can be found here: https://github.com/ptdavies17/CloudwatchFH2HEC 

 

 

 

sourcetype = os.environ['SPLUNK_SOURCETYPE']
return_message = '{"time": ' + str(log_event['timestamp']) + ',"host": "' + \
arn + '","source": "' + filterName + ':' + loggrp + '"'
return_message = return_message + ',"sourcetype":"' + sourcetype + '"'
return_message = return_message + ',"event": ' + \
json.dumps(log_event['message']) + '}\n'
return return_message + '\n'

 

 

 

 
The code block above works and it returns the formatted message:

 

 

 

{
    "time": 1234567891011,
    "host": "arn",
    "source": "some_source",
    "sourcetype": "some_source_type",
    "event": "event_data"
}

 

 

 

 When I add the logstream to the code block as such:

 

 

 

sourcetype = os.environ['SPLUNK_SOURCETYPE']
return_message = '{"time": ' + str(log_event['timestamp']) + ',"host": "' + \
arn + '","source": "' + filterName + ':' + loggrp + '"'
return_message = return_message + ',"logstream":"' + logstrm + '"'
return_message = return_message + ',"sourcetype":"' + sourcetype + '"'
return_message = return_message + ',"event": ' + \
json.dumps(log_event['message']) + '}\n'
return return_message + '\n'

 

 

 

The changes get processed by the transformation Lambda correctly( it is a valid json and the logs in Cloudwatch confirm that):

 

 

 

{
    "time": 1234567891011,
    "host": "some_host",
    "source": "some_source",
    "logstream": "logstream_in_amazon",
    "sourcetype": "some_source_type",
    "event": "some_event_data"
}

 

 

 

But the Kinesis Delivery Stream to Splunk errors out with:

 

 

 

The data is not formatted correctly. To see how to properly format data for Raw or Event HEC endpoints, see Splunk Event Data (http://dev.splunk.com/view/event-collector/SP-CAAAE6P#data). HecServerErrorResponseException{serverRespObject=HecErrorResponseValueObject{text=Invalid data format, code=6, invalidEventNumber=0}, httpBodyAndStatus=HttpBodyAndStatus{statusCode=400, body={"text":"Invalid data format","code":6,"invalid-event-number":0}...

 

 

 

Is there something I'm missing? I've tried tons of changes to the Lambda code, but all of them return this error. Maybe I cannot add a field from CW to Splunk this way?

I am new to Splunk, so I might not be asking the right question or might be missing something, but any help would be much appreciated.

Labels (1)
Tags (2)
0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...