I'm using the Splunk_TA_aws-kinesis-firehose app to get logs from Cloudwatch to Splunk via. Firehose and the logs are flowing in fine and I can see everything I expect to see. It appears that the logGroup field is broken, however, as results don't seem to be matching on that field even though Splunk itself is identifying it. Here's an example:
I am searching on the following query: index="infrastructure" sourcetype="aws:firehose:text" errorMessage=* , limited to just today, and as expected, there is one result, as follows:
2018-12-04T14:10:02.615Z 4a993bef-f7ce-11e8-a331-7f313ce0f31a {"errorMessage":"[version_conflict_engine_exception] [person][C68F52CA6A7D11D5A4AE0004AC494FFE]: version conflict, document already exists (current version [1]), with { index_uuid=\"7X5zhz5LQde60qEXF32cMQ\" & shard=\"2\" & index=\"product-mrportal.search-canonical.identities.int.2017-12-v002\" }","errorType":"Error","stackTrace":["r (/var/task/persons.js:1:57735)","a (/var/task/persons.js:1:57257)","t.<anonymous> (/var/task/persons.js:1:749015)","IncomingMessage.t (/var/task/persons.js:1:163075)","emitNone (events.js:91:20)","IncomingMessage.emit (events.js:185:7)","endReadableNT (_stream_readable.js:974:12)","_combinedTickCallback (internal/process/next_tick.js:80:11)","process._tickDomainCallback (internal/process/next_tick.js:128:9)"]}
Splunk is correctly identifying fields, most importantly, the following: logGroup =/aws/lambda/oris-es-pipeline-test-MrPersonDocManagerFunction-6EDCTGWYWK8
but when I click this field and "add to search," my search changes to the following: index="infrastructure" sourcetype="aws:firehose:text" errorMessage=* logGroup="/aws/lambda/oris-es-pipeline-test-MrPersonDocManagerFunction-6EDCTGWYWK86" and i now see no results.
is there some known issue with the logGroup field using this addon, maybe? have I messed up some configuration inadvertently?
... View more
