"Pipeline data does not have indexKey" in a simple modular input TA

[ww9rivers]

I am writing a simple TA to read a text file and turn it into a list of JSON events. I am getting a WARN message for each event from the TcpOutputProc process, such as the one below:

02-21-2025 01:06:04.001 -0500 WARN  TcpOutputProc [2061704 indexerPipe] - Pipeline data does not have indexKey.

I removed the rest of the message containing details.

It seems that I am missing something simple. I would greatly appreciate some insights/pointers towards debugging this issue.

The TA code is here in GitHub:
https://github.com/ww9rivers/TA-json-modinput

Many thanks in advance!

[kiran_panchavat]

@ww9rivers 

The warning message "Pipeline data does not have indexKey" typically indicates that the data being sent to the indexer is missing the necessary index information. 

Make sure that the inputs.conf file on your forwarder or heavy forwarder is configured with the correct index. I recommend creating and using a dedicated index instead of the main index, as main is the default index and it's better to keep your data organized.

[ww9rivers]

My earlier reply was "marked as spam" by the message board. Let me try again.

Thank you for the reply. That is the one thing I checked and double checked in my attempt to fix my problem (the events data do not reach my Splunk Cloud instance)>

But that may not be the problem as the message below (I removed data payload and hostnames for sanitary reasons) indicates that an index is provided: 

02-21-2025 01:06:04.001 -0500 WARN  TcpOutputProc [2061704 indexerPipe] - Pipeline data does not have indexKey. [_path] = /app/splunk/etc/apps/TA-json-modinput/bin/nix_input.py\n[python.version] = python3\n[_raw] = </data><done /></event><event stanza="nix_input://ni" unbroken="1"><source>nix_input://ni</source><sourcetype>hits:unix:hosts</sourcetype><index>test</index><data>{"hostname":"......",......}\n[_meta] = timestamp::none punct::"</><_/></><_=\"://\"_=\"\"><>://</><>::</><></><>{\"\":\""\n[_stmid] = GUsvaYoWsFPrNDD.H\n[MetaData:Source] = source::nix_input\n[MetaData:Host] = host::......\n[MetaData:Sourcetype] = sourcetype::nix_input\n[_linebreaker] = _linebreaker\n[_nfd] = _nfd\n[_charSet] = UTF-8\n[_time] = 1740117963\n[_conf] = source::nix_input|host::......|nix_input|28\n[_channel] = 28\n

Much appreciated!

[ww9rivers]

Thank you much for the help.

That is one thing I checked and double checked in my attempts of trying to fix the problem. But it may not be the case.

In fact, the message shows that an index is provided in the event forwarding (I removed the payload and hostname for sanitary reasons):

02-21-2025 01:06:04.001 -0500 WARN  TcpOutputProc [2061704 indexerPipe] - Pipeline data does not have indexKey. [_path] = /app/splunk/etc/apps/TA-json-modinput/bin/nix_input.py\n[python.version] = python3\n[_raw] = </data><done /></event><event stanza="nix_input://ni" unbroken="1"><source>nix_input://ni</source><sourcetype>hits:unix:hosts</sourcetype><index>test</index><data>{"hostname":"......",......}\n[_meta] = timestamp::none punct::"</><_/></><_=\"://\"_=\"\"><>://</><>::</><></><>{\"\":\""\n[_stmid] = GUsvaYoWsFPrNDD.H\n[MetaData:Source] = source::nix_input\n[MetaData:Host] = host::......\n[MetaData:Sourcetype] = sourcetype::nix_input\n[_linebreaker] = _linebreaker\n[_nfd] = _nfd\n[_charSet] = UTF-8\n[_time] = 1740117963\n[_conf] = source::nix_input|host::splunkhf-prod02|nix_input|28\n[_channel] = 28\n

Since this is actually a WARN message, now I suspect if this is not the reason that my events data do not get into my Splunk Cloud instance (the TA is run on a heavy forwarder).

Much appreciated!