Monitoring Splunk

HttpInputDataHandler - Parsing error : No data

D2SI
Communicator

Hello there,

I got the following error a lot: "ERROR HttpInputDataHandler - Parsing error : No data"

I guess it is related to HEC but I don't understand it nor find info about it.

Would anyone know more about this error?

Labels (2)

mattymo
Splunk Employee
Splunk Employee

I believe these can be safely ignored as "keep alive" calls from firehose/load balancers checking the connection but not sending data. 

Putting in docs feedback on troubleshooting hec and firehose docs for future reference

- MattyMo

ejwade
Contributor

@mattymothis happened to us as well, but only when we moved to a load balancer in front of our indexers. Our previous step, which was HEC on a heavy forwarder, we never had this issue. Do you know if this is specific to load balanced HEC?

0 Karma

mattymo
Splunk Employee
Splunk Employee

Yes, it would be specific to HEC clients that check for the endpoint's availability with tcp connections but not sending data. 

This would not happen with HF because HTTP traffic would come in HF then be sent via S2S protocol to Indexers, which wouldnt do the checks like that. 

sorry for the answer from far in the future 🙂 

- MattyMo

TellTaleMajora
Engager

Bumping this issue. 

We currently leverage AWS Kinesis firehose to ingest log data via HEC. We recently started to see an increase number of "no data" errors reported via the Splunk HEC endpoint. 

However log data appears to continue to function as expected. 

0 Karma

Paul1896
Path Finder

Figure out which configured HEC-Stanza generate the errors via "Monitoring Console --> Indexing --> Input --> HTTP Event Collector: Deployment" and check the configuration on source side.

The incoming requests from the affected source are not valid and can't be handled in a correct way by Splunk.

0 Karma

D2SI
Communicator

Thanks @Paul1896 ! I had not checked that way.

There is no invalid request.

But there are some 'parser errors'. The cool thing is that you can browse 'parser errors' by token. But like I said in the comment above, it matches plenty tokens not just one or two. Plus these tokens are OK, I mean there is data indexed through them, not no data at all.

So I am wondering what are these 'parser errors' ? I mean, from the logs, it does not seem to be timestamp issues.

0 Karma

srinikrishna
New Member

Hi @Paul1896

On my heavy forwarder i cant see this Monitoring console as the logs are not storing in local machine. however on the indexer i do not have monitoring console. Is there any other way to verify this?

I actually got this below error and stopped ingesting logs since then. I dont see any more errors also related to this hec data input in the logs after 8.30. there are other inputs working fine. and fyi i copied this splunk_httpinput folder from my old splunk instance to new splunk instance to avoid recreating all the tokens i had earlier. does this makes any issues ?

01-21-2020 07:36:24.170 +0000 ERROR HttpInputDataHandler - Failed processing http input, token name=OpenBankingAggregateProd, channel=48C994DD-C1F5-462F-BAED-FC00694CF173, source_IP=10.84.31.115, reply=9, events_processed=0, http_input_body_size=2980144
component = HttpInputDataHandlereventtype = splunkd-loghost = ip-10-84-17-157http_input_body_size = 2980144log_level = ERRORmessage = Failed processing http input, token name=OpenBankingAggregateProd, channel=48C994DD-C1F5-462F-BAED-FC00694CF173, source_IP=10.84.31.115, reply=9, events_processed=0, http_input_body_size=2980144

0 Karma

D2SI
Communicator

Now In 7.3.4, we still have the "Parsing error : No data" error.

We now have more detailed errors in splunkd logs :

02-21-2020 10:35:22.634 +0000 ERROR HttpInputDataHandler - Failed processing http input, token name=, channel=, source_IP=1.2.3.4, reply=5, events_processed=0, http_input_body_size=0

It still being generated for multiple HEC inputs, which are working (data being ingested, no invalid token or other significant errors).

And we still do not understand why it is being generated 😕

0 Karma

srinikrishna
New Member

hi @D2SI , Even i am getting these errors, but i started noticing when i upgraded splunk from 7.0.1 to 8.0.0 and copied the same splunk_httpinput app from the old instance to new instance. Is it same case with you. are you seeing this errors after the upgrade?

0 Karma

D2SI
Communicator

Hi @srinikrishna, same here, upgraded from 7.0.x from 7.2.8, then started noticing the errors. I have activated DEBUG and I believe these errors match this kind of messages indicating that no data was processed:

01-21-2020 21:35:51.653 +0000 DEBUG HttpInputDataHandler - handled token: <token>, channel: <channel>, source IP: <ip>, reply: 9, processed: 0, http input body size: 679733

But I can see these processed: 0 messages for plenty of tokens, most of them working fine and indexing data so I am confused how to interpret this.

0 Karma

danan5
Path Finder

Hi,

Did you manage to resolve this issue and work out the root cause?

 

0 Karma
Get Updates on the Splunk Community!

Splunk Classroom Chronicles: Training Tales and Testimonials

Welcome to the "Splunk Classroom Chronicles" series, created to help curious, career-minded learners get ...

Access Tokens Page - New & Improved

Splunk Observability Cloud recently launched an improved design for the access tokens page for better ...

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

&#x1f342; Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...