Hi,
Splunk Version - Splunk 7.0.2 (build 03bbabbd5c0f) - Role: Heavy Forwarder
Splunk_TA_google-cloudplatform version = 1.2.0
I have configured pub/sub inputs to collect logs from a Google Cloud Platform. As per the recommendations on the Splunk Documentation below, I have created 5 cloned pub/sub inputs for throughput and performance.
https://docs.splunk.com/Documentation/AddOns/released/GoogleCloud/Troubleshoot
Large pub/sub subscriptions
For large pub/sub subscriptions, we recommend cloning existing inputs that are ingesting the same subscriptions to increase data throughput and performance. These identical inputs can be in the same instance or in different instances.
To manage a large number of subscriptions to one Splunk instance, aggregate subscriptions belonging to the same Google Cloud Service account into one input to save resources.
I see data not being indexed on and off.
Checking the pub/sub logs I found this error:
xxxx-xx-xx xx:xx:xx,xxx level=ERROR pid=2383 tid=MainThread logger=splunk_ta_gcp.modinputs.pubsub pos=pubsub.py:_try_send_data:201 | datainput="gcp_qa_pubsub_all_2" start_time=xxxxxxxxxx| message="Not enough time to send data for indexing." lapse=8.34614610672 ttl=10
What is the reason for this error? And how do we fix it?
