About srinikrishna

srinikrishna · ‎07-08-2020

Hi We have a splunk add-on for aws to pull the logs from s3 bucket. we are using the sqs based s3 inputs created to read the logs for s3 bucket, however we are noticing that through this option splunk service seems like ommiting some files from reading even though it has consumed the sqs message and deleted the message from the quer. I am attaching one of the example from our issue where in at particular time frame which is on july 6th 19 to 20 hrs we have 59 objects in the s3 bucket but splunk had read only 58 files. This is being one of the example to show but we are having this issue very often every hour one or 2 files missing. We have around 8000 to 10000 events in each file which is missing indexed in splunk due to this issue. I have checked all the internal logs which does not show any failure messages while reading this particular s3 object to confirm it was dropped or failed while parsing and processing. Its just not there. these issue is there every day every hour missing one or the other files missed by splunk inputs. From SQS perspective SQS based S3 input is skipping some objects from s3 but deleting the message from sqs Thanks and Regards Srini

srinikrishna · ‎07-08-2020

Hi, Looks like the message that is being kept by s3 bucket upon new object is having some data which splunk is not able to parse it. Usually message will contain the metadata of which s3 , what is the key information and what is the size etc. please share the message details from sqs get message option. And also check if the messages are going to dlq if splunk is not able to parse and process, the messages will go to dlq to take the snapshot. I believe there might be some characters which splunk does not like in the s3 bucket name or object name.

srinikrishna · ‎07-08-2020

Hi, with the generic s3 input, you do have option of giving the start date and time and the end date and time. You can use that to filter the unwanted objects to be read from s3. However if you are asking for sqs based s3, you can setup the events in s3 bucket to create the sqs message on new create object only which means only from next new object placed in s3 bucket you will have the message. with sqs based s3 this is the only option you have to set on what event you need the message to be sent. if you want to read from particular time, you need to use the generic s3 inputs.

srinikrishna · ‎01-21-2020

Hi @Paul1896 On my heavy forwarder i cant see this Monitoring console as the logs are not storing in local machine. however on the indexer i do not have monitoring console. Is there any other way to verify this? I actually got this below error and stopped ingesting logs since then. I dont see any more errors also related to this hec data input in the logs after 8.30. there are other inputs working fine. and fyi i copied this splunk_httpinput folder from my old splunk instance to new splunk instance to avoid recreating all the tokens i had earlier. does this makes any issues ? 01-21-2020 07:36:24.170 +0000 ERROR HttpInputDataHandler - Failed processing http input, token name=OpenBankingAggregateProd, channel=48C994DD-C1F5-462F-BAED-FC00694CF173, source_IP=10.84.31.115, reply=9, events_processed=0, http_input_body_size=2980144 component = HttpInputDataHandlereventtype = splunkd-loghost = ip-10-84-17-157http_input_body_size = 2980144log_level = ERRORmessage = Failed processing http input, token name=OpenBankingAggregateProd, channel=48C994DD-C1F5-462F-BAED-FC00694CF173, source_IP=10.84.31.115, reply=9, events_processed=0, http_input_body_size=2980144

srinikrishna · ‎01-21-2020

hi @D2SI , Even i am getting these errors, but i started noticing when i upgraded splunk from 7.0.1 to 8.0.0 and copied the same splunk_httpinput app from the old instance to new instance. Is it same case with you. are you seeing this errors after the upgrade?

srinikrishna · ‎11-08-2019

You no longer will be able to see the course after expiry date. You may have to re-register with different user credentials?

srinikrishna · ‎11-07-2019

If your indexers are on the same splunk Enterprise setup you should get the indexers in that list. If your Enterprise splunkbwhere dbconnecr is installed is an heavy forwarder, You can manually enter the indexer you want this data to go.

srinikrishna · ‎11-06-2019

Hi, What is the splunk setup you are using, cloud instance or standalone splunk enterprise? I assume its standalon splunk enterprise instance. If so usually you can open the splunk login page @ http://localhost:8000/ if you are giving some other url , it may give this message. if not localhost you should give your server name or the ip address. if you have enabled ssl you may have to give https. Was it a new installation? or this started after you did some upgrade or something? Regards

Posts	8
Solutions	0
Karma Given	0
Karma Received	0
Member Since	‎11-06-2019

Online Status	Offline
Date Last Visited	‎04-12-2023 11:55 AM

SQS based S3 input is skipping some objects from s...

SQS based S3 input is skipping some objects from s...

Re: AWS Add on AWS-- Issue with SQS-Based S3 and c...

Re: AWS SQS Based S3 input - ignore historical dat...

Re: HttpInputDataHandler - Parsing error : No data

Re: HttpInputDataHandler - Parsing error : No data

Re: Fundamentals 1 expired

Re: DB Connect: defining indexer

Re: Ooops Page not found when I login