Splunk App for AWS: How to resolve error "Data collection for aws_cloudwatch_logs is not fully configured. Doing nothing and quitting the TA."?

[yong0622]

I've configured everything according to the instructions given by Splunk (front-end and Backend). Right permissions were also set up according to the instructions that were provided through Splunk docs. I am trying to make Cloudwatch service of AWS work with Splunk. However, whenever I try to see if we are getting data by looking at the internal log of Splunk for this (typed in the Searchbox: "sourcetype=aws:cloudwatch") I keep getting results that our Data collection for aws_cloudwatch is not fully configured.

How can we solve this? Anyone encountering the same problem. I've been really having a hard time making AWS data get into Splunk.

Thank you in advance.

[nickhills]

Where are you performing the aws data collection?

You say front end/backend. Do I presume you mean a search head (front end) and an indexer(backend), or are you using a heavy forwarder?

If the AWS collection is not happening on your search head you may want to configure the remote endpoint inside the AWS app.
This tells your search head that a remote system is doing the collection and should remove those nag messages.

./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -set -host <search_head_ip> -port <search_head_mgmt_port> -username <username> -password <password> -t_host <target_forwarder_ip> -t_username <target_username> -t_password <target_password> -t_port <target_mgmt_port>

http://docs.splunk.com/Documentation/AWS/5.0.1/Installation/Installon-prem

[yong0622]

So I did what you recommended to me. I did set up our search head and heavy forwarder by executing this command. However, it got more complicated for me. I want to understand HOW SH and HF are communicating with each other once this was executed. So my configured inputs that were there before were gone when I connected HF and SH. We did some test and we found out that whatever we configured on the GUI, gets written on the HF TA inputs.conf file, as well as the accounts. However, when I was adding role on the Add-on, that only gets written on SH. This is where we are so far. We configured some inputs, but we have more errors now than before, my cloudtrail works before, but now not at all. How do we configure our inputs? How do we make sure that this connection helps to make the AWS app work? Anyone else having this kind of same problem? I can't get any data at all. Please help me solve this issue

[nickhills]

ok, so if you have connected your SH and HF using the command above, login to your SH and goto "Configure" in the AWS App (not the TA)

Click "Add AWS Account" and enter a name, and the secret/access keys for an IAM user with the correct permissions.
http://docs.splunk.com/Documentation/AWS/5.0.0/Installation/ConfigureyourAWSpermissions

When you have done this the changes will be pushed to the HF.

Then configure your various inputs in the AWS app (not the TA) as required, and when you save these too will be saved to the HF (you can see these if you look in the inputs.conf on the HF.)

From that point on, your HF should start collecting your logs.

[yong0622]

So I did that, figured out that that's how HF and SH work along together. However, I am still having problems with Cloudwatch. We configured everything according tot he documents (both Splunk and AWS permissions) but I still get these messages from our logs when I am doing the search "sourcetype=aws:cloudwatch" to see if we are getting data in.

"No data input has been configured, exiting.."
"Not data collection tasks for aws_cloudwatch is discovered. Doing nothing and quitting the TA."

What do these messages mean? Our inputs.conf are good, i don't understand what data collection tasks mean..
Please let me know.
Thank you for always responding.

[nickhills]

are you getting any aws logs? sourcetype=aws:*

also are you seeing those log lines in the _internal index or somewhere else?

[yong0622]

Yes i am getting logs when I type that, which is the reason why I see those messages ""No data input has been configured, exiting.."
"Not data collection tasks for aws_cloudwatch is discovered. Doing nothing and quitting the TA."

And they are all coming from _internal index.

[georgec24]

Hello, has anyone figured out this issue? I face something similar. I have an Enterprise instance in an EC2 instance (all in one box, free trial) and trying to get CloudTrail logs to it using the "Splunk Add-on for AWS" (S3 Bucket > Event Notification > SNS > SQS > EC2 Instance with IAM Role ). In the logs from _internal I see that the files are picked up from S3 ( message="Wrote data to STDOUT success.", message="Sent data for indexing.",  message="Delete SQS message" etc. ) but then I get only these messages:  message="No data input has been configured, exiting..." and  message="Not data collection tasks for aws_description is discovered. Doing nothing and quitting the TA.". The CloudTrail logs do not show up in main indexer or anywhere else so everything is lost somewhere after this <<message="Sent data for indexing.">>

Again, everything is in one box in EC2 (Splunk Enterprise free trial). If anyone has a solution to this, it would be greatly appreciated, thanks!