All Apps and Add-ons

Splunk App for AWS: How to resolve error "Data collection for aws_cloudwatch_logs is not fully configured. Doing nothing and quitting the TA."?

New Member

I've configured everything according to the instructions given by Splunk (front-end and Backend). Right permissions were also set up according to the instructions that were provided through Splunk docs. I am trying to make Cloudwatch service of AWS work with Splunk. However, whenever I try to see if we are getting data by looking at the internal log of Splunk for this (typed in the Searchbox: "sourcetype=aws:cloudwatch") I keep getting results that our Data collection for aws_cloudwatch is not fully configured.

How can we solve this? Anyone encountering the same problem. I've been really having a hard time making AWS data get into Splunk.

Thank you in advance.

0 Karma

Ultra Champion

Where are you performing the aws data collection?

You say front end/backend. Do I presume you mean a search head (front end) and an indexer(backend), or are you using a heavy forwarder?

If the AWS collection is not happening on your search head you may want to configure the remote endpoint inside the AWS app.
This tells your search head that a remote system is doing the collection and should remove those nag messages.

./splunk cmd python ../etc/apps/splunk_app_aws/bin/cli/targets_helper.py -set -host <search_head_ip> -port <search_head_mgmt_port> -username <username> -password <password> -t_host <target_forwarder_ip> -t_username <target_username> -t_password <target_password> -t_port <target_mgmt_port>

http://docs.splunk.com/Documentation/AWS/5.0.1/Installation/Installon-prem

0 Karma

New Member

So I did what you recommended to me. I did set up our search head and heavy forwarder by executing this command. However, it got more complicated for me. I want to understand HOW SH and HF are communicating with each other once this was executed. So my configured inputs that were there before were gone when I connected HF and SH. We did some test and we found out that whatever we configured on the GUI, gets written on the HF TA inputs.conf file, as well as the accounts. However, when I was adding role on the Add-on, that only gets written on SH. This is where we are so far. We configured some inputs, but we have more errors now than before, my cloudtrail works before, but now not at all. How do we configure our inputs? How do we make sure that this connection helps to make the AWS app work? Anyone else having this kind of same problem? I can't get any data at all. Please help me solve this issue

0 Karma

Ultra Champion

ok, so if you have connected your SH and HF using the command above, login to your SH and goto "Configure" in the AWS App (not the TA)

Click "Add AWS Account" and enter a name, and the secret/access keys for an IAM user with the correct permissions.
http://docs.splunk.com/Documentation/AWS/5.0.0/Installation/ConfigureyourAWSpermissions

When you have done this the changes will be pushed to the HF.

Then configure your various inputs in the AWS app (not the TA) as required, and when you save these too will be saved to the HF (you can see these if you look in the inputs.conf on the HF.)

From that point on, your HF should start collecting your logs.

0 Karma

New Member

So I did that, figured out that that's how HF and SH work along together. However, I am still having problems with Cloudwatch. We configured everything according tot he documents (both Splunk and AWS permissions) but I still get these messages from our logs when I am doing the search "sourcetype=aws:cloudwatch" to see if we are getting data in.

"No data input has been configured, exiting.."
"Not data collection tasks for aws_cloudwatch is discovered. Doing nothing and quitting the TA."

What do these messages mean? Our inputs.conf are good, i don't understand what data collection tasks mean..
Please let me know.
Thank you for always responding.

0 Karma

Ultra Champion

are you getting any aws logs? sourcetype=aws:*

also are you seeing those log lines in the _internal index or somewhere else?

0 Karma

New Member

Yes i am getting logs when I type that, which is the reason why I see those messages ""No data input has been configured, exiting.."
"Not data collection tasks for aws_cloudwatch is discovered. Doing nothing and quitting the TA."

And they are all coming from _internal index.

0 Karma