Ask Questions, Get Help about Data Manager for Splunk Cloud

[wni]

Hello from Splunk Data Manager Team,

We are excited to announce the preview of Data Manager for Splunk Cloud. Before you search through previous conversations looking for assistance, we want to provide you with some basic information and quick resources.

Want to access product docs? User Manual offers detailed guidance on each stage of using Data Manager. 

Want to request more features? Add your ideas and vote on other ideas at Data Manager Ideas Portal 

Want to search for a solution? Get answers from other Splunk customers & experts on the Data Manager Forum.

Please reply to this thread for any questions or get extra help!

[drobMT]

What is the plan to have the Data Manager AWS Metadata input support all the sources that are supported by the Splunk Add-on for AWS?

[AKomorowski]

Hi, thanks for posting! This is definitely the plan, as we want to achieve widely understood parity between DM and AWS TA. This year, we plan to go live with AWS Org support. Next will be availability on CMP and custom source types for S3. After that, we'll take on more items to create full parity.

[Fola]

Hi Team,

Trying to implement Data Manager for AWS data ingestion for Lambda and I am at the point of deploying the CloudFormation Stack in both ap-southeast-2 region (where I have my resources) and us-east-1 region (As recommended as the first to be deployed).

The us-east-1 deployment succeeded with CREATE_COMPLETE while the ap-southeast-2 region has refused to deploy successfully just as always completed with the ROLLBACK_COMPLETE message.

Please, can someone point me to why I am having issue or what I am doing wrong?

Thank you.

[graceville]

Is there a way to subscribe to be notified when a change is made specifically to Splunk/data manager and the stackset? 

How do we know when something changes? is there a release notes or changelog we can subscribe to?

Thanks

[yogeshgs]

When there is a new release of Data Manager, your Cloud tenant will automatically receive it.

Release notes for every new release are published in the Docs here: https://docs.splunk.com/Documentation/DM/1.8.1/ReleaseNotes/NewFeatures

[branflakes]

Hello,

We see an issue now where the Data Manager UI REQUIRES that your AWSStackSetExecutionRole allows all actions and resources, or you cannot proceed through the creation process. We have a least privilege AWSStackSetExecutionRole that can accommodate Data Manager without issue. Can the check in the UI be ignored somehow? Thanks! 

[sureshV]

Hi, currently there is no way to by-pass the validation/check on the UI for the AWSStackSetExecutionRole in the current version of the Data Manager. This will be addressed in the future release of Data Manager app. 

For now, the only workaround is to grant full permissions(*) for the AWSStackSetExecutionRole temporarily till the onboarding on UI is complete and then update the AWSStackSetExecutionRole with granular permissions instead of *.  

[amell]

Hello. We are considering implementing data manager multi-account model. Can we confirm what the maximum number of aws accounts supported is? Is data manager likely to be performant if we configure 200+ aws accounts within the organization. I cant find anything in the documentation, its not clear to me if its designed to do this, because it already feels slow with 30 aws accounts.

If not, do we need to consider the alternative approach of ingesting an organizational cloudtrail/guardduty/iam/sechub etc feed into splunk from a consolidation account?

Appreciate feedback. thanks

[sureshV]

Hello, 

The Data Manager app does not have a limit on the total number of data accounts that can be onboarded in a multi account input.

Please share some more details on where the slowness of the app is being observed for 30 accounts or more? Is slowness is on the app (UI) or on the data ingestion side or on the template deployment ? 

Please note that some API's are expected to take more time depending on the total number of accounts and regions since the app will query AWS API's from all these accounts and regions. Hence there will be little lag on the UI when viewing input details. 

The AWS CloudFormation deployment time depends on the number of accounts and regions the stack set is deploying and it is not related to DM app. 

If you already have centralized logging accounts for CloudTrail, GuardDuty, IAM and Security Hub, then it is best to onboard just those accounts since you don't have to go through setting the pre-requisites in all of your data accounts.

[jpatcg]

Do not know what Data Manager is for, appears to be something that we didn't have before, but now getting this alert from Splunk.

Hello Splunk Admin,
There is 1 app that has Python issues on sh-i-07250f7cd46a5ce76.cybergrants.splunkcloud.com stack that needs your attention. Please check the Upgrade Readiness App for more details on addressing outstanding items.

This app is not compatible?  What needs to be done to make this stop alerting.

[om]

Hello

Data Manager is an application that is now available with the Victoria experience  - This app provides a simplified and an automated way to onboard cloud data. More details can be found in the documentation at https://docs.splunk.com/Documentation/DM/1.3.1

As for the alert you are receiving, it is a false alert from the Upgrade Readiness App. The Data Manager app is completely py3 compatible and can be safely used. A new version of Upgrade Readiness App will stop these false alerts from occurring; until then you can dismiss the Data Manager App completely in the Upgrade Readiness App. We apologize for the false alerts.

[amell]

Im also finding this annoying issue. Can we please have some clarity on what is going on with this.

[yogeshgs]

We are working on a fix being rolled out to stop sending these false alerts. Thanks for your patience. 

[boss6]

Any plans on adding a generic REST input add-on for the Data Manager?  I know lots of people that are looking for this functionality and get frustrated, since there is nothing out there.   

[boss6]

Following up on my previous post - I'd also like to see the ability for this REST call to create lookups.  The majority of my REST calls end up getting indexed, but that's only because there is no current method to make the external REST call and then simply format it and send it to a csv lookup file.  

[ibilling]

Hello!

Just to clarify understand your question here, are you looking for either

• a way to call a REST endpoint (send) on Splunk to ingest logs, like HEC
• Splunk itself to call a configurable REST endpoint that returns data for Splunk to ingest

[boss6]

Hi,

I'm looking for a way for Splunk to call an external REST endpoint (a vendor, for example) and then index that data.  For on-prem, that was typically done with add-ons, but for Cloud, it's never been allowed.  I'd like to see that functionality added, and it looks like the Data Manager would be a good place for it.  

[yogeshgs]

Hi @boss6 , thank you for this suggestion. We are looking into the possibility of providing a REST connector that can act as web hook recipient to an external REST endpoint. This is however not yet tied to a release.

Would you be able to please describe this feature request in your words here so we can track it and other users can vote for it too?

Thanks,

Yogesh (Splunk Product Mgt)

[New_splunkie]

Hi @yogeshgs ,

My splunk cloud instance does not have Data manager app and as per my understanding it ships with instance and cant be installed seperately.

Can you guide what to do in this case if I need this Data Manager for my instance.

Any response will be appreciated and thanks in advance. 

[AKomorowski]

Hi @New_splunkie,

It's great to see your interest in Data Manager! You're absolutely correct – Data Manager is a native app to the Splunk Cloud Platform, which means there's no separate installation required.

However, there are a few requirements that your deployment must meet in order to have it on your Splunk Cloud Platform. 

You can use Data Manager if your Splunk Cloud Platform deployment meets the following requirements:

Runs Splunk Cloud Platform versions 8.2.2104.1 and higher on the Victoria experience.
Is provisioned in a region that supports Data Manager. See Available regions and region differences in the Splunk Cloud Platform Service Description.

Please let me know if that helps!

Antoni