Getting Data In

Ask Questions, Get Help about Data Manager for Splunk Cloud

wni
Splunk Employee
Splunk Employee

Hello from Splunk Data Manager Team,

We are excited to announce the preview of Data Manager for Splunk Cloud. Before you search through previous conversations looking for assistance, we want to provide you with some basic information and quick resources.

Want to access product docs? User Manual offers detailed guidance on each stage of using Data Manager. 

Want to request more features? Add your ideas and vote on other ideas at Data Manager Ideas Portal 

Want to search for a solution? Get answers from other Splunk customers & experts on the Data Manager Forum.

Please reply to this thread for any questions or get extra help!

Labels (1)

amell
Engager

Hello. We are considering implementing data manager multi-account model. Can we confirm what the maximum number of aws accounts supported is? Is data manager likely to be performant if we configure 200+ aws accounts within the organization. I cant find anything in the documentation, its not clear to me if its designed to do this, because it already feels slow with 30 aws accounts.

If not, do we need to consider the alternative approach of ingesting an organizational cloudtrail/guardduty/iam/sechub etc feed into splunk from a consolidation account?

 

Appreciate feedback. thanks

0 Karma

sureshV
Splunk Employee
Splunk Employee

Hello, 

The Data Manager app does not have a limit on the total number of data accounts that can be onboarded in a multi account input.

Please share some more details on where the slowness of the app is being observed for 30 accounts or more? Is slowness is on the app (UI) or on the data ingestion side or on the template deployment ? 

Please note that some API's are expected to take more time depending on the total number of accounts and regions since the app will query AWS API's from all these accounts and regions. Hence there will be little lag on the UI when viewing input details. 

The AWS CloudFormation deployment time depends on the number of accounts and regions the stack set is deploying and it is not related to DM app. 

If you already have centralized logging accounts for CloudTrail, GuardDuty, IAM and Security Hub, then it is best to onboard just those accounts since you don't have to go through setting the pre-requisites in all of your data accounts.


0 Karma

jpatcg
Engager

Do not know what Data Manager is for, appears to be something that we didn't have before, but now getting this alert from Splunk.

 

Hello Splunk Admin,
There is 1 app that has Python issues on sh-i-07250f7cd46a5ce76.cybergrants.splunkcloud.com stack that needs your attention. Please check the Upgrade Readiness App for more details on addressing outstanding items.

This app is not compatible?  What needs to be done to make this stop alerting.

om
Splunk Employee
Splunk Employee

Hello

Data Manager is an application that is now available with the Victoria experience  - This app provides a simplified and an automated way to onboard cloud data. More details can be found in the documentation at https://docs.splunk.com/Documentation/DM/1.3.1

As for the alert you are receiving, it is a false alert from the Upgrade Readiness App. The Data Manager app is completely py3 compatible and can be safely used. A new version of Upgrade Readiness App will stop these false alerts from occurring; until then you can dismiss the Data Manager App completely in the Upgrade Readiness App. We apologize for the false alerts.

0 Karma

amell
Engager

Im also finding this annoying issue. Can we please have some clarity on what is going on with this.

0 Karma

yogeshgs
Splunk Employee
Splunk Employee

We are working on a fix being rolled out to stop sending these false alerts. Thanks for your patience. 

Splunk Product Management, Getting Data In
0 Karma

boss6
Loves-to-Learn

Any plans on adding a generic REST input add-on for the Data Manager?  I know lots of people that are looking for this functionality and get frustrated, since there is nothing out there.   

0 Karma

boss6
Loves-to-Learn

Following up on my previous post - I'd also like to see the ability for this REST call to create lookups.  The majority of my REST calls end up getting indexed, but that's only because there is no current method to make the external REST call and then simply format it and send it to a csv lookup file.  

0 Karma

ibilling
Splunk Employee
Splunk Employee

Hello!

Just to clarify understand your question here, are you looking for either

  • a way to call a REST endpoint (send) on Splunk to ingest logs, like HEC
  • Splunk itself to call a configurable REST endpoint that returns data for Splunk to ingest
0 Karma

boss6
Loves-to-Learn

Hi,

I'm looking for a way for Splunk to call an external REST endpoint (a vendor, for example) and then index that data.  For on-prem, that was typically done with add-ons, but for Cloud, it's never been allowed.  I'd like to see that functionality added, and it looks like the Data Manager would be a good place for it.  

0 Karma

yogeshgs
Splunk Employee
Splunk Employee

Hi @boss6 , thank you for this suggestion. We are looking into the possibility of providing a REST connector that can act as web hook recipient to an external REST endpoint. This is however not yet tied to a release.

Would you be able to please describe this feature request in your words here so we can track it and other users can vote for it too?

Thanks,

Yogesh (Splunk Product Mgt)

Splunk Product Management, Getting Data In
0 Karma
Get Updates on the Splunk Community!

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...