All Apps and Add-ons

Unable to get Azure Activity Log, Azure Diagnostic Logs and Metrics in Azure addon for splunk.

lmorillogonzazl
Explorer

Hi, it has been detected problems when it comes to communicating azure with the splunk tool, I have already configured azure and splunk, but they fail to connect, it gives the several errors, for example:

ERROR ExecProcessor - message from 'python //$SPLUNK_HOME/TA-Azure_Monitor/bin/azure_monitor_metrics.py' Error caught in get_metrics_for_subscription, type: , value: Get Token request returned http error: 400 [Problem start date and time].
Has anybody solve this?.

1 Solution

jconger
Splunk Employee
Splunk Employee

Based on the error, there may be a problem with your Azure AD application or Key Vault permissions.

Here are some blogs that step you through the setup for the Azure side and the Splunk side:
https://www.splunk.com/blog/2018/04/20/splunking-microsoft-azure-monitor-data-part-1-azure-setup.htm...
https://www.splunk.com/blog/2018/05/07/splunking-microsoft-azure-monitor-data-part-2-splunk-setup.ht...

Scripts are also available that will create all you need on the Azure side and print out a list of which parameters to input on the Splunk side. Check them out here -> https://github.com/Microsoft/AzureMonitorAddonForSplunk/tree/master/scripts
There is a PowerShell script and a bash script. You can choose either one as they both do the same thing.

Also note that Activity Log and Diagnostic Log data inputs use AMQP to connect to event hub over TLS using ports 5671 / 5672 as described in the AMQP 1.0 Service Bus and Event Hubs protocol guide. So, if you are having connection/authentication issues, check that these ports are open on your Splunk instance.

View solution in original post

0 Karma

jconger
Splunk Employee
Splunk Employee

Based on the error, there may be a problem with your Azure AD application or Key Vault permissions.

Here are some blogs that step you through the setup for the Azure side and the Splunk side:
https://www.splunk.com/blog/2018/04/20/splunking-microsoft-azure-monitor-data-part-1-azure-setup.htm...
https://www.splunk.com/blog/2018/05/07/splunking-microsoft-azure-monitor-data-part-2-splunk-setup.ht...

Scripts are also available that will create all you need on the Azure side and print out a list of which parameters to input on the Splunk side. Check them out here -> https://github.com/Microsoft/AzureMonitorAddonForSplunk/tree/master/scripts
There is a PowerShell script and a bash script. You can choose either one as they both do the same thing.

Also note that Activity Log and Diagnostic Log data inputs use AMQP to connect to event hub over TLS using ports 5671 / 5672 as described in the AMQP 1.0 Service Bus and Event Hubs protocol guide. So, if you are having connection/authentication issues, check that these ports are open on your Splunk instance.

0 Karma

rekhaagarwal60
Engager

No luck in getting the data to splunk from azure? Any one getting it?

0 Karma

psmaan
New Member

Hi,
I am trying to get similar setup done. I could not understand the requirement of having port 5671/5672 ports open on Splunk for this communication. Can you please elaborate that?, and how to do that.
In my case, splunk is initiating TLS handshake with Azure , however after handshake when Splunk tries to switch session to aqmp (by sending syn on port for aqmp), it gets the RESET from azure. I am assuming that firewall is playing dirty here as I am behind a PaloAlto, however still want to check with you.

Tcpdump capture at splunk server:
16:17:35.939016 IP 192.X.X.X.33411 > 104.208.16.3.amqps: Flags [S], seq 2945244674, win 14600, options [mss 1460,sackOK,TS val 435408421 ecr 0,nop,wscale 7], length 0
16:17:35.939190 IP 104.208.16.3.amqps > 192.X.X.X.33411: Flags [R.], seq 0, ack 2945244675, win 14600, length 0

0 Karma

thambisetty
SplunkTrust
SplunkTrust

The reason why 5671/5672 need to be opened is because The TA uses different protocol aqmp/aqmps to fetch data from azure.
The defining features of AMQP are message orientation, queuing, routing (including point-to-point and publish-and-subscribe), reliability and security.

So, you need to allow these two ports on your firewall to make successful connection to azure.

————————————
If this helps, give a like below.
0 Karma

psmaan
New Member

Yup, that`s what I was thinking. The response from jcogner is misleading when it suggest to have these ports open on Splunk instance itself. In stead, it should recommend these ports should be opened for Splunk at firewalls if there is any firewall in between.

0 Karma

MousumiChowdhur
Contributor

Hi,

Inspite of doing all the configurations as per the document, I am not able to get data in Splunk. Kindly help me with what I am missing.

Thank you!

0 Karma

N92
Path Finder

Also add-on not fetching complete logs. Some of the types of logs are missing. Ex: Audit.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

How did you get this app worked?

I have proxy in between client and server, not able to set proxy in adal-node.

Any ideas?

————————————
If this helps, give a like below.
0 Karma

MousumiChowdhur
Contributor

Hi @lmorillogonzazlez,

Even I am getting the same error. Can you help me with how exactly you solved the issue?

Thank you!

0 Karma

thambisetty
SplunkTrust
SplunkTrust

Are you still getting an error or able to collect logs? Have you tried collecting Azure activity logs?

————————————
If this helps, give a like below.
0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...