All Apps and Add-ons

Azure AD user data (azure:aad:user) is ingesting the first 100 users over and over again every second.

def65483
Explorer

I have logging set to debug. nothing interesting except that it is pulling in the exact same skip token (100 users) every second nonstop.
I have completely removed the input and made another with a new name. But, it does the exact same thing. No errors, just the same graph call every.... single... second...

1 Solution

sleclerc1
Explorer

Found a solution, was working with version 2.1.0:

This code is a snippet from "input_module_MS_AAD_user.py", located at $SPLUNK_HOME/etc/apps/TA-MS-AAD/bin

First, I found the syntax error in the second item, "nusers...". After changing that, the query was working successfully, but returning no data. I then tried the v1.0 version and success! Our organization has >600K users and the query took over an hour.

EDIT:

After additional troubleshooting I simplified the answer even more. I thought I had found two problems, but actually only one.

The second part I thought that also needed fixing was the version of the API it was using. I thought that v1.0 had to be used, and while it does work, and pulls significantly less fields than it's beta variant, and is not compatible with the Microsoft Azure App for Splunk. I thought it wasn't working because the BETA API responses were yielding an HTTP 200 w/ 0 bytes transferred....but reviewing logs in our Splunk environment, it was successfully querying/ingesting the data.

View solution in original post

0 Karma

sleclerc1
Explorer

Found a solution, was working with version 2.1.0:

This code is a snippet from "input_module_MS_AAD_user.py", located at $SPLUNK_HOME/etc/apps/TA-MS-AAD/bin

First, I found the syntax error in the second item, "nusers...". After changing that, the query was working successfully, but returning no data. I then tried the v1.0 version and success! Our organization has >600K users and the query took over an hour.

EDIT:

After additional troubleshooting I simplified the answer even more. I thought I had found two problems, but actually only one.

The second part I thought that also needed fixing was the version of the API it was using. I thought that v1.0 had to be used, and while it does work, and pulls significantly less fields than it's beta variant, and is not compatible with the Microsoft Azure App for Splunk. I thought it wasn't working because the BETA API responses were yielding an HTTP 200 w/ 0 bytes transferred....but reviewing logs in our Splunk environment, it was successfully querying/ingesting the data.

0 Karma

def65483
Explorer

I did something similar by just editing the line in place to force the next page link to the one in the manifest i saved down. It's annoying, but i'm happy someone else is seeing this issue. it is very odd each page is referencing itself 😕

I've asked some MS peeps as well. I'll post an update as soon as i hear anything back.

0 Karma

sleclerc1
Explorer

To my understanding, the syntax error I found caused the API to loop the original call, rather than actually use the next page link.

0 Karma

def65483
Explorer

Also, your attachment is not showing. :(. I suspect you are referring to line: 54 and fixing so it creates the user response instead of "nuser" response.

I had a similar issue when changing it where i still had a previous shim in place and made it fail entirely but yea. thats the only part i see. big oof there.

0 Karma

sleclerc1
Explorer

Yeah looks like the image was uploaded to a "temp" location, so I guess it was deleted. But yes, you are correct.

Nothing like a single character syntax error to break the entire API haha

0 Karma

def65483
Explorer

ahhh i see that now. lol whyyyyyy

0 Karma

def65483
Explorer

I'll note that the skip token pull and the @odata.nextlink are always the same (itself).

And now that i look at it closer its more like 2-3 times per second (depending on how fast i can pull 100 users)

0 Karma

sleclerc1
Explorer

Same issue. Been trying to dig in to odata.nextlink and pagination, but haven't turned up anything useful to assist. I've event duplicated the issue within Graph Explorer; performing the user pull only grabs the first 100 users, but I see the pagination "attempt". I have a feeling it's on Microsoft's side... Although I hope I'm wrong.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...