topic Re: Powershell Script Input via JSON not parsing correctly? in Getting Data In

Powershell Script Input via JSON not parsing correctly?

Stephan_BP — Tue, 26 Jul 2022 14:35:37 GMT

Hi 🙂

i have a curious problem. (btw. not my first Powershell input 🙂 )

I am trying to Input some Active Directory Data into Splunk right now. Below a bit changed output of my Script:

[ { "SpecialUsers_S": false, "SpecialUsers_X": false, "SpecialUsers_U": false, "SpecialUsers_A": false, "SpecialUsers_TBM": false, "SpecialUsers_T": false, "HR_Canceled_Users": false, "HR_Inactive_Users": false, "HR_Temporary-Inactive_Users": false, "FehlerStatus": "0", "PasswordNeverExpires_State": "null", "OU_State": "null", "Account_State": "null", "Manager_State": "null", "Account_Expiration_Date": "null", "EmployeeNumberError": "null", "DescriptionError": "null", "ManagersViaGroup": "null", "Wrong_Name": "null", "Wrong_EMail": "null", "Manager_Description": "null", "Multiple_SpecialGroups": "null", "Multiple_HR_Groups": "null", "SamAccountName": "SamAccount01", "Enabled": true, "EmployeeNumber": "11112", "SN": "Surname01", "Description": "0200000000", "Department": "Department01", "Company": "The Firm", "emailaddress": "Email01@domain.com", "DistinguishedName": "The Distinguished Name 01", "hkDS-EntryDate": "09.09.1991 02:00:00", "LastLogonDate": "18.07.2022 07:22:38", "PasswordLastSet": "02.06.2022 09:22:36" }, { "SpecialUsers_S": false, "SpecialUsers_X": false, "SpecialUsers_U": false, "SpecialUsers_A": false, "SpecialUsers_TBM": false, "SpecialUsers_T": false, "HR_Canceled_Users": false, "HR_Inactive_Users": false, "HR_Temporary-Inactive_Users": false, "FehlerStatus": "0", "PasswordNeverExpires_State": "null", "OU_State": "null", "Account_State": "null", "Manager_State": "null", "Account_Expiration_Date": "null", "EmployeeNumberError": "null", "DescriptionError": "null", "ManagersViaGroup": "null", "Wrong_Name": "null", "Wrong_EMail": "null", "Manager_Description": "null", "Multiple_SpecialGroups": "null", "Multiple_HR_Groups": "null", "SamAccountName": "SamAccount02", "Enabled": true, "EmployeeNumber": "11113", "SN": "Surname02", "Description": "000000000", "Department": "Department02", "Company": "The Firm", "emailaddress": "email02@Domain.com", "DistinguishedName": "The Distinguished Name 01", "hkDS-EntryDate": "10.10.2002 02:00:00", "LastLogonDate": "18.07.2022 08:07:31", "PasswordLastSet": "26.05.2022 17:27:42" } ]

Exported into File and testet with Validators all is fine.

But what i see in Splunk is:

"SpecialUsers_S": false, "SpecialUsers_X": false, "SpecialUsers_U": false, "SpecialUsers_A": false, "SpecialUsers_TBM": false, "SpecialUsers_T": false, "HR_Canceled_Users": false, "HR_Inactive_Users": false, "HR_Temporary-Inactive_Users": false, "FehlerStatus": "0", "PasswordNeverExpires_State": "null", "OU_State": "null", "Account_State": "null", "Manager_State": "null", "Account_Expiration_Date": "null", "EmployeeNumberError": "null", "DescriptionError": "null", "ManagersViaGroup": "null", "Wrong_Name": "null", "Wrong_EMail": "null", "Manager_Description": "null", "Multiple_SpecialGroups": "null", "Multiple_HR_Groups": "null", "SamAccountName": "SamAccount01", "Enabled": true, "EmployeeNumber": "null", "SN": "", "Description": "null", "Department": "null", "Company": "", "emailaddress": null, "DistinguishedName": "The Distinguished Name", "hkDS-EntryDate": "null", "LastLogonDate": "null", "PasswordLastSet": "null" }

As u can see i am missing a lot of information, and i cant figure out why... Some like SamAccountName and DistinguishedName is working but other variables like Company, Department or Description are missing...

Skript is rather long but if needed i can post Parts of it how i do stuff 🙂

the inputs.conf for this is:

[powershell://Get_AD_Report] script = . "$SplunkHome\etc\system\bin\Powershell\GetADReport.ps1" schedule=15 * * * * sourcetype=_json index=hk_office365

Maybe someone as some kind of clue whats happening there for me?

Would really help 🙂 am on this for much to long already 😉 and tried so many different ways now...

Re: Powershell Script Input via JSON not parsing correctly?

Stephan_BP — Mon, 12 Sep 2022 13:18:54 GMT

No One? 🙂 at least a discussion would be nice... i can also see errors in the internal log... some parsing / linebreaker errors... but i cant find those on the file if i output vie | out-file <filename> from powershell... the resulting .json is ok according to testtools and i cant find any problems... i am at a loss...

Errors:

had parsing error:Unexpected character while looking for value: ',' - data_source="powershell://Get_AD_Report", data_host="nope", data_sourcetype="_json"

JSON StreamId:2583583993661161172 had parsing error:Unexpected character while looking for value: '}' - data_source="powershell://Get_AD_Report", data_host=" nope ", data_sourcetype="_json"

JSON StreamId:2583583993661161172 had parsing error:Unexpected character while parsing backslash escape: 'x' - data_source="powershell://Get_AD_Report", data_host=" nope ", data_sourcetype="_json"

JSON StreamId:2583583993661161172 had parsing error:Unexpected character while looking for value: ']' - data_source="powershell://Get_AD_Report", data_host=" nope ", data_sourcetype="_json"

So yes i would also say there should be a /x somewhere in the data... cant find it after i output it... i "think" its a encoding/decoding problem of some kind... i assume powershell direkt output is some Charset and maybe i just have to find the right Charset in Splunk... but somehow i simply dont get my actual problem 🙂

Re: Powershell Script Input via JSON not parsing correctly?

Stephan_BP — Tue, 20 Sep 2022 08:34:16 GMT

Ok, i gotten a bit further now... it seems to work properly "ONCE" after i restart the SplunkForwarderService, but the next scheduled time it executes it goes back to leave data out of the events... cant see errors anymore either...

I am really confused by now 🙂 maybe someone seeing this behavior?

Props.conf on Universalforwarder thats working "ONCE" after Restart:

[_json2]
pulldown_type = true
INDEXED_EXTRACTIONS = json
AUTO_KV_JSON = false
TRUNCATE = 100000
CHARSET = utf-8
KV_MODE = none
category = Structured

Re: Powershell Script Input via JSON not parsing correctly?

scelikok — Tue, 20 Sep 2022 08:41:50 GMT

Hi @Stephan_BP,

Can you please try writing output without an array? I mean each result will be a single JSON that contains only one account.

Re: Powershell Script Input via JSON not parsing correctly?

Stephan_BP — Wed, 21 Sep 2022 07:29:50 GMT

Hi scelikok,

not sure i get what u mean 🙂

Yes my Output of the Powershellscript is in the Format of:

[ { User1values}, {User2values}, ... {UserXvalues} ]

and this might be a problem?

Splunk correctly creates the events like i want it to For Each User 1 Event. From this point of view i feel its working fine...

I simply dont get why the values are wrong... and not only wrong... simply the same for each User except for a few fields...

and right after a Universalforwarderservice restart it seems to work now but only there... (except i have double extractions now and have to fix this now aswell) ... each scheduled start of the script it again gives me wrong values...