Powershell Script Input via JSON not parsing correctly?

[Stephan_BP]

Hi 🙂 

i have a curious problem. (btw. not my first Powershell input 🙂 ) 

I am trying to Input some Active Directory Data into Splunk right now. Below a bit changed output of my Script: 

 [
   {
        "SpecialUsers_S":  false,
        "SpecialUsers_X":  false,
        "SpecialUsers_U":  false,
        "SpecialUsers_A":  false,
        "SpecialUsers_TBM":  false,
        "SpecialUsers_T":  false,
        "HR_Canceled_Users":  false,
        "HR_Inactive_Users":  false,
        "HR_Temporary-Inactive_Users":  false,
        "FehlerStatus":  "0",
        "PasswordNeverExpires_State":  "null",
        "OU_State":  "null",
        "Account_State":  "null",
        "Manager_State":  "null",
        "Account_Expiration_Date":  "null",
        "EmployeeNumberError":  "null",
        "DescriptionError":  "null",
        "ManagersViaGroup":  "null",
        "Wrong_Name":  "null",
        "Wrong_EMail":  "null",
        "Manager_Description":  "null",
        "Multiple_SpecialGroups":  "null",
        "Multiple_HR_Groups":  "null",
        "SamAccountName":  "SamAccount01",
        "Enabled":  true,
        "EmployeeNumber":  "11112",
        "SN":  "Surname01",
        "Description":  "0200000000",
        "Department":  "Department01",
        "Company":  "The Firm",
        "emailaddress":  "Email01@domain.com",
        "DistinguishedName":  "The Distinguished Name 01",
        "hkDS-EntryDate":  "09.09.1991 02:00:00",
        "LastLogonDate":  "18.07.2022 07:22:38",
        "PasswordLastSet":  "02.06.2022 09:22:36"
    },
    {
        "SpecialUsers_S":  false,
        "SpecialUsers_X":  false,
        "SpecialUsers_U":  false,
        "SpecialUsers_A":  false,
        "SpecialUsers_TBM":  false,
        "SpecialUsers_T":  false,
        "HR_Canceled_Users":  false,
        "HR_Inactive_Users":  false,
        "HR_Temporary-Inactive_Users":  false,
        "FehlerStatus":  "0",
        "PasswordNeverExpires_State":  "null",
        "OU_State":  "null",
        "Account_State":  "null",
        "Manager_State":  "null",
        "Account_Expiration_Date":  "null",
        "EmployeeNumberError":  "null",
        "DescriptionError":  "null",
        "ManagersViaGroup":  "null",
        "Wrong_Name":  "null",
        "Wrong_EMail":  "null",
        "Manager_Description":  "null",
        "Multiple_SpecialGroups":  "null",
        "Multiple_HR_Groups":  "null",
        "SamAccountName":  "SamAccount02",
        "Enabled":  true,
        "EmployeeNumber":  "11113",
        "SN":  "Surname02",
        "Description":  "000000000",
        "Department":  "Department02",
        "Company":  "The Firm",
        "emailaddress":  "email02@Domain.com",
        "DistinguishedName":  "The Distinguished Name 01",
        "hkDS-EntryDate":  "10.10.2002 02:00:00",
        "LastLogonDate":  "18.07.2022 08:07:31",
        "PasswordLastSet":  "26.05.2022 17:27:42"
    }
]

 Exported into File and testet with Validators all is fine. 

But what i see in Splunk is: 

        "SpecialUsers_S":  false,
        "SpecialUsers_X":  false,
        "SpecialUsers_U":  false,
        "SpecialUsers_A":  false,
        "SpecialUsers_TBM":  false,
        "SpecialUsers_T":  false,
        "HR_Canceled_Users":  false,
        "HR_Inactive_Users":  false,
        "HR_Temporary-Inactive_Users":  false,
        "FehlerStatus":  "0",
        "PasswordNeverExpires_State":  "null",
        "OU_State":  "null",
        "Account_State":  "null",
        "Manager_State":  "null",
        "Account_Expiration_Date":  "null",
        "EmployeeNumberError":  "null",
        "DescriptionError":  "null",
        "ManagersViaGroup":  "null",
        "Wrong_Name":  "null",
        "Wrong_EMail":  "null",
        "Manager_Description":  "null",
        "Multiple_SpecialGroups":  "null",
        "Multiple_HR_Groups":  "null",
        "SamAccountName":  "SamAccount01",
        "Enabled":  true,
        "EmployeeNumber":  "null",
        "SN":  "",
        "Description":  "null",
        "Department":  "null",
        "Company":  "",
        "emailaddress":  null,
        "DistinguishedName":  "The Distinguished Name",
        "hkDS-EntryDate":  "null",
        "LastLogonDate":  "null",
        "PasswordLastSet":  "null"
    }

As u can see i am missing a lot of information, and i cant figure out why... Some like SamAccountName and DistinguishedName is working but other variables like Company, Department or Description are missing... 

Skript is rather long but if needed i can post Parts of it how i do stuff 🙂 

the inputs.conf for this is: 

[powershell://Get_AD_Report]
script = . "$SplunkHome\etc\system\bin\Powershell\GetADReport.ps1"
schedule=15 * * * *
sourcetype=_json
index=hk_office365

Maybe someone as some kind of clue whats happening there for me? 

Would really help 🙂 am on this for much to long already 😉 and tried so many different ways now... 

[scelikok]

Hi @Stephan_BP,

Can you please try writing output without an array? I mean each result will be a single JSON that contains only one account.

[Stephan_BP]

Hi scelikok, 

not sure i get what u mean 🙂 

Yes my Output of the Powershellscript is in the Format of: 

[ { User1values}, {User2values}, ... {UserXvalues} ]

and this might be a problem? 

Splunk correctly creates the events like i want it to For Each User 1 Event. From this point of view i feel its working fine... 

I simply dont get why the values are wrong... and not only wrong... simply the same for each User except for a few fields...

and right after a Universalforwarderservice restart it seems to work now but only there... (except i have double extractions now and have to fix this now aswell) ... each scheduled start of the script it again gives me wrong values... 

[Stephan_BP]

Ok, i gotten a bit further now... it seems to work properly "ONCE" after i restart the SplunkForwarderService, but the next scheduled time it executes it goes back to leave data out of the events... cant see errors anymore either... 

I am really confused by now 🙂 maybe someone seeing this behavior? 

Props.conf on Universalforwarder thats working "ONCE" after Restart: 

[_json2]
pulldown_type = true
INDEXED_EXTRACTIONS = json
AUTO_KV_JSON = false
TRUNCATE = 100000
CHARSET = utf-8
KV_MODE = none
category = Structured

[Stephan_BP]

No One? 🙂 at least a discussion would be nice... i can also see errors in the internal log... some parsing / linebreaker errors... but i cant find those on the file if i output vie | out-file <filename> from powershell... the resulting .json is ok according to testtools and i cant find any problems... i am at a loss... 

Errors: 

had parsing error:Unexpected character while looking for value: ',' - data_source="powershell://Get_AD_Report", data_host="nope", data_sourcetype="_json" 

JSON StreamId:2583583993661161172 had parsing error:Unexpected character while looking for value: '}' - data_source="powershell://Get_AD_Report", data_host=" nope ", data_sourcetype="_json" 

JSON StreamId:2583583993661161172 had parsing error:Unexpected character while parsing backslash escape: 'x' - data_source="powershell://Get_AD_Report", data_host=" nope ", data_sourcetype="_json" 

JSON StreamId:2583583993661161172 had parsing error:Unexpected character while looking for value: ']' - data_source="powershell://Get_AD_Report", data_host=" nope ", data_sourcetype="_json" 

So yes i would also say there should be a /x somewhere in the data... cant find it after i output it... i "think" its a encoding/decoding problem of some kind... i assume powershell direkt output is some Charset and maybe i just have to find the right Charset in Splunk... but somehow i simply dont get my actual problem 🙂