Getting Data In

How to display multiline columns

shocko
Path Finder

I'm using Splunk 6.1.4 (soon to be 7.x). I've processed some windows event log data and as per normal Spunk processing, the Windows Event which is XML in windows land Data element contains the text-based message of the event. This is extracted at index time to the field message. Now, the particular event I'm processing might have a message field like this as seen in splunk:

***Password UserChange for user 'MYTESTUSER' rejected.
The password did not meet the requirements specified in policy 'MyPassword Tool - Standard user Complex Password'.

Upper:1:FALSE
RegularExpression:please do not use any of the following characters ~ € # £:FALSE"***

So these are basically reasons for failure for a users password change attempt. What I am trying to do is display them like this in a table visualization:

alt text

The key here is that the three fields Outcome, Component and Detail are display as shown.
Is this possible?

0 Karma

shocko
Path Finder

Guys, I think I was over complicating things. I resolved as follows:

mymacro | rex field=Message "^Password UserChange for user '(?.+)'\srejected." | rex field=Message "The password did not meet the requirements specified in policy '(?.)'." | rex field=Message max_match=0 "(?.+):(?.):(?(TRUE|FALSE))" | Table _time Policy User Outcome Rule Details

0 Karma

shocko
Path Finder

Guys, resolved this just piping each piece into REX e.g.:

mymacro | rex field=Message "^Password UserChange for user '(?.+)'\srejected." | rex field=Message "The password did not meet the requirements specified in policy '(?.)'." | rex field=Message max_match=0 "(?.+):(?.):(?(TRUE|FALSE))" | Table _time Policy User Outcome Rule Detai

0 Karma

Sukisen1981
Champion

@shocko
your code has got corrupted while pasting as a comment,please paste as an answer and accept your answer for the benefit of the forum in the future.
Please upvote mine or @niketnilay's answers if they helped you significantly to arrive at your own answer

0 Karma

shocko
Path Finder

Done now Sulisen!

0 Karma

niketnilay
Legend

@shocko if you are feeding windows event logs to Splunk you have option to send it as xml or KV pair. If you are sending the same as XML, and the sourcetype has KV_MODE set as xml, fields should get automatically extracted. Which means depending on the structure of XML event (root node and xml heirarchy) your Data field should automatically be extracted. Since all the information you need is available in the Data field, once you know the field name you can proceed by adjusting following run anywhere example as per your needs:

| makeresults
| eval _raw="<EventData>
     <Data>Password UserChange for user 'MYUSER' rejected.
         The password did not meet the requirements specified in policy 'Complex Password'.

         MaximumLength:15:FALSE
         DisallowUserName:1:FALSE
         DisallowConsecutive:3:FALSE
     </Data>
 </EventData>"
| spath
| rex field="EventData.Data" "(?ms)Password UserChange for user '(?<UserName>[^\']+)\'\srejected.\s+The password did not meet the requirements specified in policy 'Complex Password'.\s+(?<Reason>.*)"
| eval Reason=replace(Reason,"\s+","###")
| eval Reason=split(Reason,"###")
| table UserName Reason
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

shocko
Path Finder

HI, no not feeding in as XML so will be parsed to KV. As such, my data of interest is filed message e.g.:

*Message=Password UserChange for user 'MYUSER' rejected.
The password did not meet the requirements specified in policy 'Complex Password'.

     MaximumLength:15:FALSE
     DisallowUserName:1:FALSE
     DisallowConsecutive:3:FALSE*

So need to parse this and it will have line feeds to .+ won't work here.

0 Karma

shocko
Path Finder

Thanks for the reply. I think I have explained this poorly. Some examples might help further. So the windows event in XML format will look something like this:

  <System>
    <Provider Name="Password Policy" />
    <EventID Qualifiers="0">202</EventID>
    <Level>3</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2019-08-12T15:27:55.000000000Z" />
    <EventRecordID>356042</EventRecordID>
    <Channel>Application</Channel>
    <Computer>mydc.lab</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Password UserChange for user 'MYUSER' rejected.
        The password did not meet the requirements specified in policy 'Complex Password'.

        MaximumLength:15:FALSE
        DisallowUserName:1:FALSE
        DisallowConsecutive:3:FALSE
    </Data>
</EventData>

Splunk will parse these fields as Windows Event Log format is supported. The message field will be the value of the XML tag. As such, for password change failure we are parsing the value of the message field. Now, each reason for failure has the format Reason:ReasonPolicyValue:Outcome so DisallowConsecutive:3:FALSE means 'do not allow use of the same character, 3 being the limit of consecutive characters and FALSE being the outcome of the users password against this policy component.

Now, to make life harder we could have a policy with 10 or more of this policy components e.g.

alt text

So we need a way to account for this somewhat arbitrary number of reasons that need to be parsed!

0 Karma

Sukisen1981
Champion

Try this , I am assuming when the password is rejected for a particular reason it will be identified by a TRUE flag, for example , max length violation will be something like : MaximumLength:15:TRUE in the logs if max length is set by user to above 15. you might need to tinker with the IF conditions based on that, run the below code as is and check

| makeresults 
| eval user="08/12/2019 04:27:55 PM
LogName=Application
SourceName=Password Policy
EventCode=202 EventType=3 Type=Warning
ComputerName=mydc.lab TaskCategory=The
operation completed successfully.
OpCode=Info RecordNumber=356042
Keywords=Classic Message=Password
UserChange for user 'MYTESTUSER'
rejected. The password did not meet
the requirements specified in policy
'Complex Password'.
MaximumLength:15:FALSE
DisallowUserName:1:FALSE
DisallowConsecutive:3:FALSE"
| rex field=user "UserChange for user\s+\'(?<user1>.*?)\'" 
| rex field=user "MaximumLength:15:(?<reason1>.*)"
| rex field=user "DisallowUserName:1:(?<reason2>.*)"
| rex field=user "DisallowConsecutive:3:(?<reason3>.*)"
| eval reason1=if(reason1="FALSE","","MaximumLength")| eval reason2=if(reason2="FALSE","","DisallowUserName")| eval reason3=if(reason3="FALSE","","DisallowConsecutive")
| eval reason=reason1+"."+reason2+"."+reason3
|eval reason = split(reason,".")
| table _time,user,reason
0 Karma

shocko
Path Finder

Here is the raw windows event as I see it (real usernames removed of course!):

*

08/12/2019 04:27:55 PM
LogName=Application
SourceName=Password Policy
EventCode=202 EventType=3 Type=Warning
ComputerName=mydc.lab TaskCategory=The
operation completed successfully.
OpCode=Info RecordNumber=356042
Keywords=Classic Message=Password
UserChange for user 'MYTESTUSER'
rejected. The password did not meet
the requirements specified in policy
'Complex Password'.
MaximumLength:15:FALSE
DisallowUserName:1:FALSE
DisallowConsecutive:3:FALSE

As such the reasons for the failure are manifold namely 3 reasons:

  • MaximumLength
  • DisallowUserName
  • DisallowConsecutive

🙂

0 Karma

Sukisen1981
Champion

it is possible but you need an identifier like an user name for which these password change attempts are rejected. The username would perhaps be in the eventlogs before user attempts to change password, please throw some more light and event log data

0 Karma

shocko
Path Finder

I have the username (I have clarified my original posting) 🙂

0 Karma

Sukisen1981
Champion

hi @shocko - You have provided your expected output, thanks for that, how does your events in splunk look like? Can you paste them as text..guessing it will be lines of raw text with user id and the password reject info.
If we get that info then we can always build your expected table output

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.