Searching, regex, learning opportunity

[jgauthier]

I am trying to refine a built in search to the Windows app.

The search is failed logins.

source="wineventlog:security" ("EventCode=4625") OR ("EventCode=529" OR "EventCode=530" OR "EventCode=531" OR "EventCode=532" OR "EventCode=533" OR "EventCode=534" OR "EventCode=535" OR "EventCode=536" OR "EventCode=537" OR "EventCode=539") (Logon_Type=3 OR Logon_Type=8 OR Logon_Type=10) | `get_user_name` | chart count by User_Name

This returns some unappealing data. Namely, machines names, and a hyphen. I can remove the hyphen easily enough by sticking a ' | search NOT User_Name="-" | ' after the get_user_name macro.

Now, I want to build a regex that removes the machine name. I attempted to use the field extractor, but the data is cut off in the extractor (it does not show the full value)

So I formed the regex, it's simple: \bAccount Name:.*\$

I can't figure out how to apply it. Thanks for the learning opportunity!

[jgauthier · Accepted solution]

I believe I have a satisfactory solution for my desires. This allows me to capture only the login failures and successes that I want.

inputs.conf

[WinEventLog:Security]
disabled=0
current_only = 1

props.conf (copied from the default and modified)

[source::WinEventLog...]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
REPORT-MESSAGE = wel-message, wel-eq-kv, wel-col-kv
KV_MODE=none
TRANSFORMS-FIELDS = strip-winevt-linebreaker
TRANSFORMS-set=SEDAccount,FilterOut
SEDCMD-translate=s/Account Name:/Account EDITED/

transforms.conf

[SEDAccount] REGEX=. DEST_KEY=queue FORMAT=indexQueue
[FilterOut] REGEX=(\bAccount Name:\s*(?<ComputerName>.*)\$)|(Logoff)|(ANONYMOUUS)
DEST_KEY=queue FORMAT=nullQueue

I installed a heavy forwarder on the Domain Controller. Basically, I take one instance of "Account Name" and change it to "Account EDITED". Then it's not longer detected as the username field, so the counts only go for users.

Then I used the regex recommended above to detect computer names in the "Account Name" field, Logoff, or ANONYMOUS. I added those to reduce indexing quantity.

I'm pleased with the results. Thanks for the help!

[David]

You probably want to go with something like adding the following at the end (I made a minor tweak to your regex, just based on my experience with windows log extractions):

...YourSearch | rex field=_raw "\bAccount Name:\s*(?<ComputerName>.*)$" 
              | search NOT ComputerName="MyComputer"

The rex command is how you apply regex field extracts on the search line. You can also then move it to the props.conf using the EXTRACT command (see http://www.splunk.com/base/Documentation/latest/admin/propsconf for more)

Expanding from comments:

If you actually have two "Account Name: " in your logs, one with the username and one with the computername, I'd go for something like the following:

...YourSearch | rex field=_raw "\bAccount Name:\s*(?<ComputerName>.*)\$" 

To separate out just the computer name. I'd also have a separate regex for the username, but since that doesn't have the $ at the end, it's harder to make a regex for it. You can paste in a section of your logs, if neither of the below work, but there are two variations that I've seen:

If your have Account Name: JSmith with a newline immediately following, I'd do:

...YourSearch | rex field=_raw "\bAccount Name:\s*(?<UserName>[^\$]*)$" 

(Note the regex $ without the escape, indicating "to the end of the line", and the usage of [^\$], indicating "match any character except for a dollar sign.") If instead of a newline, you have more whitespace (tabs, spaces, what have you), I'd go with:

...YourSearch | rex field=_raw "\bAccount Name:\s*(?<UserName>[^\$]*)\s" 

Let me know if either of those work for you. If not, it might be best to edit your original question, and paste in a segment from your logs.

[jgauthier]

I went another direction that I though would surely work! I decided to use the sed mode of rex, and rename the first "Account Name" to something else. I though that would cause it to not find that field. No such luck. It appears that the Splunk Field Discovery is seeing this before it gets to my Rex, and at that point it's useless. Using the Events Table mode of the output, I can see it has determined 3 rows are found, and it has both Account Names in the same row. I think I need to shift direction. Is there anyway to pass this through rex/sed on the forwarder? That would definitely do.

[David]

Ah ha, I misunderstood your question, but now I'm on track. In general, if it's returning a record with a computer name, you should be able to say NOT AccountName="*$". I tested this successfully on my local instance. However, you can get into some tricky waters if there are two AccountNames -- if one was the computer name, it would not return the record, even though there was a valid computer name there. If you do have two Account Name: in your logs, I'd go for a refinement of the regex. I've just added that to my main response (for formatting). Let me know if that works for you.

[jgauthier]

The data has mutliple "Account Name:" fields. One with the machine name, and one with the user who attempted log in)
So, while what I posted above (NOT User_Name = "*$") seems to work, it doesn't give me the log in names that I want. It also matches the machine name. I'm not quite sure how to accomplish what I am trying to achieve.

I would like to see the statistics without it counting the machine name as unique. Currently it returns output like this:

1 MachineName$ 2
2 Username 2

And the records are identical when I drill into them. Thanks for the input!

[jgauthier]

Interesting. Thanks for the input! It looks like ComputerName becomes the field extract, and then you can search against it. How about simply omitting it the criteria period? I attempted to do that, but not sure if I can. I know some times wild cards work, but wasn't sure if I could do something simply like "NOT User_Name = "*$"