Splunk Search

Same fields with different values in one event

kkuminsky
Path Finder

In the following Windows event log message field Account Name appears twice with different values. When I build a report by Account Name it looks like there were two events instead of one, because Splunk is indexing Account Name twice in this case.

Is there a way to avoid indexing the first one (under subject)?


LogName=SecuritySourceName=Microsoft Windows security auditing.

EventCode=4625

EventType=0

Type=Information

ComputerName=AD1.TEIG.production

TaskCategory=Logon

OpCode=Info

RecordNumber=1371401

Keywords=Audit

FailureMessage=An account failed to log on.

Subject:

Security ID: S-1-0-0

Account Name: -

Account Domain: -

Logon ID: 0x0

Logon Type: 3

Account For Which Logon Failed:

Security ID: S-1-0-0

Account Name: APP2$

Account Domain: TEIG.PRODUCTIONFailure

Information: Failure

Reason: An Error occured during Logon.

Status: 0xc0000133

Sub Status: 0x0

2 Solutions

Lowell
Super Champion

I don't think your issue is that splunk is indexing your event multiple times. Splunk will only index an event once. I think your questions is really about field extraction and multi-value fields. For WinEventLog:Security events, splunk already has setup the necessary field-extraction rules for you, so all the fields you needs should already be available. However, I think what you hitting is the fact that splunk allows multiple value for these fields. (Which is generally preferable over silently dropping one of the values.)

I'm assuming that in your example, your are ending up with a multi-value field named Account_Name that contains the values of '-' and APP2$.

So what you probably want to do is simply drop the "-" value, which can be done a couple of different ways. You can build your chart or report and simply drop the "-" value after the fact. Or you can tell splunk to keep only the first or last value of a multi-value field, or filter the multi-value field. Here are a few examples to give you an idea of what's possible:

Keep only the first value:

sourcetype="WinEventLog:Security" | eval Account_Name=mvindex(Account_Name, 0) | ...

Keep only the last value:

sourcetype="WinEventLog:Security" | eval Account_Name=mvindex(Account_Name, -1) | ...

Remove out the "-" value using mvfilter. (I think this is probably the best option for you)

sourcetype="WinEventLog:Security" | eval Account_Name=mvindex(Account_Name!="-") | ...

Remove out the "-" value (using a post-search command)

sourcetype="WinEventLog:Security" | stats count by Account_Name | search Account_Name!="-"

Here is one last search that will NOT work for what you want. (I feel this is an important gotcha to understand when searching with multi-value fields)

sourcetype="WinEventLog:Security" Account_Name!="-" | ...

Sometimes this may actually appear to work (based on how many events actually contain a dash value, vs how many event contain non-dash values). Here is what's really going on here: Any event that has a field named Account_Name where any of the multi-values is "-" will be dropped from your search results. Therefore, the example event that you provided will be excluded; which means that you will be loosing both the "-" and "APP2$" at the same time. So this approach is not what you want.

Side note: If you are fairly new to splunk, I would suggest that you play around with these different options until you get a better feel for how these different approaches work. You may find that such an exercise will end up saving you time in future splunk searches. Just a thought.


Additional resources:

View solution in original post

marcoscala
Builder

An alternative way could be to remap the Account Name field with the following directive in props.conf:

[WinEventLog:security]
EXTRACT-AccountName = Account Name:\s(?P<AccountName>[^\s^:^-]+)\s

This will prevent to have something like "Account Name: -" recognized as another value for the Account name field.

Regards, Marco Scala

View solution in original post

kkuminsky
Path Finder

Guys, thank you for the detailed info. I'm new to splunk and I'm impressed by it's search functionality.

So, I've adopted this one:

eval Account_Name=mvindex(Account_Name, -1) | ...

Mostly because:

  1. There are some events that have "-" value for both Account_name fields

  2. I'm expecting different values for the first account field, not only "-", but the one that I'm interested in is the second one.

0 Karma

Lowell
Super Champion

Sounds like good reasoning to me.

0 Karma

marcoscala
Builder

An alternative way could be to remap the Account Name field with the following directive in props.conf:

[WinEventLog:security]
EXTRACT-AccountName = Account Name:\s(?P<AccountName>[^\s^:^-]+)\s

This will prevent to have something like "Account Name: -" recognized as another value for the Account name field.

Regards, Marco Scala

marcoscala
Builder

this is useful! thanks Lowell.

0 Karma

Lowell
Super Champion

Your regex will also block any account name with a dash in it, not just "-". For example, a value of "APP-2" doesn't match. You could try this regex instead: EXTRACT-AccountName = ^\s*Account Name:\s(?P<AccountName>(?!-$).+)\s*$

0 Karma

Lowell
Super Champion

Yes, this will work, and in previous versions of Splunk this may have been the best option. Here's the downside: You now have multiple fields with the same info (AccountName and Account_Name) and some extra regex overhead. Also, if you do have events with multiple account names given (other than "-", of course), than this approach will only match the first one. Perhaps this doesn't happen in this type of event, but it's something to think about.

0 Karma

Lowell
Super Champion

I don't think your issue is that splunk is indexing your event multiple times. Splunk will only index an event once. I think your questions is really about field extraction and multi-value fields. For WinEventLog:Security events, splunk already has setup the necessary field-extraction rules for you, so all the fields you needs should already be available. However, I think what you hitting is the fact that splunk allows multiple value for these fields. (Which is generally preferable over silently dropping one of the values.)

I'm assuming that in your example, your are ending up with a multi-value field named Account_Name that contains the values of '-' and APP2$.

So what you probably want to do is simply drop the "-" value, which can be done a couple of different ways. You can build your chart or report and simply drop the "-" value after the fact. Or you can tell splunk to keep only the first or last value of a multi-value field, or filter the multi-value field. Here are a few examples to give you an idea of what's possible:

Keep only the first value:

sourcetype="WinEventLog:Security" | eval Account_Name=mvindex(Account_Name, 0) | ...

Keep only the last value:

sourcetype="WinEventLog:Security" | eval Account_Name=mvindex(Account_Name, -1) | ...

Remove out the "-" value using mvfilter. (I think this is probably the best option for you)

sourcetype="WinEventLog:Security" | eval Account_Name=mvindex(Account_Name!="-") | ...

Remove out the "-" value (using a post-search command)

sourcetype="WinEventLog:Security" | stats count by Account_Name | search Account_Name!="-"

Here is one last search that will NOT work for what you want. (I feel this is an important gotcha to understand when searching with multi-value fields)

sourcetype="WinEventLog:Security" Account_Name!="-" | ...

Sometimes this may actually appear to work (based on how many events actually contain a dash value, vs how many event contain non-dash values). Here is what's really going on here: Any event that has a field named Account_Name where any of the multi-values is "-" will be dropped from your search results. Therefore, the example event that you provided will be excluded; which means that you will be loosing both the "-" and "APP2$" at the same time. So this approach is not what you want.

Side note: If you are fairly new to splunk, I would suggest that you play around with these different options until you get a better feel for how these different approaches work. You may find that such an exercise will end up saving you time in future splunk searches. Just a thought.


Additional resources:

essklau
Path Finder

Great answer; loved the spl breakouts

0 Karma

Lowell
Super Champion

I can't delete others comments. I removed mine. (and I can remove this one later)

0 Karma

jkat54
SplunkTrust
SplunkTrust

That was then, this is now... and NOW you have mod rights so you CAN delete others comments :-). Both of you! All of us!

Cheers!

0 Karma

sideview
SplunkTrust
SplunkTrust

Oh of course. Sorry i missed that. Feel free to delete this little comment failtrain. my bad.

0 Karma

sideview
SplunkTrust
SplunkTrust

Note that your last last example ('Remove out the "-" value') will actually filter out the events that have both values, not simply clean out the value.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...