Splunk Search

Is there a way of ensuring every event has a specific field?

Robbie1194
Communicator

Hi guys,

My goal is to use the map command look over all ip addresses and mac addresses that have had the value policy=policy. However, not every event has a mac address field. Is there a way of making ever event have a mac address field even if it's null? The map search has issues when there are some ips that don't have a corresponding mac address value.

I've tried using "| fillnull value="N/A" mac" just before the map command but it didn't work. Does anyone know of anything else I could do to ensure there is always a mac address field?

The search I'm using is:

index=index sourcetype=sourcetype policy="policy1" rule="rule1"
| dedup ip
| search NOT (ip="ip address" OR ip="ip address" OR ip="ip address")
| map search="search index=index sourcetype=sourcetype ip=$ip$ mac=$mac$"
| eval evalfield = if(policy="policy1" AND status="Match", "yes", "no")
| search evalfield ="no
| stats values(status) as status, values(rule) as rule, values(policy) as policy, values(evalfield) as evalfield, count by ip, mac

If anyone needs any further clarification, let me know.

Any help would be appreciated.

Cheers!

0 Karma

micahkemp
Champion

I think you should find a way to avoid using map. Not fully understanding what you're looking for, but trying to accomplish similar results I expect your search to yield, this may be a step towards what you're looking for:

index=index sourcetype=sourcetype NOT (ip="ip address" OR ip="ip address" OR ip="ip address") NOT (policy="policy1" AND status="Match")
| fillnull value="N/A" mac
| stats values(status) as status, values(rule) as rule, values(policy) as policy, values(evalfield) as evalfield, count by ip, mac
| search policy="policy1" rule="rule1"
0 Karma

Robbie1194
Communicator

I'm not sure if this search will work but I may be getting a bit lost in my own head. The goal of the search is to:

  • Search the index for all events matching policy1
  • Take the IP and mac from each of these events and then re-search the same index for when this IP and associated mac (not always there thats why I needed to fillnull) have also matched on policy2.
  • If they have matched, then yes, else no.
  • Search yes or no (depending on what we need).
0 Karma

somesoni2
Revered Legend

Try like this
Fixed map search, escaped double quotes

index=index sourcetype=sourcetype policy="policy1" rule="rule1" 
| search NOT (ip="ip address" OR ip="ip address" OR ip="ip address")
| dedup ip
| eval mac=coalesce(mac,"N/A")
| map search="search index=index sourcetype=sourcetype ip=$ip$ 
| eval mac=coalesce(mac,\"N/A\") | search mac=$mac$"
| eval evalfield = if(policy="policy1" AND status="Match", "yes", "no")
| search evalfield ="no"
| stats values(status) as status, values(rule) as rule, values(policy) as policy, values(evalfield) as evalfield, count by ip, mac
0 Karma

Robbie1194
Communicator

I tried this but no results unfortunately.

Cheers though.

0 Karma

somesoni2
Revered Legend

See if updated search works for you.

0 Karma

Robbie1194
Communicator

I still get no results but I get this message in the job drop down:

"Unable to run query 'search index=fsctcenter sourcetype=fsctcenter_json ip="10.xx.x.xx" | eval mac=coalesce(mac,"N/A" | search mac="00:04:xx:xx:xx:xx"

The x's represent valid numbers

0 Karma

Robbie1194
Communicator

I'm not really familiar with the coalesce function and couldn't really find a good description of it online, what does it do?

0 Karma

micahkemp
Champion

| fillnull value="N/A" mac should do what you're looking for, which you already tested. Can you run the search with that in place, but exclude the map and everything after it, and see what the results for that look like?

index=index sourcetype=sourcetype policy="policy1" rule="rule1" 
| dedup ip
| search NOT (ip="ip address" OR ip="ip address" OR ip="ip address")
| fillnull value="N/A" mac
0 Karma

Robbie1194
Communicator

Yeah I tried that and it filled in the mac-less events giving 25 mac values to 25 ip values but when I run the above map search it shows that there are 9 ip addresses and 9 mac addresses, the event with the N/A value was not passed through.

I was wondering if there would be something better to use to accomplish this than map? But I'm not too sure.

0 Karma

micahkemp
Champion
| makeresults
| eval alwayspresent="alreadyhere"
| fillnull value="nolongernull" wasnull
| map search="| makeresults | eval stillpresent=\"$alwayspresent$\", wasnullpassedthrough=\"$wasnull$\""

This should show that the fillnull values make it through to the map command, but there is likely a better search to meet your needs.

Your search has index=index sourcetype=sourcetype in both the main search string and the map search string. Do both searches use the same index and sourcetype, or is the map search looking at different data?

0 Karma

Robbie1194
Communicator

I get what you're doing with the makeresults search so I'll investigate my results further to see why they're not passing over.

My searches do look at the same index and sourcetypes. Is there a better way?

0 Karma

jplumsdaine22
Influencer

remember map will only use 10 events by default

0 Karma
Get Updates on the Splunk Community!

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...