Getting Data In

Casing and FQDNs means triple host entries

CMSchelin
Path Finder

I have quite a few hosts that are listed multiple times in Splunk. For an example:

HOST01
host01
host01.fqdn

I'm relatively new to Splunk. What are my options for merging those entries?

1 Solution

woodcock
Esteemed Legend

The best thing to do is to have each forwarder report (self-identify) the way that you prefer. This is controlled in server.conf on the forwarder so read the documentation on that. This effects NEW data but old data will stay jumbled so you can do something like this before you aggregate your data with stats or whatever:

... | rex field=host mode=sed "s/\..*$//" | eval host=lower(host) ...

View solution in original post

DalJeanis
Legend

We feel your pain. Also, please note, in some types of events we tend to end up with the IP address standing in place of a missing hostname, so you can't grab everything to the left of the period until after you've checked that the entire hostname isn't of the format of an IP. Usually, we end up using upper() and some rexes to make sure we are getting the best guess at the short host.

| rex field=myhostname "^(?<shost>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[^.\]*)"
| eval shost=upper(coalesce(shost, myhostname, "((MISSING))" ))

The above grabs an IP address if it starts out the field (imprecise IP-grabbing rex for demo purposes only) then grabs everything to the left of the first period... which if there is no period, means the whole field. The only failure in the rex is if the first character is an actual period, in which case the eval/coalesce puts the invalid hostname back into place before translating the whole thing to upper case.

General notes - 1) searches are case-insensitive, but field values are case-sensitive. 2) Field aliasing is your friend.

0 Karma

woodcock
Esteemed Legend

The best thing to do is to have each forwarder report (self-identify) the way that you prefer. This is controlled in server.conf on the forwarder so read the documentation on that. This effects NEW data but old data will stay jumbled so you can do something like this before you aggregate your data with stats or whatever:

... | rex field=host mode=sed "s/\..*$//" | eval host=lower(host) ...
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...