Getting Data In

How do I capture both IP (ip_address) and hostname (host) information in my logs?

oliverj
Communicator

Hello all.

I am trying to capture BOTH IP address info AND hostname in my logs. This data needs to be part of the actual logged data, not an added lookup field.

I have found this line in the inputs.conf file:

[default]
connection_host = <ip/dns>

By messing around with this, I was able to get this default section to work:

[default]
host = localhost
connection_host = ip

By adding the "localhost", the forwarder is forced to use the connection_host line for the "host" field.
Now my search results show an IP address for the "host" field in my searches.

The problem is, I need both IP info AND hostname data, and I would really like the hostname to be in the "host" field, and put the IP address in a custom field (I assume "_meta:IP_ADDRESS::myip)

My reason for this need:
We will be working with several different groups of computers, and there will most likely be hostname or ip address overlaps.
Also, the security team likes to be able to trace activity back via IP address, as not all devices log DNS names of network activity (firewalls/switches). With so many internal networks, we have no insight to their DNS for lookups anyways. IP is much easier.

I need to have my log contain the fields:
ip_address =
host =

Currently, I can only get my "host" to be EITHER ip or hostname. I have no way to populate ip_address with data.

This is a continuation of this post from last year: https://answers.splunk.com/answers/271921/how-to-keep-ip-source-info-from-the-originating-un.html

0 Karma

muebel
SplunkTrust
SplunkTrust

Generally a lookup is the preferred way of going about something like this, but if that isn't an option, you could make the host field multivalue, and append the IP delimited by a '|', i.e.

# inputs.conf
[default]
server = myhostname.mydomain.com|192.168.1.100

And then extract as needed at search time.

Alternatively, you might be able to setup something to add an index time field. by something like:

#transforms.conf
[ip_field_set]
FORMAT = ip_address::192.168.1.100
WRITE_META = true
REGEX = (.+?)

#props.conf
[default]
TRANSFORMS-ip_field_set = ip_field_set

Or you could do some sort of combination, setting the source servername as a multivalue field, and then down the line setup props/transforms to extract the IP to a new field, while shortener the hostname to just the hostname.

Please let me know if this helps in any way 😄

0 Karma

oliverj
Communicator

And another.

0 Karma

oliverj
Communicator

I just realized something in your answer I missed before:
Your method works, but only with hardcoded values. I really need something that pulls some sort of variable, like the $decideAtStartup host variable, or $hostname.
But these do not work/I am using the wrong syntax.

0 Karma

oliverj
Communicator

Bump! Still definitely interested in how to capture both pieces of info

0 Karma

oliverj
Communicator

another bump

0 Karma

oliverj
Communicator

and another.

0 Karma

oliverj
Communicator

I tried:

[default]
host = localhost|foo
connection_host = ip 

and

[default]
host = localhost|$COMPUTERNAME
connection_host = ip 

This only gives me an IP address still. The text "foo" never shows up in the listing of log files. If I could get this to work, I think it may meet my needs, though.

I think I have to continue to use the connection_host = IP method of extracting an IP, because I cant be hardcoding ip addresses or hostnames anywhere.

0 Karma

oliverj
Communicator

One option that should work is to keep host as an ip address (working) and add a "host_name" custom field (not working).
is it possible to do something like this on the universal forwarder:

[default]
_meta = host_name::$hostname

I have been unable to get that to work. I did add a hardcoded value to _meta, but only when under a sourcetype, not in the [default] section.
Maybe _meta isn't the right option here for forcing an additional custom field?

0 Karma

Vanvanvantasy
Splunk Employee
Splunk Employee

Another bump from 6 years later. still facing similar issue and what a answer to this question

Tags (1)
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.