Archive

How to do DNS lookup for event-time

Explorer

I am facing a problem I struggle to find a solution for. I want to get the hostname that was associated to an IP address at the time the event occured.

I first tried to us external_cmd in transforms.conf and LOOKUP in props.conf, but I found out that LOOKUP only works at search-time and therefore does not solve my problem.

Now I am looking for a solution to either add the hostname to the event at index-time or to look up the hostname for specific times so that I can use serch-time lookups.

I've been searching for a while now but were not able to find any suitable solution.

New Member

Did anybody find a solution for that problem? I'm facing the same ...

I need to resolve and store the hostnames from security logs timely as possible to the event time to preserve the hostname and do further correlations on it.

0 Karma

Explorer

Hi Fred,

as I have progressed with my splunk training I came to understand that this seems to be a use case that just doesn't go well with the concept of how splunk works. Splunk simply doesn't do lots of data enrichment during indexing time. With all th advantages and disadvantages this may bring.

The only suitable solution I have come across by now is to have a seperate index that logs changes on dns entries and do a join of both indexes for dns lookup. I didn't implement it, but this should be a viable soution for the problem.

0 Karma

New Member

Thanks for your feedback.

Wow, thats extremly disappointing...

In an envíronment with having

  • > 150.000 endpoints
  • each may have several DHCP leases
  • keeping logs up to 2 years
  • > 1.5 TB Logfiles per day
  • ...

this is really a show-stopper.

;-(

Fred

0 Karma

Champion

If you have the DHCP/DNS info also stored in Splunk, you could make use of it in various ways to meet your needs. Here are two I can think of.

A. You could run scheduled searches against the DHCP/DNS logs to append to a time-based lookup file, which can be referenced for your sourcetype to perform the lookup at search time for your other data. This could potentially grow rather large over time, and may be better suited to a KVStore.
B. You could do some kinda tricky logic at search time to associate DHCP/DNS logs to other events. This may be fairly complicated (it was when I tried, at least), but theoretically is possible. This may be easiest if you can find the logs you want, then apply the DHCP/DNS search to it.

0 Karma

Explorer

Thanks for the answer, but I am actually not sure how you would actually configure these options without a lot of restrictions.

A. Your idea is to watch the logs for entries that inform about new DHCP/DNS leases, right? That might work for devices using DHCP and newly assigned DNS entries, but I wouldn't be able to query for old IPs/hostnames like long running servers or workstations that are used daily and therefore don't have leases expiring. At the moment we don't send these logs into splunk. If there weren't such big advantages I could imagine doing so, but I am not sure yet whether this is the right solution for me.

B. basically has the same problem as it relies on DHCP/DNS logs.

Is there really no way to trigger a DNS lookup on indexing? We're just replacing an ELK installation with splunk and this is the single first thing that was way easier to do with ELK.

0 Karma