Splunk Search

Regex, and extracting the IP + hostname from _internal

oliverj
Communicator

One of my ongoing gripes with splunk is that there is no way to see the IP and the hostname -- either my forwarder sends a hostname, or an IP. Not both. I know the information is there, as I can see it in the _internal splunkd log.
So, I was trying to make a list of all IP+hostnames out of this list:

index=_internal uri=* component=HttpPubSubConnection

Result:

HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_ipaddress_8089_instancename_hostname_FAB4D96E-5A4A-4593-8914-635506217E40

In the URI, it has the ip and a hostname. So, I want to extract this!
In the splunk field extractor, I type my own regex,

 ^(?:[^_]+_){4}(?P<internal_ip>[^_ ]+)

This works great! It shows all my matches, I save it, run my search, all is well.

Now if I do the same without the props.conf:

index=_internal uri=* component=HttpPubSubConnection | regex  "^(?:[^_]+_){1}(?P<internal_ip>[^_ ]+)" | table internal_ip

Nothing.
I could do it all via props.conf, but there are so many situations where that regex matches outside the specific search I want to run that I figured it was much more efficient to just do the regex in the search string. But, no extraction seems to be happening.
Any suggestions?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The regex command does not do field extractions. Use rex, instead.

... | rex field=uri "[^_]+_(?P<internal_ip>[^_ ]+)_\d+_[^_]+_(?<hostname>[^_]+)" | ...
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The regex command does not do field extractions. Use rex, instead.

... | rex field=uri "[^_]+_(?P<internal_ip>[^_ ]+)_\d+_[^_]+_(?<hostname>[^_]+)" | ...
---
If this reply helps you, Karma would be appreciated.

oliverj
Communicator

That was exactly what I needed. Thank you.

0 Karma

oliverj
Communicator

To anyone who finds this post later -- this is a terrible search. I just realized it only pulls data from hosts that use deployment servers, not all UniversalForwarders. So anything that sends without being a part of the deployment server (for us, hundreds of devices we don't own) will not show up. Sigh.

But the search/extraction still works great! Just...back to the drawing board.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try component=tcpinconnection.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...