Splunk Search

Regex, and extracting the IP + hostname from _internal

oliverj
Communicator

One of my ongoing gripes with splunk is that there is no way to see the IP and the hostname -- either my forwarder sends a hostname, or an IP. Not both. I know the information is there, as I can see it in the _internal splunkd log.
So, I was trying to make a list of all IP+hostnames out of this list:

index=_internal uri=* component=HttpPubSubConnection

Result:

HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_ipaddress_8089_instancename_hostname_FAB4D96E-5A4A-4593-8914-635506217E40

In the URI, it has the ip and a hostname. So, I want to extract this!
In the splunk field extractor, I type my own regex,

 ^(?:[^_]+_){4}(?P<internal_ip>[^_ ]+)

This works great! It shows all my matches, I save it, run my search, all is well.

Now if I do the same without the props.conf:

index=_internal uri=* component=HttpPubSubConnection | regex  "^(?:[^_]+_){1}(?P<internal_ip>[^_ ]+)" | table internal_ip

Nothing.
I could do it all via props.conf, but there are so many situations where that regex matches outside the specific search I want to run that I figured it was much more efficient to just do the regex in the search string. But, no extraction seems to be happening.
Any suggestions?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The regex command does not do field extractions. Use rex, instead.

... | rex field=uri "[^_]+_(?P<internal_ip>[^_ ]+)_\d+_[^_]+_(?<hostname>[^_]+)" | ...
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The regex command does not do field extractions. Use rex, instead.

... | rex field=uri "[^_]+_(?P<internal_ip>[^_ ]+)_\d+_[^_]+_(?<hostname>[^_]+)" | ...
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

oliverj
Communicator

That was exactly what I needed. Thank you.

0 Karma

oliverj
Communicator

To anyone who finds this post later -- this is a terrible search. I just realized it only pulls data from hosts that use deployment servers, not all UniversalForwarders. So anything that sends without being a part of the deployment server (for us, hundreds of devices we don't own) will not show up. Sigh.

But the search/extraction still works great! Just...back to the drawing board.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try component=tcpinconnection.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.