Splunk Search

Why is IP address Searching/Matching so slow?

Explorer

I have a datasource with a field that is either a url or an ip address. There are 2million records in this datasource.

If i do a search for a URL the response time is quite good.

index=sandbox | location=www.domain.co.uk

But if I do a search for an IP address it is very slow.

index=sandbox | location=192.168.1.1

As both searches have to go through the same amount of data, why the difference?

Is there an alternative/faster way for me to search IP addresses?

The fields is defined in the source type so is handled at ingestion/indexing time not at search.

1 Solution

Contributor

you could use the TERM() search operator.

Search with TERM() as described here

You can use the TERM() directive to force Splunk software to match whatever is inside the parentheses as a single term in the index. TERM is more useful for cases where the term contains minor segmenters, such as periods and commas, and is bounded by major segmenters, such as spaces or commas. In fact, TERM does not work for terms that are not bounded by major breakers. 

Hope that helps

View solution in original post

Contributor

you could use the TERM() search operator.

Search with TERM() as described here

You can use the TERM() directive to force Splunk software to match whatever is inside the parentheses as a single term in the index. TERM is more useful for cases where the term contains minor segmenters, such as periods and commas, and is bounded by major segmenters, such as spaces or commas. In fact, TERM does not work for terms that are not bounded by major breakers. 

Hope that helps

View solution in original post

Explorer

That worked perfectly....

As a guide to performance improvement; I used TERM on an IP search for a source type with 6+ million domains/IPs/urls.

Old Way:      index=test field1=192.168.1.1                    4 minutes 24 seconds
Term Way:     index=test field1=TERM(192.168.1.1)              0 Seconds!!!

And searching for URLs and Domains improved also...

Many thanks!

0 Karma

Path Finder

From docs link text

However, searching for TERM(127.0.0.1) fails for data that looks like this:
ip=127.0.0.1 - user=admin

This is because the equal symbol ( = ) is a minor breaker, not a major breaker. Additionally, the IP address portion of the event is indexed as: ip, 127, 0, 1, and ip=127.0.0.1. You are looking for 127.0.0.1, which is not an indexed term.

0 Karma