Security

How can I do a search for all URLs that have not had a hit in the last two months.

wbounni11
New Member

Hi;

I want to run a search that reveals all the URL that have not had a hit in the last two months. The feed here are access logs coming from a Symantec Proxy SG device.

The purpose of doing this is removing these URLs from the Proxy policy.

It would be great if I can get the Syntax as I am new to Skype.

Kindly
Wasfi

0 Karma
1 Solution

tscroggins
Motivator

Hi,

You'll need a source of current URLs to compare against past events, e.g. a lookup named url_lookup with a field named url.

With that lookup in hand, you can easily, although slowly, depending on your index configuration, filter the input with a NOT subsearch:

| inputlookup url_lookup where NOT [ index=yourindex sourcetype=yoursourcetype earliest=-2mon| dedup url | table url ]

Replace the index and sourcetype with the correct values or tags or event types for your environment.

(Sorry for the double post. Using mobile.)

View solution in original post

0 Karma

tscroggins
Motivator

Hi,

You'll need a source of current URLs to compare against past events, e.g. a lookup named url_lookup with a field named url.

With that lookup in hand, you can easily, although slowly, depending on your index configuration, filter the input with a NOT subsearch:

| inputlookup url_lookup where NOT [ index=yourindex sourcetype=yoursourcetype earliest=-2mon| dedup url | table url ]

Replace the index and sourcetype with the correct values or tags or event types for your environment.

(Sorry for the double post. Using mobile.)

0 Karma

wbounni11
New Member

Thank you. There is about 80 URLs in that list, would that still work?

Kindly
Wasfi

0 Karma

tscroggins
Motivator

Yes. The limiting factor would most likely be the subsearch, not the lookup. After the subsearch executes, Splunk translates the outer search to:

| inputlookup url_lookup where NOT (url=foo OR url=bar OR url=baz OR url=qux OR ...)

up to the number or live URLs. Internal Splunk limits apply, but the search string itself can be quite long. I don't think the limit is documented, but it's probably related to the maximum request size accepted by splunkweb for manually typed searches and splunkd for derived searches.

0 Karma

Sukisen1981
Champion

care to provide some sample events and expected output :)?

0 Karma
Get Updates on the Splunk Community!

Introducing Ingest Actions: Filter, Mask, Route, Repeat

WATCH NOW Ingest Actions (IA) is the best new way to easily filter, mask and route your data in Splunk® ...

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...