Check for users clicking on a totally new url in last 24 hrs

[abhik1501]

I need to search for users who clicked on totally new urls seen in last 24 hrs. If user has clicked on a link which was already previously seen i do not need those events.
Any inputs will be really helpful

[woodcock]

You need to maintain a lookup file with something like ... | stats min(_time) AS _time BY url | outputlookup firstTimeURLs.csv and then xref with ... | lookup firstTimeURLs.csv url OUPUT url AS seen | where isnull(seen) against your data.

[to4kawa]

the csv of daily access list for each user.

Is the proxy log in Splunk?

[abhik1501]

Yes proxy log is in Splunk. There cannot be a specific daily access list for each user because numbers of users are high and no restriction on the number of url accessed

[to4kawa]

index=your_index sourcetype=proxy 
| rex field=url "https?:\/\/(?<domain>[^\/]+)"
| stats values(eval(if(status=200,domain,NULL))) as domain
| mvexpand domain
| eval flag="access"
| outputcsv domain.csv

First, create access domain csv, and

index=your_index sourcetype=proxy 
| rex field=url "https?:\/\/(?<domain>[^\/]+)"
| inputlookup append=t domain.csv
| where flag!="access"
| stats values(domain) as domain by user

Second, check new access domian by user.

It seems impossible to create a list of access domains for each user.