Splunk Search

How to exclude web crawlers from site traffic

chuckeelos
New Member

Hi, I'm trying to exclude as many crawl bots from my search and show only human hits on our website. I found the search code below when googling, but it shows me all the non-human sessions. Can anyone explain how to exclude these sessions? Or another way of showing all non-bot traffic? Thanks!

index="main" sourcetype="access_combined"
| eval usersession=clientip + "_" + useragent
| sort usersession, _time
| delta _time as visit_pause p=1
| streamstats current=f window=1 global=f last(usersession) as previous_usersession
| eval visit_pause=if(usersession==previous_usersession, visit_pause, -1)
| search visit_pause!=-1
| stats var(visit_pause) as variance by usersession
| search variance<5

Tags (2)
0 Karma

chuckeelos
New Member

Hello, Anyone?

0 Karma
Get Updates on the Splunk Community!

Celebrating Fast Lane: 2025 Authorized Learning Partner of the Year

At .conf25, Splunk proudly recognized Fast Lane as the 2025 Authorized Learning Partner of the Year. This ...

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...