Splunk Search

Ironport WSA "Top Google Search Terms" query

imarks001
Explorer

I am trying to come up with a search that would parse Google search queries made though my Ironport web proxy. I would like to add this as a real time report on a dashboard eventually. Does anyone have any tips for a regex that could parse the a Google search request URL and display on the words in the search? Bing results would be great too.

Thanks

Tags (2)

pschulz
Engager

If someone's stumbling upon this, I've made a simple field extration regex which does the job by filtering out all text between ampersands (&) where "q=" is showing up.

Just go to "Manager" - "Field extractions" - Click "New" and enter the line below in the field "Extraction/Transform" after choosing your app & source/sourcetype/host:
(?i)&q=(?P<_google_query>\w+[^&]*)&

Regards,
Phil

PS: Maybe someone will cure the bug of not beeing able to post "<thisisnotag!!!>" as it's regarded as html-tag 😉

0 Karma

tidavids
Engager

Assuming you have the WSA app installed that extrapolates the cs_url field, try the following:

"www.google.com/search" "q=" | rex field=cs_url "q=(?P<search>[^&]*)\&" | rex field=search mode=sed "s/\+/ /g" | top search

nkpiquette
Path Finder

Digging up an old threat, but this just came in handy. The sed I had to use in 6.2 was as follows:
rex field=search mode=sed "s/+/ /g"

0 Karma

imarks001
Explorer

Thanks, I tried this but I receive the following error. 1. Error in 'rex' command: Encountered the following error while compiling the regex 'q=(?P[^&]*)&': Regex: unrecognized character after (?P

0 Karma

imarks001
Explorer

The URL is getting truncated when I post it.

Here is everything after google.com

search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&gs_rfai=Cqt3eklUiTPiZO4XyzASIzfSXBgAAAKoEBU_Q0UPQ&fp=d2eda4910a7445ea

0 Karma

imarks001
Explorer

Sorry for the confusion. I would be looking to parse the actual search query out of our WSA logs. For instance, I would want to pull "world cup winners" out of the following log entry and be able to report on top searches.

1277318549.935 295 192.168.150.6 TCP_MISS/200 9957 GET http://www.google.com/search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&g... - DIRECT/www.google.com text/html DEFAULT_CASE-NoAuthPolicy-NoAuthGlobal-NONE-NONE-DefaultRouting

0 Karma

Simeon
Splunk Employee
Splunk Employee

Your question is confusing as there is no sample data to reference. Also, your use of parse is confusing. My guess is that you want to produce an extracted field for query terms within google and bing URLs. Please update your question with more details.

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...