topic Re: Ironport WSA "Top Google Search Terms" query in Splunk Search

Ironport WSA "Top Google Search Terms" query

imarks001 — Wed, 23 Jun 2010 19:20:05 GMT

I am trying to come up with a search that would parse Google search queries made though my Ironport web proxy. I would like to add this as a real time report on a dashboard eventually. Does anyone have any tips for a regex that could parse the a Google search request URL and display on the words in the search? Bing results would be great too.

Thanks

Re: Ironport WSA "Top Google Search Terms" query

Simeon — Wed, 23 Jun 2010 23:13:01 GMT

Your question is confusing as there is no sample data to reference. Also, your use of parse is confusing. My guess is that you want to produce an extracted field for query terms within google and bing URLs. Please update your question with more details.

Re: Ironport WSA "Top Google Search Terms" query

imarks001 — Mon, 28 Sep 2020 09:13:57 GMT

Sorry for the confusion. I would be looking to parse the actual search query out of our WSA logs. For instance, I would want to pull "world cup winners" out of the following log entry and be able to report on top searches.

1277318549.935 295 192.168.150.6 TCP_MISS/200 9957 GET http://www.google.com/search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&gs_rfai=Cqt3eklUiTPiZO4XyzASIzfSXBgAAAKoEBU_Q0UPQ&fp=d2eda4910a7445ea - DIRECT/www.google.com text/html DEFAULT_CASE-NoAuthPolicy-NoAuthGlobal-NONE-NONE-DefaultRouting

Re: Ironport WSA "Top Google Search Terms" query

imarks001 — Mon, 28 Sep 2020 09:14:00 GMT

The URL is getting truncated when I post it.

Here is everything after google.com

search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&gs_rfai=Cqt3eklUiTPiZO4XyzASIzfSXBgAAAKoEBU_Q0UPQ&fp=d2eda4910a7445ea

Re: Ironport WSA "Top Google Search Terms" query

tidavids — Sat, 03 Jul 2010 22:02:53 GMT

Assuming you have the WSA app installed that extrapolates the cs_url field, try the following:

"www.google.com/search" "q=" | rex field=cs_url "q=(?P<search>[^&]*)\&" | rex field=search mode=sed "s/\+/ /g" | top search

Re: Ironport WSA "Top Google Search Terms" query

imarks001 — Fri, 09 Jul 2010 08:36:39 GMT

Thanks, I tried this but I receive the following error. 1. Error in 'rex' command: Encountered the following error while compiling the regex 'q=(?P[^&]*)&': Regex: unrecognized character after (?P

Re: Ironport WSA "Top Google Search Terms" query

pschulz — Mon, 28 Sep 2020 14:30:51 GMT

If someone's stumbling upon this, I've made a simple field extration regex which does the job by filtering out all text between ampersands (&) where "q=" is showing up.

Just go to "Manager" - "Field extractions" - Click "New" and enter the line below in the field "Extraction/Transform" after choosing your app & source/sourcetype/host:
(?i)&q=(?P<_google_query>\w+[^&]*)&

Regards,
Phil

PS: Maybe someone will cure the bug of not beeing able to post "<thisisnotag!!!>" as it's regarded as html-tag 😉

Re: Ironport WSA "Top Google Search Terms" query

nkpiquette — Wed, 07 Jan 2015 18:40:45 GMT

Digging up an old threat, but this just came in handy. The sed I had to use in 6.2 was as follows:
rex field=search mode=sed "s/+/ /g"