Splunk Search

How to use rex to parse Select Query in a log?

dkarthik16
New Member

Hi,

I have a log like below

"12","select a.a,b.b,c from a,b where a.a = b.a group by xxxx","impala",2017-06-30T00:59:40

I am trying to parse this to extract where condition for some performance analysis.

I tried this in regex101 and got the below regex

\"\d+\",\"select\s+(.\*)\s+from\s+(.\*)\s*where\s(.\*)\s+group by\s+(.\*)\s*\",(.\*)\s*,(.\*)

Help me to implement it in SPL.

Tags (2)
0 Karma
1 Solution

jodyfsu
Path Finder

I think you have it worked out for the most part. not seeing an actual example leaves a lot of possible incorrectness, but here is what I think you are after.
| rex "\"\d+\",\"select\s+(.*)\s+from\s+(.*)\s*where\s(?<WHERECAPTURE>.*)\s+group by\s+(.*)\s*\",(.*)\s*,(.*)"

Let us know if you need more.

View solution in original post

0 Karma

jodyfsu
Path Finder

I think you have it worked out for the most part. not seeing an actual example leaves a lot of possible incorrectness, but here is what I think you are after.
| rex "\"\d+\",\"select\s+(.*)\s+from\s+(.*)\s*where\s(?<WHERECAPTURE>.*)\s+group by\s+(.*)\s*\",(.*)\s*,(.*)"

Let us know if you need more.

0 Karma

jodyfsu
Path Finder

And, if you are only trying to capture the WHERE:

| rex "where\s(?<WHERECAPTURE>.*)\s"
0 Karma

dkarthik16
New Member

Thanks ... And as you mentioned there are some mismatches in actual logs and Since the query was written by Humans which doesn't follow any meaningful pattern. Actually I extended my analysis to extract JOINS also.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...