Splunk Search

How do we edit this search to combine our 2 individually working searches?

sm600
Explorer

Our security analyst is having an issue with his search...and I cannot for the life of me figure out the issue. Am I missing a pipe or a parenthesis?

This works perfectly - returns a specific message ID

index=mta_syslog_log CLKPER "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com"|table message_id

This works perfectly - returns data about that specific message ID

index="mta_syslog_log" CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com | table x

This FAILS:

index="mta_syslog_log" [search index="mta_syslog_log" [search index=mta_syslog_log CLKPER "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com"|table message_id] | table x]  | transaction x keepevicted=true|table _time,from,rcpt,subject,file

Thanks,

Steve

Tags (4)
0 Karma
1 Solution

woodcock
Esteemed Legend

You are starting with this:

 index="mta_syslog_log" [search index="mta_syslog_log" [search index=mta_syslog_log CLKPER "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com"|table message_id] | table x] | transaction x keepevicted=true|table _time,from,rcpt,subject,file

Which starting from the middle and working out, will change to this:

index="mta_syslog_log" [search index="mta_syslog_log" message_id=SomeValueForMessageID | table x] | transaction x keepevicted=true|table _time,from,rcpt,subject,file

Which will change to this:

index="mta_syslog_log" x=SomeValueForX | transaction x keepevicted=true|table _time,from,rcpt,subject,file

It makes no sense to do transaction x after you have already done x=SomeValue. It should still "work" (return a single transaction event) but there seems to be something more fundamentally wrong with your approach.

Frequently when a person asks a question about how to wrap up a search that is using a transaction command, what he really needs is to back up and explain his end goal so that he can get a solution that does not use transaction. I think you may be in that situation. What is it that you are really trying to do and explain it generically using data, rather than using SPL (searches).

View solution in original post

0 Karma

woodcock
Esteemed Legend

I think perhaps you are running into some variation of this problem:
http://blogs.splunk.com/2011/10/07/cannot-search-based-on-an-extracted-field/

I believe you are saying that doing a search like this:

index=blah "SomeString"

Gives a different value than this:

index=blah field="SomeString"

Which is the syntax that is returned from subsearches.

0 Karma

woodcock
Esteemed Legend

You are starting with this:

 index="mta_syslog_log" [search index="mta_syslog_log" [search index=mta_syslog_log CLKPER "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com"|table message_id] | table x] | transaction x keepevicted=true|table _time,from,rcpt,subject,file

Which starting from the middle and working out, will change to this:

index="mta_syslog_log" [search index="mta_syslog_log" message_id=SomeValueForMessageID | table x] | transaction x keepevicted=true|table _time,from,rcpt,subject,file

Which will change to this:

index="mta_syslog_log" x=SomeValueForX | transaction x keepevicted=true|table _time,from,rcpt,subject,file

It makes no sense to do transaction x after you have already done x=SomeValue. It should still "work" (return a single transaction event) but there seems to be something more fundamentally wrong with your approach.

Frequently when a person asks a question about how to wrap up a search that is using a transaction command, what he really needs is to back up and explain his end goal so that he can get a solution that does not use transaction. I think you may be in that situation. What is it that you are really trying to do and explain it generically using data, rather than using SPL (searches).

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Keep in mind that subsearches can return OR'd lists of many values for the outer search to filter by.

0 Karma

johnmccash
Explorer

Steve was posting this on my behalf. The "| transaction x keepevicted=true|table _time,from,rcpt,subject,file" portion is NOT the problem.

The query "index=mta_syslog_log CLKPER CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com|table message_id"

is just a standin, to return the messageid for further processing. It does, in fact, return the string "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com"

and if I use the remainder of the query, with the messageid plugged-in literally, "index="mta_syslog_log" CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com | table x", that returns the proper x values for further processing.

but when I embed the original query into the other one, as follows, I do not get the same x results as above:
"index="mta_syslog_log" [search index=mta_syslog_log CLKPER CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com|table message_id] | table x"

Does that make more sense?
Thanks
John McCash

0 Karma

woodcock
Esteemed Legend

What do you get when you use message_id="BLAH" instead of "BLAH" everywhere? Do your simpler examples then also fail? Splunk subsearches return field="value", not "value", so your "tests" are not (necessarily) exactly equivalent (see my new answer).

johnmccash
Explorer

That was it! Thanks! The message id was getting parsed into a different field name in some of the other entries.

Final search is:
index="mta_syslog_log" [search index="mta_syslog_log" [search index=mta_syslog_log CLKPER "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com" |eval hdr_mid=messageID|table hdr_mid]|table x] | transaction x keepevicted=true|table _time,from,rcpt,subject,file

next stupid question.... How to include a field in the final output that's a different field which was returned by that very first subsearch "search index=mta_syslog_log CLKPER "CA+v91aLeKkW-cLRx67G-ASwtNLAKnWT5xmfSVz4bOe78fgTibg@mail.gmail.com"

Thoughts?

0 Karma

woodcock
Esteemed Legend

I don't understand your new question but it would probably be best to accept an answer and close this question off and ask a new question.

0 Karma

woodcock
Esteemed Legend

Right but not in his example, which may or may not (my assumption) have been reduced for simplicity.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...