topic Complex subsearch: Need to extract value of field and pipe into another search in Splunk Search

Complex subsearch: Need to extract value of field and pipe into another search

whateverman — Thu, 21 Feb 2013 21:06:12 GMT

So I’m trying to link a couple different fields together to get the data I’m looking for, but it involves a couple steps and not sure how to put this subsearch together. I’ve been able to extract fields, but still some more steps to go. Anyway here is what I’m trying to do:

1 - Find MID’s from lines with “Veriifcation-Fail”

Jan 31 11:43:28 x.x.x.x mail_logs: Info: MID 401106733 Custom Log Entry: Verification-Fail

2 - Then search for that same MID to find the associated ICID

Jan 31 11:44:35 x.x.x.x mail_logs: Info: MID 1252214406 ICID 2116684223 From: user@company.com (with < > around email - will not come up in this editor)

3 - Then I take the ICID and search for that

grep 2116684223 logfile

Jan 31 11:44:35 x.x.x.x maillogs: Info: New SMTP ICID 2116684223 interface Data 1 (10.10.10.10) address 10.10.10.10 reverse dns host host.domain.local verified no

The data I need are the FROM: and the SMTP IP and HOST in bold above.

Any help at all is appreciated. Very new to splunk, but just need nudge in the right direction.

Thanks.

Re: Complex subsearch: Need to extract value of field and pipe into another search

sideview — Fri, 22 Feb 2013 03:55:28 GMT

I think this is very similar to some other questions that have nice detailed answers --

http://splunk-base.splunk.com/answers/29172/transaction-problems-with-lots-of-events-and-multiple-fields

http://splunk-base.splunk.com/answers/29489/using-transaction-with-one-to-many-relationships

I would check out the transaction command thoroughly, and just put the subsearch idea on hold for a while. Then after you feel you have a grasp for transaction as a tool take a look at searchtxn. The searchtxn command was to a significant extent created precisely to address dcid - mid - icid use cases, however I think you'll get the hang of it faster if you go slow and bone up on transaction first.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction

http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Searchtxn

Re: Complex subsearch: Need to extract value of field and pipe into another search

whateverman — Fri, 22 Feb 2013 07:27:08 GMT

ok, thanks! I just read the transaction reference page, and I feel much more confident. Exactly what I was looking for. Let's hope it works well though. I will update with my results. Thanks!