Best way to create a transaction of three events?

[dnolan]

I've got data that looks (functionally) like this:

Event 1 contains String-A Field-X
Event 2 contains String-B Field-X Field-Y
Event 3 contains String-C Field-Y

I'm trying to correlate these three events together. I want to create a single transaction so I can calculate the duration between Event 1 and Event 2.

After some trial and error I came up with this query:

String-A OR (String-B Field-Y=*) | transaction Field-X | append [search String-C] | transaction Field-Y

That works, but it seems really ugly to me, and I'm wondering if I'm missing some better way to do this. Ideally I'd also like to be able to find examples of Event-1 with no matching Event-3, but I think I can do that with transaction as well, using keepevicted=1.

Any suggestions for a better path to my desired results?

Edit:

Ok, maybe I boiled down my example too much. Here's the original data:

Event 1:

2,51216,117,117,626,1284525367018,4172,6876,17:reqid=-1284474701,0:,0:,0,(49|A136:Msg: type = 39 BRMWroteMsg: backupid=us-pghdbp1.ariba.com_1284523604, KBytes written=3444, End of Data=1, KBytes per sec=20935.5, copy=0|)

This is a Symantec NetBackup job completing. The key pieces of Event 1 are "End of Data=1" (There were lots of records before this with that value as 0), and the backupid. I'm extracting the backupid as a field.

Event 2:

0,51216,111,111,60899,1284525448888,7992,7172,0:,203:
    ImageCopyExt_Record => {
        MasterServerName => hubmaster
        BackupID => us-pghdbp1.ariba.com_1284523604
        CopyNumber => 2
        JobID => 2698
        LifecycleSourceTag => 1
    },33:ImageCatalogImpl::updateImageCopy,1

This is Symantec scheduling the data in this backupid as part of a replication job between storage arrays. They may be multiple of these entries, for multiple backup objects all grouped together in one replication. The key elements I've identified are "ImageCopyExt_Record", the backupid (same as event 1) and the jobid (will match event 3). Other events exist with the search string and the backupid but without a jobid.

Event 3:

2,51216,116,116,5,1284526119527,6788,7860,10:jobid=2701,0:,0:,0,(14|A4:NONE|A10:SLP_HUBSLP|A3:Dup|A1:0|A50:the requested operation was successfully completed|)

This is Symantec logging that the replication completed. Here we see the matching jobid from event 2, 'A3:Dup' means this is about a replication event, and the string says completed.

So the query I had built was:

("End of Data=1" OR (ImageCopyExt_Record netbackup_jobid=*)) | transaction netbackup_backupid | append [search index=backups A3:Dup completed] | transaction netbackup_jobid 

I'm grabbing event1 and event 2 and transactioning them on the backupid, then append to that the event3 records and transactioning again. I'm sure there is a better way.

[gkanapathy · Accepted solution]

Please see my comment, as I really have no idea how this is supposed to work, but I wonder if:

String-A OR String-B OR String-C | transaction Field-X Field-Y

would do what you want.

[gkanapathy]

Yes, transaction will be transitive on each field. If a field has a value, it won't link, but if it is null, then it will be allowed to link. This behavior I think is related to the connected=t option.

[dnolan]

I swear I tried that yesterday and it didn't give me useful results.

But I just tried it again and what I found was it found cases where event1 exist but no event2 or event3. (Either backups that aren't configured to replicate, or backups that are still replicating.) So I added to the end of that "| search completed" and it works. Somehow I didn't expect transaction to transaction on EITHER field, but on BOTH. Thanks!

[gkanapathy]

It's not clear at all to me what your transaction is supposed to do. You have abstracted away everything and provided a single example, so I find it very difficult to understand your intention, i.e., what you want it to do. I can see what your search does do, but with no way to generalize, I can't come up with something better.