Connect two transactions

[Simon]

Hi guys

Have a look at my events indexed in Splunk:

Jan 12 09:29:11 myhost -bash: HISTORY: PID=28489 UID=501 id
Jan 12 09:29:11 myhost -bash: HISTORY: PID=28489 UID=501 pwd
Jan 12 09:29:09 myhost -bash: HISTORY: PID=28489 UID=501 echo hi
Jan 12 09:29:06 myhost bash: HISTORY: SUDO_UID=10002 PID=28489 New session as user mytestuser (UID=501) started by foo.bar (ORIG_UID=10002)
Jan 12 09:29:06 myhost -bash: HISTORY: PID=28474 UID=10002 sudo -i -u mytestuser
Jan 12 09:29:04 myhost -bash: HISTORY: PID=28474 UID=10002 pwd
Jan 12 09:29:04 myhost -bash: HISTORY: PID=28474 UID=10002 ls -als

As you can see, there are basically two different transactions when using the PID field as transaction identifier. Between those two transactions, there's an event which helps me to connect both of them.
Any idea how to use the transaction search command, to connect those two transactions using the event with the additional informations?

Note: If necessary, I can rename the fields in the event with the additional informations.

Thanks
Simon

[southeringtonp]

Try normalizing the field name before calling transaction. This will give you one transaction instead of two.

If each line in your original post is a separate event, then you should be able to do something like:

eval txuid=coalesce(ORIG_UID, UID) 
| transaction txuid

The eval statement will create a new field which contains the ORIG_UID if it exists, and otherwise contains the UID. The transaction will be based on this normalized value, so should include all events in your example.

Also, you may want to consider using the startswith option to transaction -- e.g., transaction startswith=(SOMETHING) -- to make sure that each login session is captured in a separate transaction.

[rtadams89]

What if you did a transaction comamnd to join events with the same PID. In your exampel above, you would then have two resultant events. The event with "PID=28489 UID=501" would the nalso have a field "ORIG_UID". Then pipe that to a selfjoin where ORIG_UID=UID.

[Simon]

That was giving me some interesting hints, thanks. Unfortunately, the selfjoin comman doesn't support anything else than joining by a single field.
So I tried to get a field, which contains the same value in both results after the 'transaction PID', like this:
searchterm | transaction PID | eval TRANS_UID=if(len(ORIG_UID)>0,ORIG_UID,UID)
Now piping this to selfjoin results in showing only the first event of the 'transaction PID' command... Using transaction TRANS_UID results in what I've noted in the comment above:
Splunk reports one matching event but doesn't show anything.
Any ideas?

[Drainy]

Could you not just take your original transaction that returns the above and pipe that into another transaction with the event that connects the two?

Ok so you would perform a generic search to gather your data and you could then pipe to transaction to combine the events. You could then pipe it again to transaction to combine events based on a different field, so lets assume that HISTORY is the similar factor and you want to combine all transactions that have HISTORY, and we'll assume this HISTORY is extracted to field 'type'.

searchterm | transaction pid | transaction type

From memory I believe that would do what you want to achieve. Transaction is quite a heavy command though so others may have a better solution and you wouldn't want to do this over particularly large time periods unless you had the time to wait. Have a look at the docs (which I assume you already have from your comment) to see ways to extend and tune this search;
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/transaction

EDIT:
If you run the first transaction what is returned? If you then pipe into a second transaction and include keeporphans=true what is returned?
this will force Splunk to return events that didn't meet the transaction rules.

[Drainy]

yes this is something it can do. Essentially with the pipe function you are passing the results from the previous search along to the next search command, this will then apply the filter to the events. In the first transaction you construct events that have a certain field that matches. In the second pipe it will treat these as events and apply the transaction again. Read my second Edit I'll add for some other bits to try.

[Simon]

Draineh, somehow piping results of a transaction into another one, results in showing there was one matching event, but no results are displayed... I can try this e.g. like that:
searchterm | transaction PID | transaction sourcetype
or whatever... always the same result. Is piping transaction results in another transaction command basically supported by splunk?

[Drainy]

Oh ok, I will edit my answer with a little more detail then

[Simon]

Hey Draineh, thanks for your answer. The output above is the result of a very simple search, it's no transaction yet. There might be other events before and between these lines, of course with different values in the fields. Now I'd like to have the events above in a single transaction block. Using connected=true didn't help so far.