Nested transactions/tables

[BryanBerry]

This is really tricky to explain, so please bear with me. I'm open to different display approaches, so if you disagree with how I want to show this data, please feel free to propose a better design.

I'm working on a TIBCO app to show events for a service run, the job it runs as, and the child jobs running under it. I'm having difficulty is displaying the child jobs in relation to the parent job that represents the whole execution of the service. I have a search like this:

index=test-tibco parent_job_id=80353 | transaction service_name, parent_job_id, child_job_id | eval event_time=_time | convert ctime(event_time) | transaction service_name, parent_job_id | eval newtime=_time | convert ctime(newtime) | table newtime, service_name, parent_job_id, job_status, event_time, child_job_id, MessageText

parent_job_id is only for my testing purposes - I'm more concerned about the formatting.

The results look like this:

newtime	service_name	parent_job_id	event_time	child_job_id	MessageText
04/12/2012 13:27:56.907	DetermineFeatureEligibility	80353	04/12/2012 13:27:56.907 1 Sending Request to BB Service 04/12/2012 13:27:56.909 2 Received response from BB Service 3 4

I want them to look like this:

newtime	service_name	parent_job_id	event_time	child_job_id	MessageText
04/12/2012 13:27:56.907	DetermineFeatureEligibility	80353	04/12/2012 13:27:56.907 1 Sending Request to BB Service 04/12/2012 13:27:56.907 2 <No Message> 04/12/2012 13:27:56.909 3 Received response from BB Service 04/12/2012 13:27:56.909 4 <No Message>

EDIT for additional info: Essentially, my goal is to have a table within my end table to show the child job' info and bubble up into the parent job.

Is this possible? If so, how? (I know the data seems junky, but some of these really are 1-millisecond requests of container jobs and whatnot.)

I'm also curious if anyone has figured out a low-effort manner to achieve the design described in http://splunk-base.splunk.com/answers/40142/how-can-i-compute-durations-of-nested-method-calls.

[cphair]

This is ugly, but it might work for you. This is modified from a query I ran on my own test data, so it may need tinkering.

index=test-tibco parent_job_id=80353 | eval foo=event_time . "::" . child_job_id ."::" . MessageText . "::" |  transaction service_name, parent_job_id | rex max_match=20 field=foo "^(?.?)::(?.?)::(?.*?)::\s" | table service_name, parent_job_id, EventTime, ChildJobId, MessageTextItem

The first eval joins your three target three fields into one field for each event. I used a double colon to join, but it can be any character sequence that doesn't appear in your data. After the transaction command, foo contains the sequence of all events in the transaction. The rex command splits the foo field back into its components. Note the max match parameter must be set to some number greater than 1 to match multiple entries in the field. Unfortunately there doesn't seem to be a way to make it unlimited (setting it to 0 didn't work), so you'll have to set it high enough to cover the maximum number of transaction events in your environment. I don't know what the performance implications of a high max match would be.

Miscellaneous notes: Your MessageText field has spaces in the data, while my test data does not. Since there is a terminating :: delimiter at the end of foo, I think this will still work, but those spaces are something to keep in mind if you have to play with the regex. Also, I tried to keep the three fields joined as one, but the formatting didn't work out; trying to figure out tab characters in headers and such didn't make sense.

I hope this helps.

[sideview]

Hi BryanBerry - if you send me some sample data to nick@sideviewapps.com I'd be happy to tinker with it and give you back a search or two to get what you want. I have been working off and on to make an app for Tibco BWEngine data, and I'm always interested in seeing more sample data in this area. I'd be happy to throw myself at some hard problems for you if you'd share a little data.

[BryanBerry]

@cphair The first transaction was just messing around. I found it made no difference - I should have removed it before posting.

I have not tried that with the eval. That may be a viable option. Any suggestions on how to accomplish the formatting, i.e. have them delimited from one another clearly and show a title for each piece?

Thanks for taking a look

[cphair]

@BryanBerry, I'm not sure I'm parsing your data correctly. Why do you need the first transaction command that includes the child_job_id? Also, have you considered using eval to create a new field that concatenates event_time, child_job_id, and MessageText before you calculate the transaction?