https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265122#M79686 <P>I find it helps to visualise Splunk as a single giant database table with unlimited columns. So in fact your events are already "joined" and all you need to do is group statistics together in a meaningful way.</P> <P>Assuming each mID,jID pair is unique, you can group all you fields together like so</P> <PRE><CODE>index=performance (source=subtask OR source=maintask ) | stats values(*) AS * by mID,jID </CODE></PRE> <P>This should give you a table of all the fields that belong to each mID jID pair. The fields will be multivalued, so you may need to split them with mvexpand</P> <P>There is a great flowchart on when you use each aggregation command here: <A href="http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Abouteventcorrelation">http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Abouteventcorrelation</A></P> <P>See Stats: <A href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Stats">http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Stats</A> <BR /> See Mvexpand: <A href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Mvexpand">http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Mvexpand</A></P> Fri, 05 Feb 2016 10:41:34 GMT jplumsdaine22 2016-02-05T10:41:34Z https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265119#M79683 <P>I have a new application that I need to extract one field, <STRONG>taskname</STRONG>, from the main task table (5+ million records) and merge into the subtask table (less 1 million records). When I ran with the 'transaction' statement, I got incomplete results.</P> <P>Here are my file structures. One main task will have multiple subtasks and each subtask might run in serialized or parallel.</P> <PRE><CODE>Main taskA (mID, jID, mCPU, taskname, main task beginning/ending time) subtask1: mID, jID, sCPU, subtask beginning/ending time subtask2: mID, jID, sCPU, subtask beginning/ending time . . . . . </CODE></PRE> <P>I need to search each detailed subtask attributes by task name across different time frame. And, another search will be summarized by time distribution. I know the later one can be achieved with <CODE>| timechart sum(sCPU) by taskname span=1h</CODE>. But, I got the performance issue for the first search. I did search to use summary indexing and post processing. It seems they suggest to aggregate fields mID and jID. However, I will lose the subtask detailed information. Any suggestion?</P> Wed, 03 Feb 2016 01:29:43 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265119#M79683 jlo 2016-02-03T01:29:43Z https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265120#M79684 <P>Can you post the query with <CODE>transaction</CODE> command that you are using?<BR /> Is your main task table and subtask table are different files?</P> Wed, 03 Feb 2016 05:01:35 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265120#M79684 Murali2888 2016-02-03T05:01:35Z https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265121#M79685 <P>the base search,</P> <P>index=performance source=subtask OR source=maintask <BR /> | transaction mID jID startswith=(source="subtask") endswith=(source="maintask")</P> <P>....</P> <P>use the search base for the query1</P> <P>| timechart sum(sCPU) by taskname span=1h</P> <P>....</P> <P>use the search base for the query2</P> <P>| table mID jID sCPU taskname sATT1 sATT2 sATT3<BR /> | sort 0 -sCPU | head 100</P> <P>where sATTx are minor test attributes from the source=subtask; and taskname is from source=maintask.</P> Wed, 03 Feb 2016 06:45:02 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265121#M79685 jlo 2016-02-03T06:45:02Z https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265122#M79686 <P>I find it helps to visualise Splunk as a single giant database table with unlimited columns. So in fact your events are already "joined" and all you need to do is group statistics together in a meaningful way.</P> <P>Assuming each mID,jID pair is unique, you can group all you fields together like so</P> <PRE><CODE>index=performance (source=subtask OR source=maintask ) | stats values(*) AS * by mID,jID </CODE></PRE> <P>This should give you a table of all the fields that belong to each mID jID pair. The fields will be multivalued, so you may need to split them with mvexpand</P> <P>There is a great flowchart on when you use each aggregation command here: <A href="http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Abouteventcorrelation">http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Abouteventcorrelation</A></P> <P>See Stats: <A href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Stats">http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Stats</A> <BR /> See Mvexpand: <A href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Mvexpand">http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Mvexpand</A></P> Fri, 05 Feb 2016 10:41:34 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-deal-with-performance-issues-using-the-transaction/m-p/265122#M79686 jplumsdaine22 2016-02-05T10:41:34Z