topic Re: How can we speed up JSON array lookup? in Getting Data In

How can we speed up JSON array lookup?

vinodkd — Tue, 22 Jan 2013 23:03:58 GMT

Hi,

My JSON event is in this form:

{
TotalMemUsage : 887992,
ProcMemUsage : [
{
Name : "firefox",
PID : 758,
Usg : 228972,
},
{
Name : "eclipse",
PID : 569,
Usg : 19756,
}
]
}

I've to take average of each process's memory usage and draw a chart.
For the time being, I use following query.

my_search 
| rename ProcMemUsage{}.Name as PNAME | rename ProcMemUsage{}.Usg as PUSAGE
| eval x=mvzip(PNAME,PUSAGE) 
| mvexpand x|eval x=split(x,",")
| eval P_NAME=mvindex(x,0)
| eval P_USAGE = mvindex(x,1)
| stats avg(P_USAGE) as MU by P_NAME 
| sort -MU | fields MU,P_NAME
|chart avg(MU) as AvgMU by P_NAME 
| sort -AvgMU

But it takes lot of time to complete the operation. (Approx 5 minutes with only 30K records.).
Is there any way to optimize it? Can we use jsonutils somehow?

Re: How can we speed up JSON array lookup?

lguinn2 — Tue, 22 Jan 2013 23:53:07 GMT

I don't really think this will make things faster, but it might help a little:

my_search 
| rename ProcMemUsage{}.Name as PNAME | rename ProcMemUsage{}.Usg as PUSAGE
| fields PNAME PUSAGE
| eval x=mvzip(PNAME,PUSAGE) 
| mvexpand x
| eval x=split(x,",")
| eval P_NAME=mvindex(x,0)
| eval P_USAGE = mvindex(x,1)
| chart avg(P_USAGE) as AvgMU by P_NAME 
| sort -AvgMU

It's also a bit cleaner. Post back with your results.

Re: How can we speed up JSON array lookup?

vinodkd — Wed, 23 Jan 2013 23:14:49 GMT

@Iguinn: It doesn't give a noticeable speedup :'(