Is there a search to show bundle size in the Dispatch Runner configuration initialization?

[randy_moore]

We are getting:

Dispatch Runner: Configuration initialization for splunk\var\run\searchpeers\ really long string of letters and numbers took longer than expected.

Confirmed that it not a disk IO slowdown/bottleneck/latency , so one of the other options is that a bundle size is huge. Not sure how to see that though. Is there a search or REST query that will let me see the bundle size?

Looking at you @woodcock 🙂

[cmerriman]

In Splunk v 7.3+, you can use the rest call, as long as your lookup tables have definitions in transforms.conf created.

| rest splunk_server=* /servicesNS/-/-/data/transforms/lookups getsize=true f=size f=title f=type f=filename f=eai*|fields splunk_server filename title type size eai:appName
|where isnotnull(size)|eval MB = round(size / 1024 / 1024, 2)|search MB>{0}|fields - size

The docs do not have the getsize param defined yet, but there has been feedback submitted to have it added.

[splunkreal]

|search MB>0 without {} works

[woodcock · Accepted solution]

I use this in my health dashboards:

<query>|rest/services/data/lookup-table-files splunk_server=local
| search eai:acl.app="$env:app$"
| rename dispatch.* AS *
| rename eai:acl.* AS *
| map maxsearches=99 search="
| inputlookup $$title$$
| rename COMMENT1of3 AS \"Some field names have single-quotes which will cause this error:\"
| rename COMMENT3of3 AS \"{map}: Failed to parse templatized search for field 'Bad Field's Name Here'\"
| rename COMMENT3of3 AS \"So rename those fields before we process them to replace ' with _\"
| rename *'*'*'*'* AS *_*_*_*_*, *'*'*'* AS *_*_*_*, *'*'* AS *_*_*, *'* AS *_*
| eval T3MpJuNk_bytes=0, T3MpJuNk_cols=0, T3MpJuNk_field_names=\",\"
| foreach _*
[ eval T3MpJuNk_bytes = T3MpJuNk_bytes + coalesce(len('<<FIELD>>'), 0)
| eval T3MpJuNk_cols = T3MpJuNk_cols + 1
| eval T3MpJuNk_field_names = T3MpJuNk_field_names . \"<<FIELD>>\"]
| rename _* AS *, T3MpJuNk_* AS _T3MpJuNk_*
| foreach *
[ eval _T3MpJuNk_bytes = _T3MpJuNk_bytes + coalesce(len('<<FIELD>>'), 0)
| eval _T3MpJuNk_cols = _T3MpJuNk_cols + 1
| eval _T3MpJuNk_field_names = _T3MpJuNk_field_names . \"<<FIELD>>\"]
| rename COMMENT AS \"Account for the commas, too!\"
| eval bytes = bytes + (cols - 1)
| stats sum(_T3MpJuNk_bytes) AS bytes count AS lines first(_T3MpJuNk_cols) AS cols first(_T3MpJuNk_field_names) AS field_names
| rename COMMENT AS \"Account for the header line, too!\"
| eval bytes = bytes + (len(field_names) - 1)
| eval title=\"$$title$$\"
| eval owner=\"$$owner$$\""
| eval bytes = coalesce(bytes, 0)
| addtotals row=false col=true labelfield=title label="$TOTAL_FIELD_VALUE$"
| eval "bytes/line" = if(title=="$TOTAL_FIELD_VALUE$", "N/A", round(coalesce(bytes/lines, 0), 2))
| eval owner = if(title=="$TOTAL_FIELD_VALUE$", "N/A", owner)
| eval cols  = if(title=="$TOTAL_FIELD_VALUE$", "N/A", coalesce(cols, "N/A"))
| eval MB = round(bytes / 1024 / 1024, 2)
| eval bundlePct = round(100 * bytes / 838860800, 2)
| eval status=case(
   title=="$TOTAL_FIELD_VALUE$", if((bundlePct < 90),                         "OK", "DANGEROUS TERRITORY"),
   true(),                       if((bundlePct < 25 AND lines < 10000000), "OK", "Consider KVStore"))
| sort 0 - bytes
| table title status bundlePct owner bytes MB lines cols bytes*line
| eval _drilldown  = if(title=="$TOTAL_FIELD_VALUE$", "*", title)</query>

[woodcock]

See the other answer from @cmerriman; it is the right way to do it.

[kevinmabini]

Hi @randy_moore, @woodcock -

This one's a really good SPL to determine the status of the bundles. However, i tried to run it in my side and i'm not getting any data for the bytes and other numerical fields. Though i can see the lookup names and owners.

Hope you can give insights on this. Thank you in advance for your assistance!

[woodcock]

You probably do not have enough admin permissions to access the REST endpoint of other people's KOs. Take the general debug approach. Strip off pipes ( | ) of commands from the bottom, moving up until the portion that is left is doing what you would expect it to do. Then move forward adding them back in until you find what is not working. Then make it work.

[kevinmabini]

Found the problem. I forgot that the code you posted was from a dashboard. I just had to remove the extra '$' in the title and owner. It's working fine now. Thanks @woodcock ! 🙂

[woodcock]

Don't forget to UpVote!

[randy_moore]

Modified it a bit and it works like a charm . Thanks @woodcock !

[woodcock]

Please post or DM me your improvement @randy_moore!

[randy_moore]

I only needed to modify a couple of lines:

this one
From:
| search eai:acl.app="$env:app$"
to
| search eai:acl.app="search"

since (1) my search isn't via a dashboard, and yours is, and (2) we run a pretty vanilla splunk env. No customized apps until next year.

Also removed the $$ around "title" and "owner" and made them single $... see #1 above for why.

Changed the sort..because I like mine better.

[woodcock]

But how did you know that I had that dandy in my back pocket? Are you spying on me?

[randy_moore]

Splunk user groups slack channel (splunk-usergroups.slack.com) is my friend 🙂