topic Re: Run SPL command once and store result to access faster in Splunk Search

Run SPL command once and store result to access faster

indeed_2000 — Sun, 16 Feb 2020 05:52:23 GMT

Hi
How can I Run SPL command once and store result to access result faster next time.
for e.g. I need to analyses large logs every night and in next day access to "save search" and "dashboards" quickly without waiting to query on data when open "save search" and "dashboards".
I mean every night Splunk after analyze logs run queries on that exist on "save search" and "dashboards" and store output, so next day when I open "save search" and "dashboards" Splunk load result quickly and display them.

Any recommendation?
Thanks

Re: Run SPL command once and store result to access faster

to4kawa — Sun, 16 Feb 2020 13:36:23 GMT

Createandeditreports

check this.

Re: Run SPL command once and store result to access faster

manjunathmeti — Sun, 16 Feb 2020 18:02:16 GMT

You can use summary indexing. Analyze large data and write output to summary index. Use summary index in the report so that it runs faster on less amount of data when compared to raw data.

For more info check this:
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Usesummaryindexing

Re: Run SPL command once and store result to access faster

indeed_2000 — Sun, 16 Feb 2020 18:43:10 GMT

Hi
I try commands (sichart, sitimechart, sistats, sitop, and sirare) But not work.

Below is my SPL and I visual it with “single value“ pivot:
source=index | search error* OR fail*

For e.g. i try this
source=index | sistats search error* OR fail*
source=index | collect search error* OR fail*

Any idea?

Re: Run SPL command once and store result to access faster

to4kawa — Sun, 16 Feb 2020 20:03:05 GMT

first search:

source=index error* OR fail* | collect index=error_index

second search:

index=error_index as_you_like

If you create summary index, The next time you search there, it will be faster.

I think a report is enough for your usage.

Re: Run SPL command once and store result to access faster

martinpu — Sun, 16 Feb 2020 20:48:01 GMT

Save your job as a report, run it based on your schedule and use the loadjob command.

| loadjob savedsearch="admin:search:Savedsearch"

when running loadjob on scheduled report, only latest result will be shown.

You can additionally use these as a base query to run other queries on if you are using multiple transformative commands on the same dataset.

Re: Run SPL command once and store result to access faster

indeed_2000 — Mon, 17 Feb 2020 01:56:41 GMT

Still not work,
|search error* OR fail* | loadjob savedsearch="admin:search:Savedsearch"

What is the last part of command “admin:search:Savedsearch“?

Re: Run SPL command once and store result to access faster

martinpu — Mon, 17 Feb 2020 17:21:27 GMT

| loadjob savedsearch="YourUserID:App:SavedSearchname"

Loadjob must be the first line.

Why are you using:
|search error* OR fail* ?

You can also try

index=yourindex 
| rex "(?<errorOrFail>error|fail)"
|eval errorOrFail=if(isnull(errorOrFail), "False","True")
| search errorOrFail="True"

Or try with

index=yourindex error* OR fail*

see if you can use sourcetype/index or extract some statistical data. I think you need to optimize your query, use index and zero-in on the data that you need.

Re: Run SPL command once and store result to access faster

hc_joycechen — Tue, 18 Feb 2020 09:13:30 GMT

Run SPL command : ....your spl ... | outputlookup nnn.csv
dashboard: | inputlookup nnn.csv

Re: Run SPL command once and store result to access faster

indeed_2000 — Tue, 18 Feb 2020 15:41:25 GMT

1-First command create file? Where it will be atore?
2-load file on dashboard?

Re: Run SPL command once and store result to access faster

hc_joycechen — Wed, 19 Feb 2020 07:02:18 GMT

1-First command create file? Where it will be atore?
| makeresults
| eval aa="1", bb="2", cc="3"
| outputlookup nnn.csv

file : /opt/splunk/etc/apps/your_app/lookups

2-load file on dashboard?

Sample

<panel>
  <table>
    <search>
      <query>| inputlookup nnn.csv</query>
    </search>
    <option name="count">30</option>
    <option name="drilldown">none</option>
    <option name="refresh.display">progressbar</option>
  </table>
</panel>