topic Re: Indexing the results of a search in Splunk Search

Indexing the results of a search

beano501 — Fri, 04 Jul 2025 10:07:45 GMT

All,

I'm ingesting data from Azure that contains (as part of it) a syslog message, I have the vendor specific application for this syslog message format.

Simplified structure below.

{ CollectorHostName: xxxxx Computer: xxxxx EventTime: 2025-06-27T07:19:45Z Facility: local7 HostIP: xx.xx.xx.xx SyslogMessage: logver=704072731 timestamp=1750983585 devname="xxx" devid="xxx" vd="root" date=2025-06-27 time=00:19:45 eventtime=1751008785866135964 tz="-0700" logid="0100032002" type="event" subtype="system" level="alert" ........." }

What I would ideally like to be able to do, is extract (via a search) the "SyslogMessage" field and then re-index this into a new index (and appropriate sourcetype) so that the syslog message can be processed in "the normal way" by the vender specific application.

Does anyone know how I can achieve this? Many thanks in advance.

Re: Indexing the results of a search

ITWhisperer — Fri, 04 Jul 2025 10:16:17 GMT

Have you considered using a summary index to hold the extracted field(s)?

Re: Indexing the results of a search

gcusello — Fri, 04 Jul 2025 10:43:36 GMT

Hi @beano501 ,

as also @ITWhisperer hinted, the best solution is scheduling a search that extracts only the fields you need and storage results in a summary index using the collect command.

Only one attention: you cannot use a sourcetype different than "stash" because, otherwise you have to pay twice the license.

Ciao.

Giuseppe

Re: Indexing the results of a search

ITWhisperer — Fri, 04 Jul 2025 10:56:44 GMT

As @gcusello says, with summary indexing, you have to bear in mind the licensing costs, and it partly depends on the sourcetype (stash vs non-stash), but it also depends on ingest- or workload- -based licensing model - for more details see here

Re: Indexing the results of a search

beano501 — Fri, 04 Jul 2025 12:45:07 GMT

Thanks for the responses, I had only really considered using "summary indexes" as part of the usual summary index commands sitimechart etc.

What I have got working is

index=xxxxx sourcetype="mscs:kql" | eval _raw = SyslogMessage | fields _raw | collect index=main sourcetype=fortigate_event run_in_preview=true

Which achieves what I am after. I appreciate this approach would impact licensing, but it will be low volume.

Thanks again

Re: Indexing the results of a search

gcusello — Fri, 04 Jul 2025 13:09:05 GMT

Hi @beano501 ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉