topic Re: How can I purposely duplicate events to multiple indexes? in Getting Data In

How can I purposely duplicate events to multiple indexes?

juniormint — Tue, 19 Nov 2013 22:07:09 GMT

I realize this will incur twice the license usage...but lets ignore that fact for a moment.

How can I get an event into two indexes?

My events are coming from a universal forwarder with a default index of "ABC" and source="XYZ".

On the indexer I have indexes "ABC" and "DEF".

What I would like is to send all source XYZ events to the ABC index and a subset to DEF.

Re: How can I purposely duplicate events to multiple indexes?

ShaneNewman — Wed, 20 Nov 2013 02:23:22 GMT

I have done this with the index and forward option, although I will say that I don't do it anymore. My preferred method now, is to create a summary index on a 3 minute delay to send the filtered events to the other index. This means that we don't have to worry about the license issue.

Is there a reason why this is not acceptable for you? You can stream the _raw to your summary...

Re: How can I purposely duplicate events to multiple indexes?

juniormint — Wed, 20 Nov 2013 03:13:03 GMT

I'm interested in both approaches..can you point me at relevant doc pages? or provide an example?

Re: How can I purposely duplicate events to multiple indexes?

ShaneNewman — Mon, 28 Sep 2020 15:19:19 GMT

For the summary index option:

[_some_search_name]
action.summary_index = 1
action.summary_index._name = DEF
action.summary_index.stat_tag = #Treat this as sourcetype#
cron_schedule = */1 * * * *
dispatch.earliest_time = -3m@m
dispatch.latest_time = -2m@m
enableSched = 1
realtime_schedule = 0
search = index=ABC sourcetype=your_sourcetype (add an additional search field to filter the results you want to send)

You can preform a sistats at the end if you want, although it is not needed if you just want the raw events.

Re: How can I purposely duplicate events to multiple indexes?

ShaneNewman — Wed, 20 Nov 2013 03:23:56 GMT

This would go into a savedsearches.conf file.

Re: How can I purposely duplicate events to multiple indexes?

juniormint — Wed, 20 Nov 2013 14:36:25 GMT

That is great and would work for my stated problem...I think. What is the index and forward option?

Re: How can I purposely duplicate events to multiple indexes?

juniormint — Fri, 22 Nov 2013 20:33:15 GMT

http://docs.splunk.com/Documentation/Splunk/6.0/Forwarding/Routeandfilterdatad#Route_inputs_to_specific_indexers_based_on_the_data.27s_input

I think the index and forward option is what is shown on this page

Re: How can I purposely duplicate events to multiple indexes?

ShaneNewman — Fri, 22 Nov 2013 20:43:16 GMT

What I have show above mimics index and forward, it is just less cumbersome and makes it easier to change. Changes using this method also do not require a restart of the instance either, just a reload of the EAI Objects.