Getting Data In

Have forwarder duplicating data to 2 indexes.

mataharry
Communicator

For some inputs on a forwarder, I want to send the same data to the same indexer, but duplicate it in 2 indexes (they have different permissions/retention)

This is sending to the main index, I want to main and public.
[monitor:///var/log/feed]
disabled = false
followTail = 0
sourcetype = one
index=main

Tags (2)
1 Solution

yannK
Splunk Employee
Splunk Employee

2 methods :
A - Use another instance of splunk monitoring the same file and specifying a different index
(by example on windows you can have 1 UF and 1 LMF/HF/indexer, on linux, as many instances as you want)

B - use a symlink to the files/folders and have a secondary monitor on the symlink (with a different index destination)
see screenshot for the result

create a symlink
example :
ln -s feed symlink

and define 2 inputs one on the original, the other on the symlink

[monitor:///var/log/feed] 
disabled = false 
followTail = 0 
sourcetype = one 
index=main 

[monitor:///var/log/symlink] 
disabled = false 
followTail = 0 
sourcetype = one
 # or any other sourcetype 
crcSalt=< SOURCE >
 # required to force splunk to differentiate files based on the path/filename, write SOURCE in caps (the html formatting may hide it), and remoce the space in the tag.
index=public
 # the index of your choice 
followSymlink=true 
 # to make sure that the symlink will be followed. 

View solution in original post

yannK
Splunk Employee
Splunk Employee

2 methods :
A - Use another instance of splunk monitoring the same file and specifying a different index
(by example on windows you can have 1 UF and 1 LMF/HF/indexer, on linux, as many instances as you want)

B - use a symlink to the files/folders and have a secondary monitor on the symlink (with a different index destination)
see screenshot for the result

create a symlink
example :
ln -s feed symlink

and define 2 inputs one on the original, the other on the symlink

[monitor:///var/log/feed] 
disabled = false 
followTail = 0 
sourcetype = one 
index=main 

[monitor:///var/log/symlink] 
disabled = false 
followTail = 0 
sourcetype = one
 # or any other sourcetype 
crcSalt=< SOURCE >
 # required to force splunk to differentiate files based on the path/filename, write SOURCE in caps (the html formatting may hide it), and remoce the space in the tag.
index=public
 # the index of your choice 
followSymlink=true 
 # to make sure that the symlink will be followed. 

mataharry
Communicator

I tested with multiple groups in outputs.conf but I cannot change the index and they all go to the same index.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk App Dev Quarterly Roundup: AI, Agents, and Innovation!

Another quarter, another wave of innovation. From complex integrations to pushing the limits ...

Federated Search for Dynamic Data Self Storage Is Now Generally Available on Splunk ...

 Splunk is excited to announce the General Availability of Federated Search for Dynamic Data Self Storage ...

Index This | What has many keys but can’t unlock a door?

July 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...