Getting Data In

Have forwarder duplicating data to 2 indexes.

mataharry
Communicator

For some inputs on a forwarder, I want to send the same data to the same indexer, but duplicate it in 2 indexes (they have different permissions/retention)

This is sending to the main index, I want to main and public.
[monitor:///var/log/feed]
disabled = false
followTail = 0
sourcetype = one
index=main

Tags (2)
1 Solution

yannK
Splunk Employee
Splunk Employee

2 methods :
A - Use another instance of splunk monitoring the same file and specifying a different index
(by example on windows you can have 1 UF and 1 LMF/HF/indexer, on linux, as many instances as you want)

B - use a symlink to the files/folders and have a secondary monitor on the symlink (with a different index destination)
see screenshot for the result

create a symlink
example :
ln -s feed symlink

and define 2 inputs one on the original, the other on the symlink

[monitor:///var/log/feed] 
disabled = false 
followTail = 0 
sourcetype = one 
index=main 

[monitor:///var/log/symlink] 
disabled = false 
followTail = 0 
sourcetype = one
 # or any other sourcetype 
crcSalt=< SOURCE >
 # required to force splunk to differentiate files based on the path/filename, write SOURCE in caps (the html formatting may hide it), and remoce the space in the tag.
index=public
 # the index of your choice 
followSymlink=true 
 # to make sure that the symlink will be followed. 

View solution in original post

yannK
Splunk Employee
Splunk Employee

2 methods :
A - Use another instance of splunk monitoring the same file and specifying a different index
(by example on windows you can have 1 UF and 1 LMF/HF/indexer, on linux, as many instances as you want)

B - use a symlink to the files/folders and have a secondary monitor on the symlink (with a different index destination)
see screenshot for the result

create a symlink
example :
ln -s feed symlink

and define 2 inputs one on the original, the other on the symlink

[monitor:///var/log/feed] 
disabled = false 
followTail = 0 
sourcetype = one 
index=main 

[monitor:///var/log/symlink] 
disabled = false 
followTail = 0 
sourcetype = one
 # or any other sourcetype 
crcSalt=< SOURCE >
 # required to force splunk to differentiate files based on the path/filename, write SOURCE in caps (the html formatting may hide it), and remoce the space in the tag.
index=public
 # the index of your choice 
followSymlink=true 
 # to make sure that the symlink will be followed. 

mataharry
Communicator

I tested with multiple groups in outputs.conf but I cannot change the index and they all go to the same index.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...