Can i single file be forwarder to 2 different indexes ??

[rakesh_498115]

Hi Team.

I have a file name called /var/log/messages which I need to send to two different indexes (say index="A" and index="B") on 2 set of indexer servers.

i.e index="A" sourcetype="A" on the indexers set 1 - and index="B" sourcetype="B" on the indexers set 2 -- should give me the same source list i.e /var/log/messages/

please help

thanks.
rakesh.

[somesoni2]

I believe it would involve two steps, data cloning and metadata override. Before we go further, let me remind you that it would cause double license usage for the file.

Assuming you've a data input (inputs.conf) setup to monitor the file with assigned index=indexA and sourcetype=sourcetypeA.

Step 1: Data cloning
You could configure your forwarder to clone the same data to two different indexers. See below link on corresponding outputs.conf configuration

http://docs.splunk.com/Documentation/Forwarder/6.5.0/Forwarder/Configureforwardingwithoutputs.conf#C...

Step 2: Metadata override
On Indexer two have following props.conf and transforms.conf

props.conf

[sourcetypeA]
...other linebreaking/timestamp recognition settings..
TRANSFORMS-override = changeindex,changesourcetype

transforms.conf

[changesourcetype]
FORMAT = sourcetype::sourcetypeB
DEST_KEY = MetaData:Sourcetype
REGEX = .

[changeindex]
DEST_KEY =_MetaData:Index
REGEX = .
 FORMAT = indexB

[vr2312]

That should work if i am not wrong. Cause we do have a lot of issues where there is duplicacy in the logs getting indexed.

I would recommend you to try it out on a test environment to be cautious.