Hi @livehybrid  Thank you so much for your answer and for sharing those resources  I really appreciate it. Regarding where to put the app, I believe hosting it on a HF will be the best approach, and I'll make sure that the HF have an access on the AWS hosted app. Thanks again for your help   Nicolas ... View more

OK now I get it, so I need to define one more stanza in my transforms to overwrite the _TCP_ROUTING For example [mydevice_overwrite_tcprouting] REGEX = . DEST_KEY = _TCP_ROUTING FORMAT = an_empty_output Props.conf [my_device_clone] TRANSFORMS-deletetcprouting = mydevice_overwrite_tcprouting   Something like this ? is it possible to define an empty values in an outputs.conf ? Thanks Nicolas       ... View more

Ok so if I understand, when I clone a sourcetype, he will clone it's destination too ? Not sure to understand, I have some other log sources that I clone, and forward to a secondary splunk with the same clone methods. Cloning the sourcetype to sourcetype:anonymized In the transforms I applied on the cloned sourcetype some regex for anonymization And this sourcetype is routed via _TCP_ROUTING to an output that is a heavy forwarder that route to the secondary splunk. For example  Transforms.conf [firewall_log-clone] CLONE_SOURCETYPE = firewall_log:clone REGEX = .* DEST_KEY = _TCP_ROUTING FORMAT = output_externalhf Props.conf [firewall_log] TRANSFORMS-clone = firewall_log-clone This is working, logs are properly sent to an HF that will forward those logs to a secondary splunk But it's not syslog log source so maybe this is the difference ? Thanks for the help ... View more

Hi @livehybrid  Yes in my meening, setting a _TCP_ROUTING on the input will forward the logs on the local indexers. And, at the sametime, sourcetype is cloned to be forwarded to the second splunk. Maybe I didn't understand something 😕 ... View more

Hi Splunk Community, I have a new issue concerning this config, some particular behaviour that I don't understand Here is my configuration #classic input on TCP, this is syslog logs [tcp://22000] sourcetype = mydevice:sourcetype index = local_index _TCP_ROUTING = local_indexers:9997 # Idea is to clone the sourcetype, but not logs containing LAN1 and LAN2 logs, it's not necessary for the second splunk [mydevice-clone] CLONE_SOURCETYPE = mydevice:clone REGEX = ^((?!LAN1|LAN2).)*$ DEST_KEY = _SYSLOG_ROUTING FORMAT = sending_to_second_splunk # on the props I apply the configuration made on the transforms [mydevice:sourcetype] TRANSFORMS-clone = mydevice-clone #IP of the HF that will send data to second splunk [syslog:sending_to_second_splunk] server = 10.10.10.10:port type = tcp   Issue Encountered This configuration works partially: Data is properly indexed to the second Splunk, without LAN1 and LAN2 data. Data containing LAN1 and LAN2 is indexed on the local indexer. However, the sourcetype mydevice:clone is also indexed on my local indexer, resulting in some data being indexed twice with two different sourcetypes. I don't understand why this is happening and I am seeking help to resolve this issue, I have the feeling that I miss something Thanks, Nicolas ... View more

Thanks all for the help, I will try a regex that match both. I learn a lot with you guys thanks !!!! ... View more

Hello Had an additional questions, if I have to extract datas from two fields For example, I will NOT send to distant splunk, all the datas that in the dest_zone and src_zone contains DMZ-NETWORK and GUEST-NETWORK My config is something like that TRANSFORMS.CONF [clone_only_dmz-network] REGEX = ^((?!dmz-network).)*$ CLONE_SOURCETYPE = cloned:firewall_clone DEST_KEY = _SYSLOG_ROUTING #it's syslog logs FORMAT = distant_splunk PROPS.CONF [firewall_sourcetype] TRANSFORMS-clone = clone_only_dmz-network OUTPUTS.CONF [syslog:distant_splunk] server = ip of the distant HF This is actually working, I tested it But, if I want to not send mutliple fileds, how can I achieve that ? ... View more

Hi @livehybrid , @isoutamo  Thanks a lot for your help! I’ll try the configuration with CLONE_SOURCETYPE and will come back here to let you know if it works for me. 🙂 Thanks again for your support!   ... View more

Hello @livehybrid  Thanks for your time OK, I understand now. I see what I was missing. Strangely, what I had done was working, and I was perplexed about that. 😕 I will test with the configuration you provided; it makes more sense. But I have a quick question: if the logs need to be anonymized before they are sent to the distant_HF, will putting the two outputs in the _TCP_ROUTING in the inputs.conf work? Many thanks for you clear answer !!!   ... View more

Hello kiran, yes this is a typo . ... View more

Hello @livehybrid , yes, data are still getting in the original index and they contain "TheAppResourceGroupName"   ... View more

Hello, I just checked, and the Microsoft Cloud Services manage checkpoints locally on heavy forwarders. However, there is a configuration in the app that allows you to store checkpoints in a container within an Azure storage account. This way, when you need to start log collection on another heavy forwarder, it could facilitate the process. Will configure that and test, I let you know ! Thanks Nico ... View more

Hello @PickleRick  Thats an interesting topic, I will dig more information about it. I let you know here If I found something interesting Thanks ! Nicolas ... View more

Hello @gcusello  Thanks for the answer.  Ok I understand, I will install the app on both HF and just activate it on one. When you say yu have to manage checkpoints between HFs. How is that possible in Splunk ? Assuming that logs are stored on the source for 2 weeks in case of an outage, when I activate log collection on the second HF, it will start collecting logs from the day it is activated, and it won't be aware of the logs already ingested into Splunk? ... View more

The storage will on a DELL EMC storage, and considering the splunk recomandations and the SAN caracteristics, it will work smoothly, on the paper. I will test with few index and check Thanks a lot for your help ! ... View more

Ok I understand. And considering having two different NFS volumes on a SAN, one volume for each indexer, but the mounting point on the OS will have the same name for both indexers Can this solution work ? ... View more

Hi marnall, thanks for the answer. So there is no chance to put cold datas on a NFS network storage without implementing smartstore ?   ... View more

Ok, to be honest I had to check more on the config to properly clone and forward my datas, the behaviour of the conf it's strange. But thanks a lot for your help, I appreciate ! ... View more

Ok I understand what you say. But sorry I forgot to mentionned that I have a TCPOUT default on my conf: [tcpout] defaultGroup = my_indexers forceTimebasedAutoLB = true forwardedindex.filter.disable = true [tcpout:my_indexers] server = indexer1:9997, indexer2:9997 So if I'am correct, the inputs.conf : [inputs.conf] [udp://22210] index = my_logs_indexer sourcetype = log_sourcetype disabled = false Redirect the logs to the default outputs, because no outputs is specified . Correct me if I'm wrong, and sorry to forgot this config at the first question ... View more

Hi PickleRick, thanks for your response and time. The cloned logs are routing only to one instance, specified into the outputs.conf The "original" logs, not the cloned one are directed to my local indexers, just the cloned sourcetype is directed with another heavy forwarder specified in the outputs.conf placed in the same app as the props and transforms. Not sure if i'm clear 😕 ... View more