How to edit my props and transforms to route data to an index based on index field name ?

[Sot_Sochetra]

Hi all
I'm building a distributed Splunk architecture with:

• 1 Search Head

• 2 Indexers (not in a cluster)

• 1 Heavy Forwarder (HF) to route logs from Universal Forwarders (UFs)

  I want to route logs to different indexers based on the index name, for example:

  • Logs from AD servers should go to indexer01, using index=ad_index

  • Logs from File servers should go to indexer02, using index=fs_index

    Here is my current config on the HF 

    props.conf

    [default]
    TRANSFORMS-routing = route_to_index02

    transforms.conf

    [route_to_index02]
    REGEX = ^fs_index$|^ad_index$
    DEST_KEY = _TCP_ROUTING
    FORMAT = index02

    outputs.conf

    [tcpout]

    [tcpout:index01]
    server = <IP>:9997

    [tcpout:index02]
    server = <IP>:9997

    And here is the example inputs.conf from AD Server

    [WinEventLog://Security]
    disabled = 0
    index = ad_index
    sourcetype = WinEventLog:Security

    [WinEventLog://System]
    disabled = 0
    index = ad_index
    sourcetype = WinEventLog:System

    But right now, everything is going to index02, regardless of the index name.

    So my question is 

    How can I modify props.conf and transforms.conf on the HF so that:

    • ad_index logs go to index01

    • fs_index logs go to index02

      Thank in advance for any help

[richgalloway]

Why do you want to associate indexes with indexers?  Doing this breaks parallelism and will make searching each index take twice as long as it would if the data was evenly distributed across both indexers.

If you're seeing all of your data on indexer02 then the load balancing settings in the HF should be adjusted.  Or, better yet (as @isoutamo suggests), eliminate the HF.

[isoutamo]

Why you want to use HF between indexers and UF? The best practices is send events directly from UF to IDX. If you can do that way just add another outputs.conf to all UF:s and then define used targets in inputs.conf. That's much easier and robust way than using HF between UFs and IDXs.

If you must use HF (e.g. security policy) then you should have at least two HW making routing between UFs and IDXs.

[PickleRick]

It has been a recommended way for a long time to send directly from UFs to indexers but even Splunk acknowledges the practice of having an intermediate layer of HFs (which has its pros and its cons) - https://docs.splunk.com/Documentation/SVA/current/Architectures/Intermediaterouting

[Sot_Sochetra]

Thanks for the advice! The main reason I’m using a Heavy Forwarder is because from what I’ve read, it can parse data before sending it to the indexer. For example, I’m planning to collect logs from some network devices (like firewalls or routers), and I thought sending them through the HF would help with parsing or enriching the data first.

Also, I’m still pretty new to Splunk, so sorry if I’m misunderstanding anything or asking something obvious. 

Best regard

Chetra

[isoutamo]

Ok.

If possible you should participate Data Administration class or something similar. It contains basic stuff how to ingest data into Splunk  https://education.splunk.com/Saba/Web_spf/NA10P2PRD105/app/me/learningeventdetail;spf-url=common%2Fl...

https://www.splunk.com/en_us/pdfs/training/splunk-enterprise-data-administration-course-description....

Depending on your needs you could also parse that data also on indexers instead of use separate HFs for that. It's hard to say which option is better for you as we don't know enough well your needs.

Also you should think again if you need clustered indexers instead of use separate. That will make your environment more robust than what you have now. Of course it needs more disk space etc. but I'm quite sure that it's worth of those additional costs. You will save those when you have 1st issue/crash with your individual indexer...

I think that you should take some courses or learn same information by yourself from net or take some local Splunk company/consultant/architect to help you to define and setup your environment. I'm quite sure that you will save more money that way than starting from scratch w/o enough knowledge and later setup it again.

[Sot_Sochetra]

Thank you for your suggestion i will check it out 

[livehybrid · Accepted solution]

Hi @Sot_Sochetra 

Use two separate transforms that match the metadata field index and send to different TCP groups:

# props.conf (on the Heavy Forwarder)
[default]
TRANSFORMS-routing = route_ad_to_idx01, route_fs_to_idx02

# transforms.conf (on the Heavy Forwarder) 
[route_ad_to_idx01] 
SOURCE_KEY = MetaData:Index 
REGEX = ^ad_index$
DEST_KEY = _TCP_ROUTING
FORMAT = index01

[route_fs_to_idx02] 
SOURCE_KEY = MetaData:Index 
REGEX = ^fs_index$
DEST_KEY = _TCP_ROUTING
FORMAT = index02

Applying this to your HF with the outputs.conf you've already got should route the fs/ad indexes as required.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[Sot_Sochetra]

Thank you for the reply I will test it out tomorrow as I am out of work right now and I will tell you how it goes .