Splunk Search

Custom index for forwarded data

eholz1
Contributor

Hello all,
I have a working universal forwarder that happily sends data to my Enterprise indexer.
The data shows up under the forwarder's hostname on the indexer.
I would like to have a custom index for the data that comes from the Universal forwarder (my_fwd_server.net)
I can search the data by entering the hostname in the search field: host="my_fwd_server.net

I would like to be able to create a search string like: host="my_fwd_server.net" index="fwd_index", etc.
Is this possible or is this unnecessary for the data coming from the forwarder since I know the hostname?

Thanks for outstanding product,

eholz1

0 Karma
1 Solution

DavidHourani
Super Champion

Hi @eholz,

Yes by all means split your data into multiple indexes. Also, make sure you have a different sourcetype for the various types of data you're sending.

You want to keep your data well separated for better scalability, easier access control management and performance. If you mix everything up into one index then you will just end up slowing down all your searches because Splunk would have to read everything to find the relevant information you're looking for in your queries.

Before defining a new target index on your forwarder in inputs.conf, make sure you create it first on your indexer as follows :
https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Setupmultipleindexes#Create_events_index...

Let me know if you need more help.

Cheers,
David

View solution in original post

0 Karma

eholz1
Contributor

Hello David,

Correct, and I did just as you suggest, and it even works!!

Thanks Again,

Eholz1

0 Karma

DavidHourani
Super Champion

Hi @eholz,

Yes by all means split your data into multiple indexes. Also, make sure you have a different sourcetype for the various types of data you're sending.

You want to keep your data well separated for better scalability, easier access control management and performance. If you mix everything up into one index then you will just end up slowing down all your searches because Splunk would have to read everything to find the relevant information you're looking for in your queries.

Before defining a new target index on your forwarder in inputs.conf, make sure you create it first on your indexer as follows :
https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Setupmultipleindexes#Create_events_index...

Let me know if you need more help.

Cheers,
David

0 Karma

somesoni2
SplunkTrust
SplunkTrust

You want to create a new index for each forwarder server OR just a new index which will hold data from all forwarders? Which index your data is going to right now?

Read this to understand why people generally create multiple indexes. If it's just for having a separate search for each forwarder, then I don't think you need a new index for each forwarder. The field host is a metadata and can uniquely identify data coming from that host (or data that has host field set your forwarder's name)
https://docs.splunk.com/Documentation/Splunk/7.3.0/Indexer/Setupmultipleindexes#Why_have_multiple_in...

0 Karma

eholz1
Contributor

Hello Again,
just modified the inputs.conf file on the forwarder to see if it will use the
custom index I created on the indexer.
thanks

0 Karma

eholz1
Contributor

Hello somesoni2,
Thank you for the reply - much appreciated.

I have only one forwarder. currently the data is going into the "main" index. I just thought that it would be a
good idea or practice to put the data from the forwarder in a different index.

As always, I am open to suggestions

eholz

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...