index based on first three characters of hostname

[dglass0215]

Hello,

I am trying to implement setting a specific index based on part of the hostname. For ALL of my data that I index I would like the index to be the first three characters of the hostname. is it possible to make this global and not on a source/sourcetype basis?

Any help is appreciated!
Davis

[niketn]

@dglass0215 do you really want this to be global? or at app level? or may be for specific patterns of hosts, sources and sourcetypes? Overriding all indexes globally seems too intrusive.

[dglass0215]

Thanks for your reply! Yes I agree with you that it is very intrusive. I just thought that this would be a quick way for me to help get data to the right index and then fine tune it from there. Basically we have about 10 different locations with possibly adding up to 100 more locations and every host at each location begins with a 3 character acronym of the location. So this would be a quick and dirty way to make sure the data is getting to the proper index. I think it will eventually need to be expanded on.

[jnudell_2 · Accepted solution]

Hi @dglass0215 ,

You could do this by modifying props.conf & transforms.conf on your heavy forwarders/indexers:

props.conf

[default]

TRANSFORMS-set_index = set_index

transforms.conf

[set_index]

SOURCE_KEY = MetaData:Host

DEST_KEY = _MetaData:Index

REGEX = ^(?:host::)?(\w\w\w)

FORMAT = $1

This will take every event, look at the first 3 characters of the host value, and set those 3 characters as the target index for ingestion. You would have to make sure those indexes exist, or the events will be dropped because the index does not exist.

Hope this helps.

[dglass0215]

Hi @jnudell_2, According to all the documentation (And my training courses) this looks like it should work, however it is not working for me. I have a bunch of other props/transforms that work correctly, I just added the default stanza to the bottom. That shouldn't be an issue right? How does one go about debugging props/transforms? I did a btool check and that didn't return anything.

[jnudell_2]

You can verify that it's set properly with the following btool command:
/opt/splunk/bin/splunk btool props list
You should notice that for EVERY stanza there will be a line that says:
TRANSFORMS-set_index = set_index
That shows that the props.conf part of it is setup correctly.

Found out the issue. Host is reported as host::<hostname> not <hostname>. I have updated the answer above, and validated it works on a standalone instance I'm working on.

[dglass0215]

Thanks! Works great!