Getting Data In

How to add constant prefix to all indexes and forward it

mzn1979
Explorer

 

Hello everyone

I want to add a constant prefix to all my indexes and then forward them

this is my props.conf

 

props.conf

[default]
TRANSFORMS-index = rename-index

 

 

and here is my transforms.conf

 

transforms.conf

[rename-index]
SOURCE_KEY = _MetaData:Index
REGEX = .
FORMAT = foo-$1
DEST_KEY = _MetaData:Index

 

 

Actually, splunk rename all my indexes to foo-$1 while I want to rename my index to, for example, foo-eventlog, foo-iislog, and so on.

 

any help would be appreciated

Thanks in advance

Labels (3)
1 Solution

mzn1979
Explorer

Hi,

Thank you for your suggestion.

It worked with a little bit difference.

transforms.conf

[rename-index]
SOURCE_KEY = _MetaData:Index
REGEX = (.*)
FORMAT = foo-$1
DEST_KEY = _MetaData:Index

 

View solution in original post

somesoni2
Revered Legend

Try this:

 

transforms.conf

[rename-index]
SOURCE_KEY = _MetaData:Index
REGEX = (.)
FORMAT = foo-$1
DEST_KEY = _MetaData:Index

mzn1979
Explorer

Hi,

Thank you for your suggestion.

It worked with a little bit difference.

transforms.conf

[rename-index]
SOURCE_KEY = _MetaData:Index
REGEX = (.*)
FORMAT = foo-$1
DEST_KEY = _MetaData:Index

 

gcusello
SplunkTrust
SplunkTrust

Hi @mzn1979,

as you said, you have to create a stanza for each group of logs (grouped e.g. for sourcetype?) and in each stanza use a fixed value for the index value:

in props.conf

[default]
TRANSFORMS-index_eventlog = rename-index_eventlog
TRANSFORMS-index_iislog = rename-index_iislog

In transforma.conf

[rename-index_eventlog]
SOURCE_KEY = _MetaData:Index
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = foo-eventlog

[rename-index_iislog]
SOURCE_KEY = _MetaData:Index
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = foo-iislog

 

I have only a question for you: why to do this?

Splunk isn't a database where you use a table for each kind of logs, indexes are siloses usually chosen according to two rules:

  • retention,
  • access grants.

In other words: I usually put in the same index the logs with the same retention period and the same access grants, there's no reason to manage many indexes!

You can identify a data flow by sourcetype not by index .

Ciao.

Giuseppe

mzn1979
Explorer

 

thank you for your help

In fact, I do agree with you but before I join my company someone created more than 30 indexes! and I have to maintain and manage them.

In this case, I must send all logs to another organization. For this purpose I have one HF for sending all logs. Now they want me to add a constant name for each index. (Because they have many sites that send logs to)

I know this way but it's very tiresome!

I want to know is there any way that I can rename all indexes together without creating many stanzas for each index?

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...