topic Re: How to change index based on MetaData:Source. in Splunk Search

How to change index based on MetaData:Source.

mskrzynski — Mon, 07 Nov 2022 14:46:56 GMT

Hello, can anyone tell me why this configuration isn’t working?

I would like to change index name from main to hue, I’m getting data from db_connect from HF.

I would like to change the index name on main indexer.

transforms.conf

[set_index_hue]

SOURCE_KEY = MetaData:Source

REGEX = ^source::(stream\:Splunk_Postgres)$

DEST_KEY = _MetaData:Index

FORMAT = hue

props.conf

[stream:postgres]

TRANSFORMS-stream-postgres = set_index_hue

Best regards M.

Re: How to change index based on MetaData:Source.

gcusello — Mon, 07 Nov 2022 15:55:36 GMT

Hi @mskrzynski ,

what's a sample of the source to use for the regex?

the syntax is correct, the only possible problem is that the regex isn't correct.

Ciao.

Giuseppe

Re: How to change index based on MetaData:Source.

mskrzynski — Mon, 07 Nov 2022 16:38:06 GMT

Hello, @gcusello I've attached some screenshots.

Best regards M.

Re: How to change index based on MetaData:Source.

gcusello — Mon, 07 Nov 2022 16:46:05 GMT

Hi @mskrzynski,

I'm sorry but I probably didn't express myself well: the regex works on the source field, so I need a couple of examples of this field.

Maybe is it "stream:Splunk_Postgres"?

in this case, if it's fixed, you don't need to have the SOURCE_KEY in transforms.conf and you can use an easier regex:

props.conf:

[stream:postgres] TRANSFORMS-stream-postgres = set_index_hue

transforms.conf

[set_index_hue] REGEX = . DEST_KEY = _MetaData:Index FORMAT = hue

Ciao.

Giuseppe

Re: How to change index based on MetaData:Source.

mskrzynski — Mon, 07 Nov 2022 16:58:02 GMT

Hello again, sample of sources:

stream:Splunk_IP
stream:Splunk_Tcp
stream:Splunk_SSLActivity
stream:Splunk_Udp
stream:Splunk_DNSRequestResponse
stream:Splunk_DNSIntegrity
stream:Splunk_DNSServerQuery
stream:Splunk_DNSServerResponse
stream:Splunk_DNSClientQueryTypes
stream:Splunk_DNSClientErrors
stream:Splunk_Postgres

I would like to catch only stream:Splunk_Postgres

Re: How to change index based on MetaData:Source.

richgalloway — Mon, 07 Nov 2022 16:58:36 GMT

Why not change the inputs.conf setting to specify the proper index?

Re: How to change index based on MetaData:Source.

gcusello — Mon, 07 Nov 2022 17:01:41 GMT

Hi @mskrzynski,

if al this data source must go in the same index, you can specify this index in the input, the method you used is to override the index value.

Ciao.

Giuseppe

Re: How to change index based on MetaData:Source.

mskrzynski — Mon, 07 Nov 2022 17:04:30 GMT

Hello, @gcusello I know, I have to move only stream:postgres to diferent index

Re: How to change index based on MetaData:Source.

gcusello — Mon, 07 Nov 2022 17:09:16 GMT

Hi @mskrzynski,

please try this:

props.conf:

[source::stream:Splunk_postgres] TRANSFORMS-stream-postgres = set_index_hue

transforms.conf

[set_index_hue] REGEX = . DEST_KEY = _MetaData:Index FORMAT = hue

Check the value of the data source for the stanza header in props.conf, it must be

[source::<data_dource>]

Ciao.

Giuseppe

Re: How to change index based on MetaData:Source.

mskrzynski — Mon, 07 Nov 2022 17:31:37 GMT

Hello @gcusello, no luck 😞

props.conf [source::stream:Splunk_postgres] TRANSFORMS-stream-postgres = set_index_hue

transforms.conf [set_index_hue] REGEX = . DEST_KEY = _MetaData:Index FORMAT = hue

Re: How to change index based on MetaData:Source.

gcusello — Tue, 08 Nov 2022 07:50:57 GMT

Hi @mskrzynski,

where do you located these files?

they must be in Indexers or (when present), as in your case, on Heavy Forwarders.

Then are you sure that the source is exactly "stream:Splunk_postgres" with attention to the letter case?

Ciao.

Giuseppe