topic Re: How to modify host and sourcetype at the same time in Getting Data In

How to modify host and sourcetype at the same time

legaldan — Fri, 08 Apr 2022 03:33:39 GMT

When using HF to collect logs on the cloud，

Because the add-on used cannot set host，

So the host of the data is the name of HF，

but it needs to reflect that the data comes from an impassable environment，

And the same data type uses the same sourcetype.

At present, the way I use is First, use different sourcetypes to access data

At this time, they have the same host （HF name）

then, I use props and transforms to modify their host

and Change their sourcetype to the same one

the question is modify host and change sourcetype

Only one will take effect.

Is there a way to modify the host first and then modify the sourcetype？

Or something better ？

Re: How to modify host and sourcetype at the same time

VatsalJagani — Fri, 08 Apr 2022 04:25:00 GMT

@legaldan

All the TRANSFORMS will get executed after the host/source/sourcetype stanzas are decided. And the process happens only once.

So for example, if you want to change the host and based on the new hostname you want to modify sourcetype then it's not possible.

But you can write the TRANSFORMS on old sourcetype/source/host and write your REGEX accordingly.

Kindly provide sample examples of source/sourcetype/host values if you want help with the configuration.

----
I hope this helps!!

Re: How to modify host and sourcetype at the same time

legaldan — Fri, 08 Apr 2022 05:29:07 GMT

Thank you for your reply！

examples

MetaData：

index=test

host=hf_splunk

sourcetype=tomcat_prod

props.conf

[tomcat_prod]
TRANSFORMS =prod_changehost
TRANSFORMS =tomcat_changesourcetype

transforms.conf

[prod_changehost]
REGEX = .*
FORMAT = host::hostname_prod
DEST_KEY = MetaData:Host

[tomcat_changesourcetype]
REGEX = .*
FORMAT = sourcetype::tomcat:access
DEST_KEY = MetaData:Sourcetype

Expected results：

index=test

host=hf_splunk → host=hostname_prod

sourcetype=tomcat_prod → sourcetype=tomcat_access

Re: How to modify host and sourcetype at the same time

VatsalJagani — Fri, 08 Apr 2022 05:35:46 GMT

@legaldan - Use class names with TRANSFORMS to specify they are different attributes.

props.conf

[tomcat_prod] TRANSFORMS-for_host_change = prod_changehost TRANSFORMS-for_sourcetype_change = tomcat_changesourcetype

transforms.conf remains as you have.

Re: How to modify host and sourcetype at the same time

legaldan — Fri, 08 Apr 2022 06:27:19 GMT

I know Splunk executes code in alphabetical order （A-Z）

and I named the code to modify the host as the letter at the top of the sort

Modify the code of sourcetype and name it as the letter at the end of the sorting

It seems to be working now

I'll do more tests. Thank you for your support！

Re: How to modify host and sourcetype at the same time

legaldan — Fri, 08 Apr 2022 06:47:03 GMT

My colleague gave me a better way to execute the code from left to right

transforms.conf

TRANSFORMS = A,B

In this way, execute A first and then B

Re: How to modify host and sourcetype at the same time

VatsalJagani — Fri, 08 Apr 2022 08:01:55 GMT

That is true it executes in alphabetical order.

But in your case order doesn't matter.

Order only matters if you are extracting some field that you want to later override by some other TRANSFORMS.

Here you have two different transforms one updating host and the other updating sourcetype, so regardless of which executes first or second, you will have both updated.

-----------
I hope I explained to you when orders will be useful. And kindly accept the answer if it resolves your issue.