topic Re: How to create multiple source types from a single log file? in Getting Data In

How to create multiple source types from a single log file?

acsanders — Mon, 17 Apr 2017 14:44:33 GMT

I am ingesting 1 file that has multiple server IP addresses. I need to source type each server based on the IP address. I have tried using the props.conf and transforms.conf with no luck. Any help would be much appreciated.

Re: How to create multiple source types from a single log file?

somesoni2 — Mon, 17 Apr 2017 16:12:21 GMT

We would need some sample events and your current props.conf/transforms.conf. Meanwhile, check if something like this works for you.

#Inputs.conf on forwarder
[monitor://<<path of file>]
index = ..
sourcetype = some_default_sourcetype

#props.conf on Indexers/Heavy Forwarder
[some_default_sourcetype]
...event parsing stuffs..
TRANSFORMS-overridest = change_st_by_IP1,change_st_by_IP2,change_st_by_IP3,....

#transforms.conf on Indexers/Heavy Forwarder. Replace IPs with your exact values)
[change_st_by_IP1]
REGEX = (10\.11\.12\.13)
FORMAT = sourcetype::yourNewSourceType1
DEST_KEY = MetaData:Sourcetype

[change_st_by_IP2]
REGEX = (20\.21\.22\.23)
FORMAT = sourcetype::yourNewSourceType2
DEST_KEY = MetaData:Sourcetype

..
similar stanza for other IPs...

Re: How to create multiple source types from a single log file?

acsanders — Mon, 17 Apr 2017 19:25:14 GMT

That did exactly what I was trying to accomplish. Thanks so much for the fast response.

Re: How to create multiple source types from a single log file?

acsanders — Tue, 29 Sep 2020 13:41:42 GMT

I have an additional question. I need to do the same thing with a string that I am doing with an IP address. Whats the correct way to do this. How do I set up the REGEX for a string?

transforms.comf
[change_st_by_IP9]
REGEX = Plinapp748

FORMAT = sourcetype::McAfee_ePO
DEST_KEY = MetaData:Sourcetype

Re: How to create multiple source types from a single log file?

somesoni2 — Tue, 18 Apr 2017 14:49:02 GMT

It's the same way as IP. IP has a special character dot so I had to escape it. If your string just has alphanumeric values, just specify them as it is in REGEX.

Re: How to create multiple source types from a single log file?

acsanders — Tue, 18 Apr 2017 16:30:30 GMT

Thanks so much for the help. Worked like a charm.

Re: How to create multiple source types from a single log file?

vanheer — Fri, 19 Aug 2022 18:29:55 GMT

Hi,

I have a question here, can we use different index for each sourcetype in these conf files?