topic Re: Props/transforms issue with host extraction and Line breaking in Getting Data In

Props/transforms issue with host extraction and Line breaking

sidhantbhayana — Tue, 29 Sep 2020 17:39:45 GMT

Transforms.conf

[force_host_for_testdata]
DEST_KEY = MetaData:Host
REGEX = server:([^\]+)
FORMAT = host::$1

[force_host_for_testdata_1]
DEST_KEY = MetaData:Host
REGEX = MQ:\s+([^\]+)

FORMAT = host::$1

Props.conf
[test_st]
TZ = GMT
LINE_BREAKER = \d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s+Information:
SHOULD_LINEMERGE = false
TRANSFORMS-force_host_for_testdata = force_host_for_testdata
TRANSFORMS-force_host_for_testdata_1 = force_host_for_testdata_1

This config works on my local machine, but when pushed to heavy forwarders it doesn't work!

Need suggestions as to what is going wrong?

Re: Props/transforms issue with host extraction and Line breaking

mayurr98 — Tue, 16 Jan 2018 07:22:18 GMT

you need to put props.conf and transforms.conf on indexer not on forwarders. and then restart the indexer.

Re: Props/transforms issue with host extraction and Line breaking

harsmarvania57 — Tue, 16 Jan 2018 08:53:00 GMT

Is test_st sourcetype data coming from Heavy Forwarder ? If yes then props.conf and transforms.conf should be on HF and not on IDX.

Additionally can you please provide some sample data (Please mask any sensitive data).

Re: Props/transforms issue with host extraction and Line breaking

sidhantbhayana — Tue, 16 Jan 2018 08:54:31 GMT

this feed is coming from HF so is there still a need to deploy on indexers

Re: Props/transforms issue with host extraction and Line breaking

sidhantbhayana — Tue, 16 Jan 2018 08:57:49 GMT

It is coming from HF.

2017-08-02 02:16:15 Information: Process returned code XXX
ProcessLauncher\PL (Fast)
Command output:
Operating system is 32 bit
Application is running in 32 bit mode

SOME PROCESS SUCCESSFULLY EXECUTED - CODE XXX

Process exited with code XXX
Parameters:
Test.exe /component:Solution /process:"Test Process" /platform:Data /server:test-server\ABCD,12345 /db:TEST
Start: 02 Aug 2017 02:15:31
End: 02 Aug 2017 02:16:15

Re: Props/transforms issue with host extraction and Line breaking

harsmarvania57 — Tue, 16 Jan 2018 09:40:46 GMT

If your server name is test-server then try below config in transforms.conf

[force_host_for_testdata]
DEST_KEY = MetaData:Host
REGEX = server\:([^\\]+)
FORMAT = host::$1

And you have provided only one sample data so I am not sure about another config but give this a try or provide sample data for regex MQ:\s+([^\]+)

[force_host_for_testdata_1]
DEST_KEY = MetaData:Host
REGEX = MQ\:\s+([^\\]+)
FORMAT = host::$1

Re: Props/transforms issue with host extraction and Line breaking

sidhantbhayana — Wed, 31 Jan 2018 06:39:08 GMT

Found it working, thanks