The overview: a syslog server acting as a UF receives data via syslog to various local files. Inputs.conf is specified on the server and tags all files with index and sourcetype. This works.
We have a new source that can only send over udp 514, meaning that we have multiple sourcetypes in a single log (we'll say sourceA and sourceB).
Inputs.conf on the UF labels the file with sourcetype and index.
Inputs.conf:
[monitor:///var/log/net-default.log]
disabled = false
sourcetype=sourceA
index=A
All data from UFs are sent to balanced HFs. The HFs have an app with a props.conf and transforms.conf file that are trying to identify the traffic and sourcetype/index as desired.
On the HF:
props.conf
[source::....net-default.log]
TRANSFORMS-force_sourcetype_for_B = force_sourcetype_for_B
TRANSFORMS-force_index_for_B = force_index_for_B[source::....log]
TRANSFORMS-set_host = set_hosttransforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = sourcetype::sourceB[force_index_for_B]
DEST_KEY = _MetaData:Index
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = index::B
The regex correctly picks up the data via "regex _raw". Unfortunately everything is going into indexA.
What we want:
Syslog/HF(file(A/B) -> HF(parse file - tag events as B) -> Indexers (file:indexB)
Now that everyone is thoroughly confused...
Any help is appreciated.
- Tim
@heywiz
Where are you defining sourcetype=sourceB?
In your props.conf, you reference the transform set_host, but it's not in your transforms.conf.
That is a great question. I was thinking I was defining it in the transforms.conf on the HF.
transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = .blah.
FORMAT = sourcetype::sourceB
Incorrect syntax?