Splunk Search

Filtering on HF via regex not working

twhisnant
New Member

The overview: a syslog server acting as a UF receives data via syslog to various local files. Inputs.conf is specified on the server and tags all files with index and sourcetype. This works.

We have a new source that can only send over udp 514, meaning that we have multiple sourcetypes in a single log (we'll say sourceA and sourceB).

Inputs.conf on the UF labels the file with sourcetype and index.

Inputs.conf:
[monitor:///var/log/net-default.log]
disabled = false
sourcetype=sourceA
index=A

All data from UFs are sent to balanced HFs. The HFs have an app with a props.conf and transforms.conf file that are trying to identify the traffic and sourcetype/index as desired.

On the HF:

props.conf
[source::....net-default.log]
TRANSFORMS-force_sourcetype_for_B = force_sourcetype_for_B
TRANSFORMS-force_index_for_B = force_index_for_B

[source::....log]
TRANSFORMS-set_host = set_host

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = sourcetype::sourceB

[force_index_for_B]
DEST_KEY = _MetaData:Index
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = index::B

The regex correctly picks up the data via "regex _raw". Unfortunately everything is going into indexA.

What we want:
Syslog/HF(file(A/B) -> HF(parse file - tag events as B) -> Indexers (file:indexB)

Now that everyone is thoroughly confused...

Any help is appreciated.
- Tim
@heywiz

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

Where are you defining sourcetype=sourceB?

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

In your props.conf, you reference the transform set_host, but it's not in your transforms.conf.

0 Karma

twhisnant
New Member

That is a great question. I was thinking I was defining it in the transforms.conf on the HF.

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = .blah.
FORMAT = sourcetype::sourceB

Incorrect syntax?

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...