Splunk Search

Filtering on HF via regex not working

twhisnant
New Member

The overview: a syslog server acting as a UF receives data via syslog to various local files. Inputs.conf is specified on the server and tags all files with index and sourcetype. This works.

We have a new source that can only send over udp 514, meaning that we have multiple sourcetypes in a single log (we'll say sourceA and sourceB).

Inputs.conf on the UF labels the file with sourcetype and index.

Inputs.conf:
[monitor:///var/log/net-default.log]
disabled = false
sourcetype=sourceA
index=A

All data from UFs are sent to balanced HFs. The HFs have an app with a props.conf and transforms.conf file that are trying to identify the traffic and sourcetype/index as desired.

On the HF:

props.conf
[source::....net-default.log]
TRANSFORMS-force_sourcetype_for_B = force_sourcetype_for_B
TRANSFORMS-force_index_for_B = force_index_for_B

[source::....log]
TRANSFORMS-set_host = set_host

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = sourcetype::sourceB

[force_index_for_B]
DEST_KEY = _MetaData:Index
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = index::B

The regex correctly picks up the data via "regex _raw". Unfortunately everything is going into indexA.

What we want:
Syslog/HF(file(A/B) -> HF(parse file - tag events as B) -> Indexers (file:indexB)

Now that everyone is thoroughly confused...

Any help is appreciated.
- Tim
@heywiz

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

Where are you defining sourcetype=sourceB?

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

In your props.conf, you reference the transform set_host, but it's not in your transforms.conf.

0 Karma

twhisnant
New Member

That is a great question. I was thinking I was defining it in the transforms.conf on the HF.

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = .blah.
FORMAT = sourcetype::sourceB

Incorrect syntax?

0 Karma
Get Updates on the Splunk Community!

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...

Thank You for Celebrating CX Day with Splunk!

Yesterday the entire team at Splunk + Cisco joined the global celebration of CX Day - celebrating our ...