Splunk Search

Filtering on HF via regex not working

twhisnant
New Member

The overview: a syslog server acting as a UF receives data via syslog to various local files. Inputs.conf is specified on the server and tags all files with index and sourcetype. This works.

We have a new source that can only send over udp 514, meaning that we have multiple sourcetypes in a single log (we'll say sourceA and sourceB).

Inputs.conf on the UF labels the file with sourcetype and index.

Inputs.conf:
[monitor:///var/log/net-default.log]
disabled = false
sourcetype=sourceA
index=A

All data from UFs are sent to balanced HFs. The HFs have an app with a props.conf and transforms.conf file that are trying to identify the traffic and sourcetype/index as desired.

On the HF:

props.conf
[source::....net-default.log]
TRANSFORMS-force_sourcetype_for_B = force_sourcetype_for_B
TRANSFORMS-force_index_for_B = force_index_for_B

[source::....log]
TRANSFORMS-set_host = set_host

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = sourcetype::sourceB

[force_index_for_B]
DEST_KEY = _MetaData:Index
REGEX = (?ms)^\w{3}\s+\d+?\s\d{2}\:\d{2}\:\d{2}.*somestaticstring:
FORMAT = index::B

The regex correctly picks up the data via "regex _raw". Unfortunately everything is going into indexA.

What we want:
Syslog/HF(file(A/B) -> HF(parse file - tag events as B) -> Indexers (file:indexB)

Now that everyone is thoroughly confused...

Any help is appreciated.
- Tim
@heywiz

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

Where are you defining sourcetype=sourceB?

0 Karma

mwhite_splunk
Splunk Employee
Splunk Employee

In your props.conf, you reference the transform set_host, but it's not in your transforms.conf.

0 Karma

twhisnant
New Member

That is a great question. I was thinking I was defining it in the transforms.conf on the HF.

transforms.conf
[force_sourcetype_for_B]
DEST_KEY = MetaData:Sourcetype
REGEX = .blah.
FORMAT = sourcetype::sourceB

Incorrect syntax?

0 Karma
Get Updates on the Splunk Community!

Community Content Calendar, November Edition

Welcome to the November edition of our Community Spotlight! Each month, we dive into the Splunk Community to ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...