Splunk Search

Regex to move three values into sourcetype field with transforms.conf

pbugeja
New Member

Hi,

I am very new with Regex and have been struggling with simple task.

I need to change three values (Health, AuditTrail, Security) in a field called type into individual sourcetypes.

Any assistance would be greatly appreciated.

Thanks, Paul

Tags (1)
0 Karma
1 Solution

cpetterborg
SplunkTrust
SplunkTrust

Just as a note here:

Best practice would be to use a syslog server, like rsyslog or syslog-ng. Then pass the data to the indexers either by using an HTTP Event Collector or a UF or HF. It is harder to loose UDP data that way. Any restart of the Splunk (or syslog service, too) processing will result in a loss of data until the service comes back up. The UF and HF will take many times longer to restore the reception of the data.

If the amount of data coming in is not significant, then perhaps that doesn't matter, but I have one syslog server getting about 800GB/day of syslog data and it is working great (rsyslog -> nginx for load balancing -> indexers with HEC). You can get almost that with a UF alone, but you can't do any kind of parsing of that data to help you out, like separating data to different indexes. If you use an HF, then you will get about a third of that volume. But again, when you restart your Splunk process, you will loose more data than with a syslog server. I use rsyslog, and it's down less than a second, but when we used a UF, it took more than a minute, all the while dropping those UDP packets into the bit bucket.

It is also possible to sourcetype the data at the syslog level, which puts less strain on your indexers.

Something to think about while you are implementing your solution.

View solution in original post

0 Karma

DMohn
Motivator

I still don't quite understand the need for a different source type here. Do these logs have different formats, or do you just want to spilt the log sources by sourcetype? Because there are many other possibilities of splitting/grouping events (think of eventtypes etc)

0 Karma

pbugeja
New Member

Yes, we want to split the log sources by sourcetype.

0 Karma

pbugeja
New Member

With this I simply moved the "type" field into the "sourcetype" field, but I want the values from "type" into "sourcetype".

REGEX = type=(?P[^;]+);
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Good call, somesoni2! I wasn't even thinking index time until I saw what you said, then I looked at the question again to see that it said sourcetypes. Hopefully everyone that reads this one will give answers that are index time answers.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...