Getting Data In

How to ingest logs from a single file and split them into 2 source types or indexes?

daniel_augustyn
Contributor

I've been trying to ingest logs from a single log file into 2 source types. For example, looking at the 'messages' file in Linux, I want to split the logs from this file into different source types or indexes.

Sourcetype 1:

Jul 10 09:20:01 systemd: Started Session 27535 of user root.

Sourcetype 2:

Jul 10 09:13:26 rsyslogd: [origin software="rsyslogd" swVersion="X.X.X" x-pid="18406" x-info="http://www.rsyslog.com"] rsyslogd

How should I approach this?

0 Karma
1 Solution

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

View solution in original post

ArchieCrozier
Path Finder

@sundareshr Appreciate the response.  Was looking for this one.

0 Karma

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...