Getting Data In

How to ingest logs from a single file and split them into 2 source types or indexes?

daniel_augustyn
Contributor

I've been trying to ingest logs from a single log file into 2 source types. For example, looking at the 'messages' file in Linux, I want to split the logs from this file into different source types or indexes.

Sourcetype 1:

Jul 10 09:20:01 systemd: Started Session 27535 of user root.

Sourcetype 2:

Jul 10 09:13:26 rsyslogd: [origin software="rsyslogd" swVersion="X.X.X" x-pid="18406" x-info="http://www.rsyslog.com"] rsyslogd

How should I approach this?

0 Karma
1 Solution

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

View solution in original post

ArchieCrozier
Path Finder

@sundareshr Appreciate the response.  Was looking for this one.

0 Karma

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...