Getting Data In

How to ingest logs from a single file and split them into 2 source types or indexes?

daniel_augustyn
Contributor

I've been trying to ingest logs from a single log file into 2 source types. For example, looking at the 'messages' file in Linux, I want to split the logs from this file into different source types or indexes.

Sourcetype 1:

Jul 10 09:20:01 systemd: Started Session 27535 of user root.

Sourcetype 2:

Jul 10 09:13:26 rsyslogd: [origin software="rsyslogd" swVersion="X.X.X" x-pid="18406" x-info="http://www.rsyslog.com"] rsyslogd

How should I approach this?

0 Karma
1 Solution

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

View solution in original post

ArchieCrozier
Path Finder

@sundareshr Appreciate the response.  Was looking for this one.

0 Karma

sundareshr
Legend

Try this

*Props*

[source::...regex_to_match_filename] 
TRANSFORMS-st-sys = force-sourcetype-systemd
TRANSFORMS-st-rsys = force-sourcetype-rsyslogd

*Transforms*

[force-sourcetype-systemd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (systemd)
FORMAT = sourcetype::$1 
WRITE_META = true

[force-sourcetype-rsyslogd]
DEST_KEY = MetaData::Sourcetype 
REGEX = (rsyslogd)
FORMAT = sourcetype::$1 
WRITE_META = true

https://docs.splunk.com/Documentation/Splunk/6.4.1/Admin/Transformsconf

Get Updates on the Splunk Community!

Set Up More Secure Configurations in Splunk Enterprise With Config Assist

This blog post is part 3 of 4 of a series on Splunk Assist. Click the links below to see the other ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...