Installation

How to create a New Source Types in SPLUNK?

SplunkDash
Builder

Hello,

I know we can use SPLUNK GUI to create source types. But how I would create a new source type from CLI or using props.conf file. Any help will be highly appreciated, thank you.

Does this props.conf  is going to create  new source type test:audit  if it doesn't exist

[test:audit]

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]*)<MODTRANSAUDTRL>
TIME_PREFIX= <TIMESTAMP>
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%2N
MAX_TIMESTAMP_LOOKAHEAD=24
TRUNCATE=1000

 

Labels (1)
Tags (1)
0 Karma

PickleRick
Ultra Champion

Sourcetype is just a value. It's just that using this value splunk decides what to do with an event. So in general, anything that you associate with an event on input is a sourcetype. Then you're only telling splunk what to do with event which has this sourcetype by means of props and transforms.

So you don't have to "define" a sourcetype. You can set it to anything in your inputs.conf (as long as it's syntactically correct). Then you just have to set proper props.conf for handling this sourcetype.

 

0 Karma

SplunkDash
Builder

Hello,

Thank you so much for your quick response appreciates it. If this is the case what is there in the GUI to create sourcetype in SPLUNK? Thank you again.

0 Karma

PickleRick
Ultra Champion

The web client treats it a bit differently (although the underlying mechanics stay the same).

If you "create a sourcetype" in the web client, splunk defines some parsing settings for it (like line/event breaking, timestamp recognition, kv mode and such) - the stuff that you typically do in props.conf - so you can then chose this set of settings when creating an input. It's the web client that enforces this "consistency". It's just one of convenience features in splunk web aimed at less experienced users 🙂

In other words, whereas in config files you can do whatever stupidity you want 😉 like in unix, the webui holds your hand like windows 😄

But even the webui does this in a limited way. When you create an input you have to either select a sourcetype from the predefined list or create a new definition. But when you click on an already created input, you can change the sourcetype value to whatever you want regardless of whether you have it "defined" or not (in other words, whether ther are settings for parsing this sourcetype or not).

0 Karma

SplunkDash
Builder

Hello,
I created source type based on your instructions. Thank you so much, appreciated. But getting error message and also getting 1 event instead of 10 events. I am not sure what's the mistake I have done. My props/inputs configuration files and sample source data are given below. Any help will be highly appreciated. Thank you again.

[oswindows_companion:preamble]
SHOULD_LINEMERGE=false
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
category=Structured
TIMESTAMP_FIELDS=Test Date
TIME_FORMAT=%Y-%m-%d %H:%M:%S
HEADER_FIELD_LINE_NUMBER=1

 

[monitor:///opt/splunk/var/log/windows_parsed_reports/companion_preamble/*.csv]
disabled=false
sourcetype=oswindows_companion:preamble
index=oswindows

 

SplunkDash_0-1648527435127.png

 

0 Karma

PickleRick
Ultra Champion

If  this is your literal input file line, you definitely have date formatting inconsistent with the props.conf definition.

0 Karma

SplunkDash
Builder

Hello,

Thank you so much for your feedback. What should I use for this?

TIME_FORMAT=%Y/%m/%d %H:%M:%S

It's not working as well

0 Karma

PickleRick
Ultra Champion

The line from your input data doesn't show the seconds so you can't include ":%S" in the data format because it will never match anything. Also - your with your time format you should have 2022/02/24 whereas you have this illogical american date format 02/24/2022.

Try %m/%d/%Y %H:%M

Should work but can cause problems if you have 12-hour clock without explicitly provided AM/PM.

0 Karma

SplunkDash
Builder

Hello, 

Thank you so much. I tried

%m/%d/%Y %H:%M, still getting error message, but  not sure why getting better result using %/m%/d%/Y %H:%M.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...