topic How to create a New Source Types in SPLUNK? in Installation

How to create a New Source Types in SPLUNK?

SplunkDash — Tue, 29 Mar 2022 13:06:41 GMT

Hello,

I know we can use SPLUNK GUI to create source types. But how I would create a new source type from CLI or using props.conf file. Any help will be highly appreciated, thank you.

Does this props.conf is going to create new source type test:audit if it doesn't exist

[test:audit]

SHOULD_LINEMERGE=false LINE_BREAKER=([\r\n]*)<MODTRANSAUDTRL> TIME_PREFIX= <TIMESTAMP> TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%2N MAX_TIMESTAMP_LOOKAHEAD=24 TRUNCATE=1000

Re: Create a New Source Types in SPLUNK

PickleRick — Mon, 28 Mar 2022 19:04:25 GMT

Sourcetype is just a value. It's just that using this value splunk decides what to do with an event. So in general, anything that you associate with an event on input is a sourcetype. Then you're only telling splunk what to do with event which has this sourcetype by means of props and transforms.

So you don't have to "define" a sourcetype. You can set it to anything in your inputs.conf (as long as it's syntactically correct). Then you just have to set proper props.conf for handling this sourcetype.

Re: Create a New Source Types in SPLUNK

SplunkDash — Mon, 28 Mar 2022 19:14:48 GMT

Hello,

Thank you so much for your quick response appreciates it. If this is the case what is there in the GUI to create sourcetype in SPLUNK? Thank you again.

Re: Create a New Source Types in SPLUNK

PickleRick — Mon, 28 Mar 2022 19:44:03 GMT

The web client treats it a bit differently (although the underlying mechanics stay the same).

If you "create a sourcetype" in the web client, splunk defines some parsing settings for it (like line/event breaking, timestamp recognition, kv mode and such) - the stuff that you typically do in props.conf - so you can then chose this set of settings when creating an input. It's the web client that enforces this "consistency". It's just one of convenience features in splunk web aimed at less experienced users 🙂

In other words, whereas in config files you can do whatever stupidity you want 😉 like in unix, the webui holds your hand like windows 😄

But even the webui does this in a limited way. When you create an input you have to either select a sourcetype from the predefined list or create a new definition. But when you click on an already created input, you can change the sourcetype value to whatever you want regardless of whether you have it "defined" or not (in other words, whether ther are settings for parsing this sourcetype or not).

Re: Create a New Source Types in SPLUNK

SplunkDash — Tue, 29 Mar 2022 04:19:12 GMT

Hello,
I created source type based on your instructions. Thank you so much, appreciated. But getting error message and also getting 1 event instead of 10 events. I am not sure what's the mistake I have done. My props/inputs configuration files and sample source data are given below. Any help will be highly appreciated. Thank you again.

[oswindows_companion:preamble]
SHOULD_LINEMERGE=false
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
category=Structured
TIMESTAMP_FIELDS=Test Date
TIME_FORMAT=%Y-%m-%d %H:%M:%S
HEADER_FIELD_LINE_NUMBER=1

[monitor:///opt/splunk/var/log/windows_parsed_reports/companion_preamble/*.csv]
disabled=false
sourcetype=oswindows_companion:preamble
index=oswindows

Re: Create a New Source Types in SPLUNK

PickleRick — Tue, 29 Mar 2022 06:16:01 GMT

If this is your literal input file line, you definitely have date formatting inconsistent with the props.conf definition.

Re: Create a New Source Types in SPLUNK

SplunkDash — Tue, 29 Mar 2022 23:16:30 GMT

Hello,

Thank you so much for your feedback. What should I use for this?

TIME_FORMAT=%Y/%m/%d %H:%M:%S

It's not working as well

Re: Create a New Source Types in SPLUNK

PickleRick — Wed, 30 Mar 2022 09:47:13 GMT

The line from your input data doesn't show the seconds so you can't include ":%S" in the data format because it will never match anything. Also - your with your time format you should have 2022/02/24 whereas you have this illogical american date format 02/24/2022.

Try %m/%d/%Y %H:%M

Should work but can cause problems if you have 12-hour clock without explicitly provided AM/PM.

Re: Create a New Source Types in SPLUNK

SplunkDash — Wed, 30 Mar 2022 14:38:59 GMT

Hello,

Thank you so much. I tried

%m/%d/%Y %H:%M, still getting error message, but not sure why getting better result using %/m%/d%/Y %H:%M.