How to create a New Source Types in SPLUNK?

[SplunkDash]

Hello,

I know we can use SPLUNK GUI to create source types. But how I would create a new source type from CLI or using props.conf file. Any help will be highly appreciated, thank you.

Does this props.conf  is going to create  new source type test:audit  if it doesn't exist

[test:audit]

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]*)<MODTRANSAUDTRL>
TIME_PREFIX= <TIMESTAMP>
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%2N
MAX_TIMESTAMP_LOOKAHEAD=24
TRUNCATE=1000

[PickleRick]

Sourcetype is just a value. It's just that using this value splunk decides what to do with an event. So in general, anything that you associate with an event on input is a sourcetype. Then you're only telling splunk what to do with event which has this sourcetype by means of props and transforms.

So you don't have to "define" a sourcetype. You can set it to anything in your inputs.conf (as long as it's syntactically correct). Then you just have to set proper props.conf for handling this sourcetype.

[SplunkDash]

Hello,

Thank you so much for your quick response appreciates it. If this is the case what is there in the GUI to create sourcetype in SPLUNK? Thank you again.

[PickleRick]

The web client treats it a bit differently (although the underlying mechanics stay the same).

If you "create a sourcetype" in the web client, splunk defines some parsing settings for it (like line/event breaking, timestamp recognition, kv mode and such) - the stuff that you typically do in props.conf - so you can then chose this set of settings when creating an input. It's the web client that enforces this "consistency". It's just one of convenience features in splunk web aimed at less experienced users 🙂

In other words, whereas in config files you can do whatever stupidity you want 😉 like in unix, the webui holds your hand like windows 😄

But even the webui does this in a limited way. When you create an input you have to either select a sourcetype from the predefined list or create a new definition. But when you click on an already created input, you can change the sourcetype value to whatever you want regardless of whether you have it "defined" or not (in other words, whether ther are settings for parsing this sourcetype or not).

[SplunkDash]

Hello,
I created source type based on your instructions. Thank you so much, appreciated. But getting error message and also getting 1 event instead of 10 events. I am not sure what's the mistake I have done. My props/inputs configuration files and sample source data are given below. Any help will be highly appreciated. Thank you again.

[oswindows_companion:preamble]
SHOULD_LINEMERGE=false
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
category=Structured
TIMESTAMP_FIELDS=Test Date
TIME_FORMAT=%Y-%m-%d %H:%M:%S
HEADER_FIELD_LINE_NUMBER=1

[monitor:///opt/splunk/var/log/windows_parsed_reports/companion_preamble/*.csv]
disabled=false
sourcetype=oswindows_companion:preamble
index=oswindows

[PickleRick]

If  this is your literal input file line, you definitely have date formatting inconsistent with the props.conf definition.

[SplunkDash]

Hello,

Thank you so much for your feedback. What should I use for this?

TIME_FORMAT=%Y/%m/%d %H:%M:%S

It's not working as well

[PickleRick]

The line from your input data doesn't show the seconds so you can't include ":%S" in the data format because it will never match anything. Also - your with your time format you should have 2022/02/24 whereas you have this illogical american date format 02/24/2022.

Try %m/%d/%Y %H:%M

Should work but can cause problems if you have 12-hour clock without explicitly provided AM/PM.

[SplunkDash]

Hello, 

Thank you so much. I tried

%m/%d/%Y %H:%M, still getting error message, but  not sure why getting better result using %/m%/d%/Y %H:%M.