topic Re: Sourcetype Missing in Getting Data In

Sourcetype Missing

rahul2gupta — Tue, 14 Jul 2020 10:38:55 GMT

We are using the following query index=main sourcetype=wms_oracle_sessions | bucket span=5m _time | stats count AS sessions by _time,warehouse,machine,program | search warehouse=ql | stats sum(sessions) AS psessions by _time,program | timechart avg(psessions) by program

I found that sourcetype does not exist.Can we create the sourcetype with same name i.e wms_oracle_sessions.

If yes,what else do I need to do to get the events?

Regards,

Rahul

Re: Sourcetype Missing

isoutamo — Tue, 14 Jul 2020 10:47:04 GMT

Add needed definitions to props.conf and also to transformations.conf (if needed). After that those apply to the new events (not to old events unless you reindex those events).

I also moved “search warehouse=ql” to the base search.

Is the machine needed in the 1st stats as you don’t use it later?

r. Ismo

Re: Sourcetype Missing

rahul2gupta — Wed, 15 Jul 2020 05:40:23 GMT

Hi @isoutamo ,

We have three servers in our environment.

Forwarder -- axxxfd05
Indexer --axxxxlnd05
Search Head -- axxxhd05

Can you please suggest on which server I need to add definition in props.conf and how will I come to know that I need to add definition in transformations.conf as well.

What definition I need to add in props.conf and reindex those events how it is done?

Regards,

Rahul

Re: Sourcetype Missing

gcusello — Wed, 15 Jul 2020 06:33:38 GMT

Hi @rahul2gupta,

to override index definition, you have to put props.conf and transforms.conf on Indexers or (when present) on Heavy Forwarders.

Ciao.

Giuseppe

Re: Sourcetype Missing

rahul2gupta — Wed, 15 Jul 2020 06:51:40 GMT

Hi @gcusello ,

I checked in props.conf but could not understand anything.

what definition I should add in props.conf for sourcetype=wms_oracle_sessions.

Regards,

Rahul

Re: Sourcetype Missing

gcusello — Wed, 15 Jul 2020 06:55:36 GMT

Hi @rahul2gupta,

what do you mean with sourcetype doesn't exist?

sourcetype is usually defined in inputs.conf on the UFs, could you share the inputs.conf to read these logs?

Ciao.

Giuseppe

Re: Sourcetype Missing

rahul2gupta — Wed, 15 Jul 2020 07:20:22 GMT

Hi @gcusello ,

sourcetype wms_oracle_sessions is not defined in inputs.conf,perhaps there is nothing there.

[root@axxxfd01 local]# cat inputs.conf
[root@axxxfd01 local]#

Regards,

Rahul

Re: Sourcetype Missing

gcusello — Wed, 15 Jul 2020 07:29:39 GMT

Hi @rahul2gupta,

where do these logs come from?

if from file, find the correct inputs.conf and insert sourcetype definition.

If you don't know what's the correct inputs.conf, you can use btool:

./splunk cmd btool inputs list -debug > my_inputs.txt

If from DB-Connect, configure sourcetype here.

Ciao.

Giuseppe

Re: Sourcetype Missing

rahul2gupta — Wed, 15 Jul 2020 08:53:14 GMT

Hi @gcusello ,

Yeah we are trying to configure DB Connect V1 dashboard.

Actually,java bridge server was not working but now it is working.

But it's dashboard is not working.

Following is the query of EW:Oracle Sessions By Program.

What could be done to solve this enigma.

Help me,Legend.

Regards,

Rahul

Re: Sourcetype Missing

gcusello — Wed, 15 Jul 2020 09:53:40 GMT

Hi @rahul2gupta,

at first check if you can use a more recent version of DB-Conect!

then see in DB-Connect input the sourcetype assignment, maybe it's ony a different one!

try to search in index=main if there are the logs from that input, maybe they have a different sourcetype or maybe there ariived but then stopped.

Ciao.,

Giuseppe

Re: Sourcetype Missing

rahul2gupta — Wed, 15 Jul 2020 11:09:07 GMT

Hi @gcusello ,

I checked at the following location.

Here is the following screenshot .

And could not find sourcetype=wms_oracle_sessions.

Regards,

Rahul

Re: Sourcetype Missing

gcusello — Wed, 15 Jul 2020 11:15:12 GMT

Hi @rahul2gupta,

this means that there are two choices:

you haven't a connection to extract logs from Oracle with that sourcetype, so you have to create it,
you have the connection, but you're using a wrong sourcetype in your search, so you have to modify your dashboards using the correct sourcetype.

Check which is the real situation.

Ciao.

Giuseppe

Re: Sourcetype Missing

rahul2gupta — Thu, 16 Jul 2020 05:58:50 GMT

Hi @gcusello ,

We believe that we do not have a connection to extract logs from Oracle with that sourcetype, so we will create it.

who will help me with the following details,DBA Team?

Regards,

Rahul

Re: Sourcetype Missing

gcusello — Thu, 16 Jul 2020 07:02:46 GMT

Hi @rahul2gupta,

here you can find all the documentation about DB-Connect

https://docs.splunk.com/Documentation/DBX/3.3.1/DeployDBX/AboutSplunkDBConnect

and here all the steps to configure your DB input:

https://docs.splunk.com/Documentation/DBX/3.3.1/DeployDBX/ConfigureDBConnectsettings

in few words: if you already have the connection with Oracle DB (driver, identity, etc...), you have only to configure the input inserting:

connection name,
input type (usually tail),
database,
table or query to extract data,
rising column: this is the most important parameter: it's a growing identifier that permits to DB Connect to understand where the last extraction stopped, if you have't it, you have to create it using eventually the merge of two columns (e.g. date+id),
Splunk fields: sourcetype, index and host,
output format (usually the default),
timestamp column and timestamp format (as all the Splunk inputs),
interval (as all the Splunk scripts).

Anyway, my hint is to follow the documentation and not my very short summary!

Ciao.

Giuseppe

Re: Sourcetype Missing

rahul2gupta — Thu, 27 Aug 2020 06:08:42 GMT

Hi @gcusello ,

Rising column: this is the most important parameter: it's a growing identifier that permits to DB Connect to understand where the last extraction stopped, if you have't it, you have to create it using eventually the merge of two columns (e.g. date+id).

Could you please guide how to create Rising column as I couldn't find any relevant documentation.

Regards,

Rahul.

Re: Sourcetype Missing

gcusello — Thu, 27 Aug 2020 06:58:30 GMT

Hi @rahul2gupta,

the rising_column is one of the fields of your query with the feature to be always growing, e.g. a progressive number; in this way DB-Connct, at every run, writes the last value of the rising column and next time starts from it.

If you have a progressive number (prog), you can use it as rising_column, e.g.;

select date, name, prog, account from my_table

if you have a progressive number dayly resetted, you cannot use it as is but you can merge date and progressive number as rising_column (my_prog), e.g. (sorry for my SQL!):

select date, name, prog, account, date&prog AS my_prog from my_table

In other words, you move the problem from Splunk to SQL (and I cannot help you more!).

Ciao.

Giuseppe