Knowledge Management

Can you tell when a sourcetype was created in splunk over time?

aecruzp
Path Finder

Good afternoon

    By topics of analysis it is required to know when a sourcetype was created, I know that the configurations can be checked via the console in inputs.conf when monitoring a new log, but when you want to know a historical ..? We currently have 3000 sourcetypes and it would be very tedious.

   This configuration would be in the internal of splunk ?, or can we know when a sourcetye was created only when data was indexed?
  
   I hope you can help me, any comment is appreciated

Greetings.

Tags (1)
0 Karma

somesoni2
Revered Legend

I don't think there is a way to know when the sourcetype was created. Splunk doesn't create a dictionary of when a sourcetype (or any other knowledge object) was created. I believe you can only gather when was a sourcetype first reported (either use _time: time of the events, or better, _indextime: time when data arrived in Splunk). Please note that this will depend upon what's your retention period is. If you've retention period of 90 days, you can only query when was the data for a sourcetype came first in last 90 days, even though you started gathering it 2 years back.

FrankVl
Ultra Champion

You can, using the metadata command. For example:

| metadata type=sourcetypes | convert ctime(firstTime)
0 Karma

aecruzp
Path Finder
| tstats count WHERE index=* OR index=_* by index sourcetype 

the truth is that there are more source types in our cluster since many are monitored where sourcetypes are not configured in the master.

0 Karma

FrankVl
Ultra Champion

You can of course simply add _time to the by-clause of that tstats command, and then do some sorting and deduplication to filter out the oldest timestamp for each sourcetype.

0 Karma

aecruzp
Path Finder

thanks for the reply

But checking the date some sourcetypes have date 1996 😞

11/3/1996 17:15:43 1905444307 1468200621 opsc 677639 sourcetypes

any ideas? the other, currently in the indexer I have more than 400 sourcetype, and in the search I see very few.

0 Karma

FrankVl
Ultra Champion

Sounds like you have had some weird time stamped event for that sourcetype then? Could be interesting to search the events around that timeframe for that sourcetype.

Over what timeframe did you run the metadata search?

How did you find out you have over 400 sourcetypes?

Are you rewriting sourcetypes @ searchtime?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...