All Apps and Add-ons

How do I create my own _time without modifying props file?

mani
Explorer

We are setting up Splunk for our application and need to load historical logs. But the time stamp for many of the event are taken as current/wrong dates. We cannot make changes to our conf files as they are shared among multiple projects. Is there an alternative to modify '_time' values to a value in the log file. without changing conf files?

Labels (2)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

If this is a one off ingestion, then see if you can request a temporary change to the sourcetype to add the MAX_DAYS_AGO setting just to get this data in, then that's probably your best option - the other route, as suggested by @isoutamo is likely to involve far more admin hurdles than this one.

There are not really any sensible options to make this work, when Splunk has the 2000 day default. That setting is typically there to catch unlikely outliers, but in your case, it's effectively blocking the ingest correctly.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

You should set up a definition in props.conf that will extract the times correctly. There should be no reason why you cannot make a new setting for your particular log files that would not affect others.

Once the data is in Splunk, you cannot modify the event _time.

What is wrong with your existing timestamps that makes them ingest incorrectly?

If they are historical, how old are the dates - check the MAX_DAYS_AGO setting, which controls how far back Splunk will consider a date to be valid (default 2000 days)

 

0 Karma

mani
Explorer

Hey @bowesmana ,

I checked with the admin team regarding props.conf settings. Unfortunately, we are not supposed to make changes to those file.

Do we have another way to proceed? 

Below image shows the timestamps we are providing and the timestamp Splunk is considering.

mani_0-1596186314823.png

 

Thanks in advance.

0 Karma

bowesmana
SplunkTrust
SplunkTrust

If this is a one off ingestion, then see if you can request a temporary change to the sourcetype to add the MAX_DAYS_AGO setting just to get this data in, then that's probably your best option - the other route, as suggested by @isoutamo is likely to involve far more admin hurdles than this one.

There are not really any sensible options to make this work, when Splunk has the 2000 day default. That setting is typically there to catch unlikely outliers, but in your case, it's effectively blocking the ingest correctly.

 

isoutamo
SplunkTrust
SplunkTrust

Hi

I agree with @bowesmana that you should fix props.conf as it extracts incorrectly time. 

If this is politically impossible solution, then you could setup temporary HF with fixed props.conf which reads those files and send those to the indexers. 
r. Ismo

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...