Getting Data In

Why is props.conf current date not working?

yohhpark
Path Finder

test_id": "CHICKEN-0123456",
"last_test_date": "2023-09-04 12:34:00"

 

with such above file and todays date 09/25/2023

 

once it is monitored by the splunk, I cannot search this data with the 'current' date or even current time; 15 or 60mintues.

 

instead it tends to read the dates off of the file which is the 'last test date' = 09/24/2023 therefore from the search I have to put either on that day or 1day to find the data.

 

Props.conf currently set as 

DATETIME_CONFIG = CURRENT

 

I want the file to be 'read' today if it was uploaded today. (or 15 min if it was uploaded within 15min) NOT going off of the date in the file.

 

Gurus hop in plesae.

Labels (1)
Tags (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Ok. First things first - where and how do you ingest the files and where do you have the props.conf with the DATETIME_CONFIG setting? And are you sure it is being active?

yohhpark
Path Finder

forwarder is forwarding (ex) /var/log/test.txt

and the file IS test.txt

and it is active because I can see the files from the search, except the dates are not feeding.

props.conf is seating on the /etc/apps/test/local/props.conf

Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. But on which component did you put this props.conf file? On the UF? Then it's not the proper place for it. UF does not do parsing (except for indexed extractions but it's not the case I suppose) and DATETIME_CONFIG is a setting regarding parsing stage. So put it onto your indexer(s) or intermediate HF(s) if you have them.

Also - did you configure DATETIME_CONFIG for the proper sourcetype?

0 Karma

Manojbh_splunk
Loves-to-Learn

what is the timezone your server is in.

current_time takes current system time

0 Karma

yohhpark
Path Finder

it's EST.

also that is not the problem. it's the date also.

 

docuemtn name is 09042023_test.txt

and inside it has something like

ID= 101010

processed_date=09/03/2023

 

and today's date is 09/05/2023

 

 

but when the forwarder forwards, it takes the date inside of the document resulting the search has to go 2 days back to find the data

0 Karma

yohhpark
Path Finder

anything new you've found? still couldn't solved the issue.

0 Karma

Manojbh_splunk
Loves-to-Learn

Can you share your props.conf file.

i think you are forcing splunk to take _time as processedtime in logs

0 Karma

yohhpark
Path Finder

it's really nothing to share honestly.

 

[test]

DATETIME_CONFIG = current

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...