Splunk Search

How to fix one logfile with 3 different change-times?

dexxter275
Explorer

Hello,

I have the following problem with every logfile on splunk. I explain it with one logfile as example.

Logfile on the splunk-server has the change date: Modify: 2017-03-22 08:35:01.022152819 +0100
-> This is correct. In every 35th minute splunk is getting the new logfile.

Logfile on the splunk webinterface has the change date 2017/03/17 2:20:02 PM
-> This isn't correct. So I thought that splunk just crashed and didn't recognized the new logfiles.

The last entry in the logfile over the manually search on the splunk interfaces is from 2017/03/21 01:59:59 PM
-> But with that point I understand that splunk still worked over the 2017/03/17 and uploaded the logfiles until 2017/03/21

But now splunk doesn't refresh the logfiles... So i have ONE logfile with three different change dates?!

I hope you have any idea....

kind regards

PS: Filesystem isn't full. Restart doesn't help. Logfiles aren't empty.
Maybe I need to do a fielsystemcheck?

Tags (1)
0 Karma
1 Solution

dexxter275
Explorer

Wow.... I fixed it!

I didn't expect that Splunk delete his own configuration. The information about the path to the logfiles missed....
I add it about the Data Input menu. And know it works....

Thanks everybody anyway

View solution in original post

0 Karma

dexxter275
Explorer

Wow.... I fixed it!

I didn't expect that Splunk delete his own configuration. The information about the path to the logfiles missed....
I add it about the Data Input menu. And know it works....

Thanks everybody anyway

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

What kind of logfile is this? How are you ingesting the log file into Splunk? Make sure the timestamp configuration is accurate in your sourcetype definition.

0 Karma

dexxter275
Explorer

Its an normal text logfile. At this point I collect the information into one file and push this with scp on the splunk server.
The configuration and the whole system work 3 weeks really well. I have this problem since 2017/03/21...

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

It's a normal text file, elaborate on that statement.. Splunk doesn't handle when log file formats change.

Splunk is only as good as the data you put in. If it was working, sound like something changed. You should be defining the source type and timestamp recognition for this.

Is the event single line?
Multiline?
What's the time stamp format?

Provide and example of your log file.

0 Karma

dexxter275
Explorer

I choosed a path on the splunk server as source. This path includes 2 logfiles I want to check. They gonna refresh every hour. That works really good without problems. I changed nothing on the system since a week.

What do you mean with single line? Which event? Do you mean the logfile entries? They are single lines yes.
Timestamp format is since the begin in every logfile 2017/03/23 11:00:44.978

Modified example of the logfile:

/path/to/logfile/processname.log.log:2017/03/23 11:00:07.084 ERROR: errormessage {function}
/path/to/logfile/processname.log.log:2017/03/23 11:00:10.409 ERROR: errormessage occured during run {function}
/path/to/logfile/processname.log.log:2017/03/23 11:00:12.115 ERROR: errormessage {function}
/path/to/logfile/processname.log.log:2017/03/23 11:00:14.610 ERROR: NoRecordsFoundException: null {function} 
/path/to/logfile/processname.log.log:2017/03/23 11:00:18.290 ERROR: errormessage {function}
/path/to/logfile/processname.log.log:2017/03/23 11:00:19.833 ERROR: errormessage {function}
0 Karma

dexxter275
Explorer

UPDATE:
Now the logfile on the webinterface has the same changedate as the last entry in the logfile.

The problem is just that there are new logfiles on the server, but splunk doesnt recognize it.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...