Getting Data In

Any idea why the timestamp of data that send via logstash, change when stored in Splunk index?

indeed_2000
Motivator

Hi
timestamp of data that send via logstash change when store in splunk index. what is the reason?

index="influx2splunk" | spath input=_raw | table time _time @timestamp _raw

time                                          _time                               @timestamp
2023-06-15T06:06:55Z 2023-06-15 05:06:55 2023-06-15T01:36:55.000Z
2023-06-15T06:06:55Z                                               2023-06-15T01:36:55.000Z

 

here is the _raw data that get from logstash:


{"usage_irq":0,"usage_user":4.373757455295997,"results":{"statement_id":0},"@version":"1","@timestamp":"2023-06-15T01:36:55.000Z","usage_guest":0,"cpu":"cpu20","usage_iowait":0,"usage_softirq":0.39761431396001656,"http_poller_metadata":{"input":{"http_poller":{"response":{"status_code":200,"status_message":"OK","headers":{"date":"Sat, 17 Jun 2023 06:05:47 GMT","x-influxdb-build":"OSS","x-influxdb-version":"1.7.8","transfer-encoding":"chunked","x-request-id":"00a6ba2f-0cd5-11ee-981b-005056b7dda2","content-type":"application/json","request-id":"00a6ba2f-0cd5-11ee-981b-005056b7dda2"},"elapsed_time_ns":797045},"request":{"name":"cpu","original":{"url":"https://192.168.1.1:8086/query?pretty=true&db=mydb&q=myquery","headers":{"Authorization":"Token mytoken"},"method":"get"},"retry_count":0,"host":{"hostname":"srv"}}}}},"usage_idle":92.04771372774293,"usage_system":3.1809145128373424,"usage_steal":0,"time":"2023-06-15T06:06:55Z","name":"cpu","usage_nice":0,"usage_guest_nice":0}

 

 

logstash config:

filter {
split {
field => "results"
}

split {
field => "[results][series]"
}

split {
field => "[results][series][values]"
}

mutate {
rename => { "[results][series]" => "series" }
}

mutate {
rename => { "[series][name]" => "name" }
}


ruby {
code => 'series = event.get("series"); series["columns"].each_with_index {|val, index| event.set(val, event.get("[series][values][" + index.to_s() + "]"))}'
}


date {
match => ["time", "yyyy-MM-dd'T'HH:mm:ss:SSS'Z"]
target => "_time"

}


prune {
blacklist_names => [ "event", "host", "series" ]
}

 

}

 

 

Any idea?

Thanks

Labels (1)
Tags (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Timestamps don't "change" on their own. You can instruct splunk to adjust the _time field in some cases but you have to do so explicitly. Most probably your source is not properly onboarded and you don't have properly configured timestamp recognition so the parsing component assumes the timestamp of the indexing moment.

0 Karma

indeed_2000
Motivator

@PickleRickis it possible to fix it in logstash without change config in splunk?

I mean set logstash in somehow to consider timestamp as field that use as "_time"

 

Any idea?

Thanks

0 Karma

PickleRick
SplunkTrust
SplunkTrust

I have no idea. We don't know what is your splunk config, how you're ingesting the logs (you only showed the filter part of the logstash config) and so we can't know whether making both components work with each other requires changes on one side or the other or both.

Also if you paste a piece of config or a raw event, use either the code block (the </> icon on top of the editor) or the preformatted paragraph style.

0 Karma

indeed_2000
Motivator

@PickleRickone suspicious thing that happen here is  one field that called "time" and  "@timestamp" duplicate date but in raw log only one date exist!

 

when i run below SPL return two date!

index="influx2splunk" | spath input=_raw | table time _time @timestamp _raw

time                                           _time @                               @timestamp
2023-06-15T06:06:55Z    2023-06-15 05:06:55   2023-06-15T01:36:55.000Z
2023-06-15T06:06:55Z                                                   2023-06-15T01:36:55.000Z

 

 

 

on raw data the only one date store in these fields:

 

{"usage_irq":0,"usage_user":4.373757455295997,"results":{"statement_id":0},"@version":"1","@timestamp":"2023-06-15T01:36:55.000Z","usage_guest":0,"cpu":"cpu20","usage_iowait":0,"usage_softirq":0.39761431396001656,"http_poller_metadata":{"input":{"http_poller":{"response":{"status_code":200,"status_message":"OK","headers":{"date":"Sat, 17 Jun 2023 06:05:47 GMT","x-influxdb-build":"OSS","x-influxdb-version":"1.7.8","transfer-encoding":"chunked","x-request-id":"00a6ba2f-0cd5-11ee-981b-005056b7dda2","content-type":"application/json","request-id":"00a6ba2f-0cd5-11ee-981b-005056b7dda2"},"elapsed_time_ns":797045},"request":{"name":"cpu","original":{"url":"https://192.168.1.1:8086/query?pretty=true&db=mydb&q=myquery","headers":{"Authorization":"Token mytoken"},"method":"get"},"retry_count":0,"host":{"hostname":"srv"}}}}},"usage_idle":92.04771372774293,"usage_system":3.1809145128373424,"usage_steal":0,"time":"2023-06-15T06:06:55Z","name":"cpu","usage_nice":0,"usage_guest_nice":0}

 

 

Any idea?

Thanks

0 Karma

isoutamo
SplunkTrust
SplunkTrust

We have already discussed about this on https://community.splunk.com/t5/Splunk-Search/Is-it-possible-to-consider-quot-time-quot-as-quot-time.... If you have read and followed those instructions you should have correct _time field on your indexed data.

richgalloway
SplunkTrust
SplunkTrust

What is your question?

---
If this reply helps you, Karma would be appreciated.
0 Karma

indeed_2000
Motivator

@richgalloway timestamp of data that send via logstash change when store in splunk index. what is the reason?

0 Karma
Get Updates on the Splunk Community!

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Using the Splunk Threat Research Team’s Latest Security Content

REGISTER HERE Tech Talk | Security Edition Did you know the Splunk Threat Research Team regularly releases ...