Why is the Tenable vulnerability time stamp off?

[danielbb]

We're using the Tenable Add-on for Splunk (TA-tenable) to ingest data from Tenable.io. the app's props.conf, has the following -

[tenable:io:vuln]
DATETIME_CONFIG = NONE

When we run the following SPL:

index=tenable sourcetype="tenable:io:vuln"
| eval lag = _indextime - _time

We are seeing non-zero lag values, even though I expect the lag to be zero if _time truly equals _indextime.

If anything, I would expect DATETIME_CONFIG = CURRENT, what am I missing?

[PickleRick]

1. As @richgalloway mentioned, a (typically small but it realy depends on the input) lag between _time and _indextime is a normal state. Or at least on its own it doesn't mean that something is wrong.

2. DATETIME_CONFIG=none explicitly disables timestamp recognition. Are you sure it is what you want?

3. If there is a difference between the timestamp included in the raw event and the timestamp stored in the _time field, the data is not properly onboarded. Tenable.io is a cloud service so I suppose there is some modular input which pulls the data from the cloud and pushes them to Splunk. But I have no idea whether the timestamps should be parsed by the input itself and fed "as is" to Splunk or if the data should be parsed in Splunk. Infortunately, it's a third party add-on so there can be completely everything happening inside...

[aromanauskas]

DATETIME_CONFIG = CURRENT != NONE 

Current set the timestamp from the Aggregation queue time. 
None in this instance sets the timestamp from the time handed over to Splunk by the modular input script. 

Splunk then still needs to send the data to an indexer, which is where the _indextime will be set. Yes the data is cooked and time set on the HeavyForwarder but note the _indextime is NOT. 

An easy way to see this in action is to look at any of your data being ingested by DBConnect with _time being set to Current. The _indextime will usually be negative,  but every once in a while you'll see it jump to a few seconds usually due to a blocked output queue. And of course any difference between the HeavyForwarder and Indexer time will of course cause times to be off as well. 

[richgalloway]

It's normal for _indextime to not exactly match _time since there's always a delay from event transmission and processing.  How big is the lag you see?

[danielbb]

Thank you @richgalloway 

My question is, why an app sets something like?

[tenable:io:vuln]
DATETIME_CONFIG = NONE

That's what this tenable TA does, I don't get it.

[Prewin27]

@danielbb 

_time is being set by the Tenable Add-on itself, using a timestamp field from the Tenable API response (e.g., last_seen, last_found).

DATETIME_CONFIG = NONE in props.conf for tenable is intentional to prevent Splunk from trying to re-parse _time from the event's raw data and potentially overriding the TA's carefully chosen timestamp

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a kudos/Karma. Thanks!

[danielbb]

Thank you @Prewin27,

I'm looking at props.conf and transforms.conf of the TA and I don't see any references to last_seen and last_found. What am I missing?