Getting Data In

Handling events with the same timestamp

Staging_2
New Member

I am extracting logs from a file which contain entries with two timestamp log entries:
1. eventTimestamp
2. timestamp

The later is included by my logging framework. I occasionally write events where the timestamp is the same. In these cases the events gets grouped together as shown below:

The events below would all appear under the timestamp: 2016-12-28T17:07:55.946Z.
```
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"xxxx","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:55.946Z","uid":"xxx","accountDetails":{"account":"xx","email":"xxxxx,"environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:56.766Z","uid":"xxx","accountDetails":{"account":"xxxx","email":"xxxx","environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}

```

I have tried creating a props.conf file with the following configuration:


[api_reporting]
SHOULD_LINEMERGE=false
TIME_PREFIX=eventTimestamp
MAX_TIMESTAMP_LOOKAHEAD=10

However, I am continuing to experience the issue. I have followed [1] to determine if my props.conf file is read and it seems to be the case. The configuration given above were taken from [2].

[1] https://docs.splunk.com/Documentation/Splunk/6.5.1/Troubleshooting/Usebtooltotroubleshootconfigurati...
[2] https://answers.splunk.com/answers/80488/splunk-treating-multiple-lines-as-one-event-since-they-have...

Tags (2)
0 Karma

somesoni2
Revered Legend

Try this for your props.conf (on Indexer or Heavy Forwarder)

[api_reporting]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\{\"eventCode\")
TIME_PREFIX = eventTimestamp\":\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23
0 Karma

Staging_2
New Member

Thank You.

I did not get a chance to try it out yet. I will update the thread once I get a chance to test it.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...