Getting Data In

Handling events with the same timestamp

New Member

I am extracting logs from a file which contain entries with two timestamp log entries:
1. eventTimestamp
2. timestamp

The later is included by my logging framework. I occasionally write events where the timestamp is the same. In these cases the events gets grouped together as shown below:

The events below would all appear under the timestamp: 2016-12-28T17:07:55.946Z.
```
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"xxxx","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:55.946Z","uid":"xxx","accountDetails":{"account":"xx","email":"xxxxx,"environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:56.766Z","uid":"xxx","accountDetails":{"account":"xxxx","email":"xxxx","environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}

```

I have tried creating a props.conf file with the following configuration:


[api_reporting]
SHOULD_LINEMERGE=false
TIME_PREFIX=eventTimestamp
MAX_TIMESTAMP_LOOKAHEAD=10

However, I am continuing to experience the issue. I have followed [1] to determine if my props.conf file is read and it seems to be the case. The configuration given above were taken from [2].

[1] https://docs.splunk.com/Documentation/Splunk/6.5.1/Troubleshooting/Usebtooltotroubleshootconfigurati...
[2] https://answers.splunk.com/answers/80488/splunk-treating-multiple-lines-as-one-event-since-they-have...

Tags (2)
0 Karma

SplunkTrust
SplunkTrust

Try this for your props.conf (on Indexer or Heavy Forwarder)

[api_reporting]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\{\"eventCode\")
TIME_PREFIX = eventTimestamp\":\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23
0 Karma

New Member

Thank You.

I did not get a chance to try it out yet. I will update the thread once I get a chance to test it.

0 Karma