Getting Data In

Handling events with the same timestamp

Staging_2
New Member

I am extracting logs from a file which contain entries with two timestamp log entries:
1. eventTimestamp
2. timestamp

The later is included by my logging framework. I occasionally write events where the timestamp is the same. In these cases the events gets grouped together as shown below:

The events below would all appear under the timestamp: 2016-12-28T17:07:55.946Z.
```
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"xxxx","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:55.946Z","uid":"xxx","accountDetails":{"account":"xx","email":"xxxxx,"environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:56.766Z","uid":"xxx","accountDetails":{"account":"xxxx","email":"xxxx","environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}

```

I have tried creating a props.conf file with the following configuration:


[api_reporting]
SHOULD_LINEMERGE=false
TIME_PREFIX=eventTimestamp
MAX_TIMESTAMP_LOOKAHEAD=10

However, I am continuing to experience the issue. I have followed [1] to determine if my props.conf file is read and it seems to be the case. The configuration given above were taken from [2].

[1] https://docs.splunk.com/Documentation/Splunk/6.5.1/Troubleshooting/Usebtooltotroubleshootconfigurati...
[2] https://answers.splunk.com/answers/80488/splunk-treating-multiple-lines-as-one-event-since-they-have...

Tags (2)
0 Karma

somesoni2
Revered Legend

Try this for your props.conf (on Indexer or Heavy Forwarder)

[api_reporting]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\{\"eventCode\")
TIME_PREFIX = eventTimestamp\":\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23
0 Karma

Staging_2
New Member

Thank You.

I did not get a chance to try it out yet. I will update the thread once I get a chance to test it.

0 Karma
Get Updates on the Splunk Community!

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...

Part 2: A Guide to Maximizing Splunk IT Service Intelligence

Welcome to the second segment of our guide. In Part 1, we covered the essentials of getting started with ITSI ...

Part 1: A Guide to Maximizing Splunk IT Service Intelligence

As modern IT environments continue to grow in complexity and speed, the ability to efficiently manage and ...