Getting Data In

Handling events with the same timestamp

Staging_2
New Member

I am extracting logs from a file which contain entries with two timestamp log entries:
1. eventTimestamp
2. timestamp

The later is included by my logging framework. I occasionally write events where the timestamp is the same. In these cases the events gets grouped together as shown below:

The events below would all appear under the timestamp: 2016-12-28T17:07:55.946Z.
```
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"xxxx","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:55.946Z","uid":"xxx","accountDetails":{"account":"xx","email":"xxxxx,"environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}
{"eventCode":13400,"eventDetails":{"title":"xxxx","source":"xxx","code":"InvalidArgument","subscriptionType":"active","system":"xxx"},"userDetails":{"userAgent":"","userApp":"sc"},"eventTimestamp":"2016-12-28T17:07:56.766Z","uid":"xxx","accountDetails":{"account":"xxxx","email":"xxxx","environment":"xxx"},"level":"info","message":"","timestamp":"2016-12-28T17:07:39.593Z"}

```

I have tried creating a props.conf file with the following configuration:


[api_reporting]
SHOULD_LINEMERGE=false
TIME_PREFIX=eventTimestamp
MAX_TIMESTAMP_LOOKAHEAD=10

However, I am continuing to experience the issue. I have followed [1] to determine if my props.conf file is read and it seems to be the case. The configuration given above were taken from [2].

[1] https://docs.splunk.com/Documentation/Splunk/6.5.1/Troubleshooting/Usebtooltotroubleshootconfigurati...
[2] https://answers.splunk.com/answers/80488/splunk-treating-multiple-lines-as-one-event-since-they-have...

Tags (2)
0 Karma

somesoni2
Revered Legend

Try this for your props.conf (on Indexer or Heavy Forwarder)

[api_reporting]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\{\"eventCode\")
TIME_PREFIX = eventTimestamp\":\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23
0 Karma

Staging_2
New Member

Thank You.

I did not get a chance to try it out yet. I will update the thread once I get a chance to test it.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...

Monitoring AI Agents with Splunk Observability Cloud

Let’s say I’m running a travel planning AI app in production. A user asks for three concise hotel options in ...

[Puzzles] Solve, Learn, Repeat: Tiling

This puzzle (first published here) is based on finding groups of tessellated tiles (inspired by floor tiles I ...