Getting Data In

New Source type and Timestamp from Event

santhoshvelling
New Member

Hi Experts,

I have a even like below generated from my application.

{
"index": "exp_prod",
"host": "myhost.com",
"source": "app.logs",
"sourcetype": "_json",
"event": [
{
"Sender": "AZSB",
"Status": "COMPLETED",
"ApplicationMessageType": "utility",
"CustomStatus": "COMPLETED",
"ApplicationMessageId": "",
"MessageGuid": "AF61XzlbeOSc7c1yBkfQ-dTqo8VI",
"LogStart": "2020-05-08T13:31:37.053",
"Receiver": "JMS",
"CorrelationId": "AF61Xzm4KCX0sO8q3PGewmmlZqem",
"LogEnd": "2020-05-08T13:31:37.063"
},
{
"Sender": "AZSB",
"Status": "COMPLETED",
"ApplicationMessageType": "Article",
"CustomStatus": "NA",
"ApplicationMessageId": "180730",
"MessageGuid": "AF61Xzkb-vFb_xEgpfQw1mgNbPc5",
"LogStart": "2020-05-08T13:31:37.046",
"Receiver": "JMS",
"CorrelationId": "AF61XzkvcPiugQGqmXc6LrN3GQ42",
"LogEnd": "2020-05-08T13:31:37.063"
}
]
}

Now when I send this event to Spluk Cloud using HEC, it create two event's but the timestamp if the event is the current timestamp. However I want event time stamp to be populated from LogStart.

How to achieve this? I tried customer source type like below, but the result is same. Please assist.

alt text

Tags (2)
0 Karma

to4kawa
Ultra Champion
[your souecetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\[|,)\s*{\s*\"Sender
NO_BINARY_CHECK=true
SEDCMD-trim=s/\].*//g
TIME_PREFIX=LogStart\":\s\"

You'll have to split it up.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

When you use timestamp fields, Splunk combines all of the listed fields and then applies the timestamp format. In this example, the LogStart and LogEnd fields combine to "2020-05-08T13:31:37.0462020-05-08T13:31:37.063", which does not match the specified format so Splunk ignores it and sets the event time to the current time.

The Timestamp fields setting should contain the name of the event field(s) can together describe when the event happened. It does not need to list every timestamp in the event.

---
If this reply helps you, Karma would be appreciated.
0 Karma

santhoshvelling
New Member

Thank you for the response.
I tried below setting and that did not work either. The event still got the current timestamp.

DATETIME_CONFIG=

TIMESTAMP_FIELDS = LogStart
TIME_PREFIX = "LogStart": "
TIME_FORMAT = %Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=23
TZ=GMT

Not sure what is wrong! Any pointer to resolve? TIA

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try omitting DATETIME_CONFIG. Also, change TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...