topic Re: Splunk extraction help! in Splunk Search

Splunk extraction help!

kc_prane — Fri, 16 Aug 2024 19:56:48 GMT

Hello , I have a transaction which is coming as multievent. i can use the "| transaction" command to club as one event.

1) I want the transaction ID extracted based on the below-highlighted ( Green)

2) Now, I want to get the transaction time based on the below-highlighted (Yellow)

Below is the raw event log.

Thanks In advance!

Re: Splunk extraction help!

ITWhisperer — Fri, 16 Aug 2024 20:02:12 GMT

Which (if any) fields do you already have extracted?

Are the transaction ids unique i.e will there be only one "Starting ..." message and one "Successfully completed" message per transaction id?

Please can you share text versions of your events rather than pictures as they are easier to deal with when simulating a solution.

Re: Splunk extraction help!

kc_prane — Fri, 16 Aug 2024 20:10:20 GMT

Hi @ITWhisperer Thanks for your response. I had not extracted any yet cause the logs are not yet in splunk but will be soon Yes, the transaction ID are unique. The below is what i got from cloud watch.

2024-08-12T10:04:16.962-04:00 (434-abc-345789-de456ght) Extended Request Id: cmtf1111111111111111=

2024-08-12T10:04:16.963-04:00 (434-abc-345789-de456ght) Verifying Usage Plan for request: AAAAAAAAAAAAAAAAAAAAAAAA

2024-08-12T10:04:16.964-04:00 (434-abc-345789-de456ght) BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

2024-08-12T10:04:16.964-04:00 (434-abc-345789-de456ght) AAAAAAAAAABBBBBBBBBBBBBCCCCCCCCCCCCCCCCCC

2024-08-12T10:04:16.964-04:00 (434-abc-345789-de456ght) Starting execution for request: 8hhhhh-cdcd-434444-8bbb-dedr44444

2024-08-16T10:04:16.964-04:00 (434-abc-345789-de456ght) HTTP Method: POST, Resource Path: /ddd/Verifyffghhjj/ddddddd

2024-08-16T10:04:25.969-04:00 (434-abc-345789-de456ght) Successfully completed execution

2024-08-16T10:04:25.969-04:00 (434-abc-345789-de456ght) Method completed with status: 200

2024-08-16T10:04:25.969-04:00 (434-abc-345789-de456ght) AAAAAA Integration Endpoint RequestId: 11111111111111111111

Re: Splunk extraction help!

yuanliu — Sat, 17 Aug 2024 06:22:53 GMT

Do you mean something like this:

| rex "^\S+\s+\((?<transaction_id>[^\)]+)" | transaction transaction_id startswith="Starting execution for request" endswith="Successfully completed execution"

Here is an emulation of your mock sample data you can play with and compare with real data

| makeresults format=csv data="_raw 2024-08-12T10:04:16.962-04:00 (434-abc-345789-de456ght) Extended Request Id: cmtf1111111111111111= 2024-08-12T10:04:16.963-04:00 (434-abc-345789-de456ght) Verifying Usage Plan for request: AAAAAAAAAAAAAAAAAAAAAAAA 2024-08-12T10:04:16.964-04:00 (434-abc-345789-de456ght) BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB 2024-08-12T10:04:16.964-04:00 (434-abc-345789-de456ght) AAAAAAAAAABBBBBBBBBBBBBCCCCCCCCCCCCCCCCCC 2024-08-12T10:04:16.964-04:00 (434-abc-345789-de456ght) Starting execution for request: 8hhhhh-cdcd-434444-8bbb-dedr44444 2024-08-16T10:04:16.964-04:00 (434-abc-345789-de456ght) HTTP Method: POST, Resource Path: /ddd/Verifyffghhjj/ddddddd 2024-08-16T10:04:25.969-04:00 (434-abc-345789-de456ght) Successfully completed execution 2024-08-16T10:04:25.969-04:00 (434-abc-345789-de456ght) Method completed with status: 200 2024-08-16T10:04:25.969-04:00 (434-abc-345789-de456ght) AAAAAA Integration Endpoint RequestId: 11111111111111111111" | rex "^(?<_time>\S+)" | eval _time = strptime(_time, "%FT%T.%3N") | sort - _time ``` data emulation above ```

Re: Splunk extraction help!

kc_prane — Mon, 19 Aug 2024 15:55:51 GMT

Hi @yuanliu How do i get the difference of the time stamp? . I want the difference of starting timestamp and the completed time stamp

"My base query" | rex "status:\s+(?<Status>.*)\"}" | rex field=_raw "\((?<Message_Id>[^\)]*)" | rex "Path\:\s+(?<ResourcePath>.*)\"" | eval timestamp_s = timestamp/1000 | eval human_readable_time = strftime(timestamp_s, "%Y-%m-%d %H:%M:%S") | transaction Message_Id startswith="Starting execution for request" endswith="Successfully completed execution"

RAW_LOG

8/19/24

9:56:05.113 AM

{"id":"38448254623555555", "timestamp":1724079365113, "message":"(fghhhhhh-244933333-456789-rrrrrrrrrr) Startingexecutionforrequest:f34444-22222-44444-999999-0888888"}

{"id":"38448254444444444", "timestamp":1724079365126, "message":"(fghhhhhh-244933333-456789-rrrrrrrrrr) Methodcompletedwithstatus:200"}

{"id":"38448222222222222", "timestamp":1724079365126, "message":"(fghhhhhh-244933333-456789-rrrrrrrrrr) Successfullycompletedexecution"}

{"id":"38417111111111111", "timestamp":1724079365126, "message":"(fghhhhhh-244933333-456789-rrrrrrrrrr) AWS Integration Endpoint RequestId :f32222-22222-44444-999999-0888888"}

Re: Splunk extraction help!

ITWhisperer — Mon, 19 Aug 2024 17:02:18 GMT

The transaction command provides a duration field for the difference in times. Is this not sufficient for your needs?