Splunk Search

Somehow unable to fetch Time , Earlier it was working

bhaskar5428
Explorer

bhaskar5428_0-1712206774586.png

===========================================

Query used 
index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*"
|fields message
|rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})"
|rex field=_raw "sourceSystem=(?<Source>[^,]*)"
|rex field=_raw "entityType=(?<Entity>\w+)"
|rex field=_raw "\"timestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  -- this is not working
|sort Time desc
|dedup Entity
|table Source, BusDate, Entity, Time

===========================================

This how raw data looks like

bhaskar5428_1-1712206962603.png

i  would like to extract only time , also suggest how can i covert to AM/PM


 

Kindly provide solution.

 

 

Labels (1)
Tags (1)
0 Karma

bhaskar5428
Explorer

bhaskar5428_0-1712212684375.png

please check the sample raw data , where i need time only

0 Karma

bhaskar5428
Explorer

Unable to understand solution , could you please elaborate more 

 

I see in raw data as below
eventTimestamp=2024-04-04T02:24:52.762129638)

 

i would like extract time from above like = 02:24 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Your command says

"\"timestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})" 

So it will match only if you have a part of your event containing (of course the timestamp is just an example)

"timestamp":"2023-01-12T14:54

Since your event is formatted differently (most significantly, the "field" you're extracting from is not named "timestamp"), you need to adjust this regex.

Use https://regex101.com for checking/verifying your ideas.

As a side note - manipulating structured data (in your case - json) with regexes might not be the best idea.

0 Karma

bhaskar5428
Explorer

Yes, i released that its not "timestamp " and its changes to "eventTimestamp" in raw data 
However  modified query but still its not working.

======================================================================

index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*"
|fields message
|rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})"
|rex field=_raw "sourceSystem=(?<Source>[^,]*)"
|rex field=_raw "entityType=(?<Entity>\w+)"
|rex field=_raw "\"eventTimestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  --> Please suggest here 
|sort Time desc
|dedup Entity
|table Source, BusDate, Entity, Time

======================================================================

attaching sample raw screenshot for your reference

bhaskar5428_0-1712213808126.png

 

 

0 Karma

scelikok
SplunkTrust
SplunkTrust

Please try below, 

|rex field=_raw "\"eventTimestamp\"=\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})" 

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

bhaskar5428
Explorer

My apologies

i was using "eventTimestamp" instead of  "@timestamp" in my rex command 

i just realized and its working now , However i do not need date in last column need only time.
Please help how to do that.

please find below details 

================================================================================

Query

index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*"
|fields message
|rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})"
|rex field=_raw "sourceSystem=(?<Source>[^,]*)"
|rex field=_raw "entityType=(?<Entity>\w+)"
|rex field=_raw "\"@timestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  --> Please help Here 
|sort Time desc
|dedup Entity
|table Source, BusDate, Entity, Time

================================================================================

Screenshot

bhaskar5428_0-1712226050758.png

 

-------------------------------------------------------------------------------------------

raw data

{"@timestamp":"2024-04-04T02:25:59.366Z","level":"INFO","message":"Snapshot event published: SnapshotEvent(version=SnapshotVersion(sourceSystem=dbI-LDN, entityType=ACCOUNT, subType=, date=2024-04-03, version=1, snapshotSize=326718, uuid=8739e273-cedc-482b-b696-48357efc8704, eventTimestamp=2024-04-04T02:24:52.762129638), status=CREATED)","thread":"snapshot-checker-3","loggerName":"com.db.sdda.dc.kafka.snapshot.writer.InternalEventSender"}

Show syntax highlighted

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @bhaskar5428 ,

You need to change the regex capture group to cover only time, like below;

| rex field=_raw "\"@timestamp\":\"\d{4}-\d{2}-\d{2}[T](?<Time>\d{2}:\d{2})"

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

bhaskar5428
Explorer

My apologies

i was using "eventTimestamp" instead of  "@timestamp" in my rex command 

i just realized and its working now , However i do not need date in last column need only time.
Please help how to do that.

please find below details 

================================================================================

Query

index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*"
|fields message
|rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})"
|rex field=_raw "sourceSystem=(?<Source>[^,]*)"
|rex field=_raw "entityType=(?<Entity>\w+)"
|rex field=_raw "\"@timestamp\":\"(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"  --> Please help Here 
|sort Time desc
|dedup Entity
|table Source, BusDate, Entity, Time

================================================================================

Screenshot

bhaskar5428_0-1712225910578.png

-------------------------------------------------------------------------------------------

raw data

{"@timestamp":"2024-04-04T02:25:59.366Z","level":"INFO","message":"Snapshot event published: SnapshotEvent(version=SnapshotVersion(sourceSystem=dbI-LDN, entityType=ACCOUNT, subType=, date=2024-04-03, version=1, snapshotSize=326718, uuid=8739e273-cedc-482b-b696-48357efc8704, eventTimestamp=2024-04-04T02:24:52.762129638), status=CREATED)","thread":"snapshot-checker-3","loggerName":"com.db.sdda.dc.kafka.snapshot.writer.InternalEventSender"}

Show syntax highlighted

Tags (1)
0 Karma

bhaskar5428
Explorer

Still not working i replaced semicolon with "=" sign
Please check screenshot.

bhaskar5428_0-1712220600122.png

=============================================================================

Sample raw data

bhaskar5428_1-1712220639007.png

 

0 Karma

scelikok
SplunkTrust
SplunkTrust
|rex field=_raw "eventTimestamp=(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})" 
If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Watch your raw event carefully. Compare it with the regex. The difference is kinda obvious.

0 Karma

bhaskar5428
Explorer

Also please check below query which is working , however it does not giving me required output , i need only time. in Last column

===============================================================

index=* namespace="dk1017-j" sourcetype="kube:container:kafka-clickhouse-snapshot-writer" message="*Snapshot event published*" AND message="*dbI-LDN*" AND message="*2024-04-03*" AND message="*"
|fields message
|rex field=_raw "\s+date=(?<BusDate>\d{4}-\d{2}-\d{2})"
|rex field=_raw "sourceSystem=(?<Source>[^,]*)"
|rex field=_raw "entityType=(?<Entity>\w+)"
|rex field=_raw "eventTimestamp=(?<Time>\d{4}-\d{2}-\d{2}[T]\d{2}:\d{2})"   --> Need only time 
|sort Time desc
|dedup Entity
|table Source, BusDate, Entity, Time

================================================================

 

please check screenshot for more clear understanding

bhaskar5428_0-1712212072604.png

 

 

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @bhaskar5428,

Your rex command seems trying to extract Time field from @timestamp field. Can you please show the raw data by clicking "Show as raw text" selection under the raw event?

Splunk shows JSON events as formatted but rex works on real text itself.  We cannot compare your regex and raw data using this  screen capture.

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...

Manual Instrumentation with Splunk Observability Cloud: How to Instrument Frontend ...

Although it might seem daunting, as we’ve seen in this series, manual instrumentation can be straightforward ...