topic Re: Visualize splunk in Dashboards & Visualizations

Visualize splunk

hazemfarajallah — Wed, 30 Sep 2020 01:43:43 GMT

hello
i could collect all the data i need from log , but find it so hard how to Visualize them
here is my of my data item (event)
currprocessid: 0f7befb1-7570-4057-81a7-738d3495500a
currprocessname: 02.Timrapportering
currprocesstype: 0
eventId: error
mainprocessid: 0f7befb1-7570-4057-81a7-738d3495500a
mainprocessname: 2.Timrapportering
pageid: 00000000-0000-0000-0000-000000000000
pagename: Main Page
resourceName: HP4E105402
sessionNumber: 2746
sessionid: 547d706e-abad-46de-bd33-4848d6b37e20
stageid: cbbef370-aa9d-4283-8d7c-590a56d5e766
stagename: System Exception
when: 2019-08-14T11:56:52.0946127Z

trying to collect last top 20 item with when , main process name , and currprocesstype ,resourceName any tips how to query them

what i reached source="BP" | fields + c_time, ResourceID, mainprocessname, currprocesstype
| convert timeformat="%H:%M:%S" ctime(_time) AS c_time
| fields - _raw, c_time

Works as table in search but when i get to the dashboard and select limit top, its not working

Re: Visualize splunk

Sukisen1981 — Wed, 14 Aug 2019 15:37:03 GMT

need to provide better data, either you have really hit a bug (as unlikely as it gets..) or you are probably doing something wrong with time ranges / user roles when you save the table as a panel in a dashboard.
Can you elaborate more? There is no way (well, you can never say no so a very very small chance) that what works as a table won't work in a dashboard.

Re: Visualize splunk

hazemfarajallah — Thu, 15 Aug 2019 06:58:27 GMT

Hi thanks for answering, i managed to fix it 😉 . yes, I was doing crazy things until i understood my mistake from trying.

now i just need to learn how to group my data 🙂 .
here is my dashboard look like, i will try to gruop them by mainprocess name
ResourceID mainprocessname currprocesstype _time
HP4E105402 - Create orders 0 2019-08-14 13:54:31
HP4E105402 - Create orders 0 2019-08-14 13:54:31
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14
HP4E105402 NordpoolCheck 0 2019-08-14 13:53:14

Re: Visualize splunk

hazemfarajallah — Thu, 15 Aug 2019 11:09:24 GMT

thanks it worked out was my mistake

Re: Visualize splunk

hazemfarajallah — Thu, 15 Aug 2019 12:28:05 GMT

hello again.
if you kindly look at this query

source="BP"
| eval t = when 
| eval time =strptime(t,"%Y-%m-%d, , %H:%M:%S.%Q%Z") | dedup 1 sessionNumber sortby -time  
| eval Status = case (eventId="endProcess","Completed" ,eventId="error","Terminated")
| eval ResourceID = case (ResourceID="XXX","XX" ,ResourceID="XX","XX")
|stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname,ResourceID,Status | eval DurationSeconds=(endTime - startTime)
| table  startTime,endTime , mainprocessname , ResourceID,Status 
| rename mainprocessname as "Process" , ResourceID as "Runtime resource",startTime as "Start time", endTime as "End time"

and i have all the result i want
but my starttime, endtime is in this formate

2019-08-15T10:42:06.8159144Z
every time i try to convert to date time only i lose the result , i think because its gruoped by
stats .... any way around this to show normal date time

Re: Visualize splunk

richgalloway — Thu, 15 Aug 2019 12:49:29 GMT

To change the format of a timestamp string, use strptime() to convert the string to an integer then use strftime to convert the integer to the desired string format. The two can be combined like this: | eval startTime = strftime( strptime( startTime, "%Y-%m-%dT%H:%M:%S.%7NZ"), "%Y-%m-%d %H:%M:%S%Z").

Re: Visualize splunk

hazemfarajallah — Thu, 15 Aug 2019 13:14:35 GMT

wow Big Thanks ,
its my 3rd day on spunk and i just love it ...

Regards

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 07:20:20 GMT

sorry im back here
|stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname,ResourceID,Status
i cant get real time when i use earliest as start time ..
when= when the process start
any suggistions

here is my query
source="BP"
| eval t = when
| eval time =strptime(t,"%Y-%m-%d, , %H:%M:%S.%Q%Z") | dedup 1 sessionNumber sortby -time

|stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname,ResourceID,Status | eval DurationSeconds=(endTime - startTime) 
| eval startTime = strftime( strptime( startTime, "%Y-%m-%dT%H:%M:%S"), "%Y-%m-%d %H:%M:%S")
| eval endTime = strftime( strptime( endTime, "%Y-%m-%dT%H:%M:%S"), "%Y-%m-%d %H:%M:%S")
| table startTime,endTime , mainprocessname , ResourceID,Status 
| rename mainprocessname as "Process" , ResourceID as "Runtime resource",startTime as "Start time", endTime as "End time"

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 07:23:53 GMT

I am trying to get it in real time.

Re: Visualize splunk

Sukisen1981 — Fri, 16 Aug 2019 07:37:25 GMT

hi @hazemfarajallah

this code in your 3rd line - | eval time =strptime(t,"%Y-%m-%d, , %H:%M:%S.%Q%Z")
gives no output for the field time, replace this with
eval time =strptime(t,"%Y-%m-%dT%H:%M:%S.%7NZ")
after that replace the earliest(when) with earliest(time) in your stats

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 07:49:08 GMT

ok i did this i think my group by is worng |stats earliest(time) AS startTime latest(time) AS endTime by mainprocessname,ResourceID,Status
if i leave as only by mainprocessname then i get the latest but no time stamp ... not sure where is my bug

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 07:50:20 GMT

"source="BP"
| eval t = when 
| eval time =strptime(t,"%Y-%m-%dT%H:%M:%S.%7NZ") | dedup 1 sessionNumber sortby -time  
| eval Status = case (eventId="endProcess","Completed" ,eventId="error","Terminated")
|stats earliest(time) AS startTime latest(time) AS endTime by mainprocessname,ResourceID,Status | eval DurationSeconds=(endTime - startTime) 
| eval startTime = strftime( strptime( startTime, "%Y-%m-%dT%H:%M:%S"), "%Y-%m-%d %H:%M:%S")
| eval endTime = strftime( strptime( endTime, "%Y-%m-%dT%H:%M:%S"), "%Y-%m-%d %H:%M:%S")
| table startTime,endTime , mainprocessname , ResourceID,Status 
| rename mainprocessname as "Process" , ResourceID as "Runtime resource",startTime as "Start time", endTime as "End time""

Re: Visualize splunk

Sukisen1981 — Fri, 16 Aug 2019 07:55:48 GMT

@hazemfarajallah - Hang on a bit 🙂
Firstly, remove all code after your stats and just confirm if the stats output is correct and as per your expectation.
for example i cannot see what is ResourceID, i can see something like resourceName.
even before you check the stats, please check whether each of the eval is giving you the expected result.
You need to debug your code almost line by line, trust me that is easier than just changing a bit of stuff here and there in a large query and hoping that things will work.
Is Status coming out as expected, what happens if you have evetId different from endprocess or error..please verify

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 08:04:20 GMT

''source="BP" |stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname |table startTime,endTime , mainprocessname

this shows this time but its not formated
here is my sample of event .
currprocessid: ae5a9eff-1c7a-48fa-acfa-e2145d06f35f currprocessname: 04 - Ändra fakturasätt currprocesstype: 0 eventId: startDecision mainprocessid: ae5a9eff-1c7a-48fa-acfa-e2145d06f35f mainprocessname: 04 - Ändra fakturasätt pageid: 47e58161-0f42-47cc-aa94-af755a8de010 pagename: Change Invoice Type resourceName: HP20082212 sessionNumber: 3310 sessionid: c557c373-5632-4eb7-a321-f63f73cc1c34 stageid: 33e7422a-4f28-4c41-a408-513d1d704d23 stagename: 9 siffror? when: 2019-08-16T08:03:20.8916931Z

Re: Visualize splunk

Sukisen1981 — Fri, 16 Aug 2019 08:09:52 GMT

where is resourceid that you are using in your stats command in the above event?
And you will have to reformat the time back
|eval startTime =strftime(startTime ,"%Y-%m-%dT%H:%M:%S.%7NZ") |eval endTime=strftime(endTime,"%Y-%m-%dT%H:%M:%S.%7NZ")
Once again, what is resourceid and what about the status field like i asked before

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 08:36:39 GMT

Status I got from converting the event id eval Status = case (eventId="endProcess","Completed" ,eventId="error","Terminated")
and resourceId should use resourceName (MY big mistake)
I did what you said line by line 🙂 BIG thanks
now everything in a place you was right
as source="BP" | eval Status = case (eventId="endProcess","Completed" ,eventId="error","Terminated") |stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname,Status,resourceName |eval startTime = strftime( strptime( startTime, "%Y-%m-%dT%H:%M:%S.%7NZ"), "%Y-%m-%d %H:%M:%S") |eval endTime = strftime( strptime( endTime, "%Y-%m-%dT%H:%M:%S.%7NZ"), "%Y-%m-%d %H:%M:%S") |table startTime,endTime , mainprocessname,Status, resourceName
`
All in a place 🙂

Re: Visualize splunk

hazemfarajallah — Fri, 16 Aug 2019 09:19:54 GMT

sorry, i posted as answer,
my resourceid now resouceName , status is from eval my event id, but having problem with the group by due i cant show the status in the table
`source="BP"
|eval Status = case (eventId="endProcess","Completed" ,eventId="error","Terminated")
|stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname
|eval startTime = strftime( strptime( startTime, "%Y-%m-%dT%H:%M:%S.%7NZ"), "%Y-%m-%d %H:%M:%S")
|eval endTime = strftime( strptime( endTime, "%Y-%m-%dT%H:%M:%S.%7NZ"), "%Y-%m-%d %H:%M:%S")

|table startTime, endTime , mainprocessname,Status`

Re: Visualize splunk

Sukisen1981 — Fri, 16 Aug 2019 14:40:26 GMT

hi @hazemfarajallah
You can not see the status in your table because you are only doing a stats by mainprocessname.
Now, if you add status to the stats like | stats earliest(when) AS startTime latest(when) AS endTime by mainprocessname,status what output do you get?
Once again, remove all code after the stats and first verify that the stats is correct. it should give you output by mainprocess and status if the stats is correct

Re: Visualize splunk

Sukisen1981 — Mon, 19 Aug 2019 14:57:34 GMT

hi @hazemfarajallah
Please let us know if your issue is solved or do you still need some more help on this?
If you issue is solved, please let me know if I can convert the comment into an answer for your acceptance

Re: Visualize splunk

hazemfarajallah — Tue, 20 Aug 2019 06:56:14 GMT

Hi @suskisen
Very big thanks, I started to follow your idea line by line and fixed almost all th problem
thanks