All Apps and Add-ons

How to get max memory used value in each message and create time chart to show max memory used value and average?

hungln9
Explorer

Hi Teams,

I am newbie to splunk, I have log message like this:

4/5/22
6:03:22.697 PM
 
2022-04-05T10:03:22.697Z 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO: Insert batch 0/6 END RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 REPORT RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 Duration: 601.44 ms Billed Duration: 602 ms Memory Size: 1024 MB Max Memory Used: 97 MB

 

I want to get Max Memory Used value in each message and create time chart to show Max Memory Used value and the Max Memory Used average value. Can anyone help me in this!

Labels (1)
0 Karma
1 Solution

gcusello
Esteemed Legend

Hi @hungln9,

in general, you have to find a rule to apply the regex: if in your case you want the string after INFo, you could use a regex like this:

| rex "INFO\s+(?<instance>[^ ]+)"

Tell me if I can help you more, otherwise, please, accept my answer for the other people of Community.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

View solution in original post

hungln9
Explorer

Many thanks @gcusello ,

That is exactly what I need.

Can I ask you 1 more things?

I want to include the instances infomation to the chart, in this message is  INFO 802cf235-b8d6-454e-bb1a-25d16f6b5f21, can you guide me:

4/5/22
6:03:22.697 PM
 
2022-04-05T10:03:22.697Z 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO: Insert batch 0/6 END RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 REPORT RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 Duration: 601.44 ms Billed Duration: 602 ms Memory Size: 1024 MB Max Memory Used: 97 MB
0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

in general, you have to find a rule to apply the regex: if in your case you want the string after INFo, you could use a regex like this:

| rex "INFO\s+(?<instance>[^ ]+)"

Tell me if I can help you more, otherwise, please, accept my answer for the other people of Community.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

hungln9
Explorer

Thanks for your support, @gcusello 

I mean that I have a lot of instances(instance1, instance2....), and I want to show all of them on only 1 time chart, can you tell me, how can I do that?

0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

it mainly depends on how many instances you have to display.

There a limit in chart but it's very high (hundreds of bars), the main limit is the readability of your chart.

maybe you could create more panels displaying a group of instances.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

hungln9
Explorer

Hi @gcusello ,

I have many sources in search result, can you guide me how can I group some resource into 1 chart?

index=my_index*
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used by source

hungln9_0-1649841124208.png

I tried to group with this query, but seem it's incorrect:

index=my_index* (source=source1 or source=2)
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used by source

0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

it's always better to ask a new question to the Community, so more people can help you better and quicker!

Anyway, using the second search, you use the same grouping options than the first but you filter your results  (in teh main search) taking only events from two sources.

if it doesn't run. check the "(source=source1 OR source=source2)" condition and check if the regex you used is correct for those events..

Anyway, probably host could be more interesting than source.

Ciao.

Giuseppe

0 Karma

hungln9
Explorer

Many thanks @gcusello ,

I have other question in Comunity, if you are free please help me take a look on that:

https://community.splunk.com/t5/All-Apps-and-Add-ons/How-to-caculate-percentage-of-memory-used-value...

0 Karma

hungln9
Explorer

Many thanks @gcusello ,

I want to create send email alert when "max memory used" is greater than 1024.

I'm trying to save search results as alert, but can not find the way to set or define this condition.

hungln9_0-1649229629561.png

Can you please guide me?

0 Karma

gcusello
Esteemed Legend

Hi @hungln9.,

put this condition at the end of your search and set your alert to trigger when results>0:

index=your_index
| rex "INFO\s+(?<instance>[^ ]+)"
| timechart max(max_memory_used) AS max_memory_used
| where max_memory_used>1024

Ciao.

Giuseppe

hungln9
Explorer

Thanks for your support @gcusello 

I tried to add condition, but seem it not work, even I tried with value=10

index=my_index*
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used(MB)
| rex "INFO Done Lamda function\s+(?<D365>[^ ]+)"
|where max_memory_used>10

hungln9_0-1649236181382.png

While without condition, It worked

index=my_index*
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used(MB)
| rex "INFO Done Lamda function\s+(?<D365>[^ ]+)"

hungln9_1-1649236350108.png

Can you pls take a look on this!

0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

in the timechart, you renamed "max_memory_used(MB)" whilein the where condition you used "max_memory_used" that's different!

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

hungln9
Explorer

Hi @gcusello ,

Sorry for many question from me.

I have new trouble

I already created alert to send notify email to me once the max memory used is over. But I recevied a lot email notify in 1 minute, once alert was  triggered. I just want to trigger action send me 1 or some notify, Could you please guide me?

hungln9_0-1649389038166.png

hungln9_1-1649389695429.png

 

0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

no problems for your questions!

Anyway, in this case you have to configure throttle in your alert, the period, after a triggering that the alert doesn't run.

You can find it in the Alert definition.

Ciao.

Giuseppe

0 Karma

hungln9
Explorer

Many thanks @gcusello , 

This is thing, what I need

0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

you're always welcome.

Ciao and happy splunking.

Giuseppe

0 Karma

hungln9
Explorer

Many thanks @gcusello , 

My problem was solved, no issue at all.

0 Karma

gcusello
Esteemed Legend

Hi @hungln9,

good for you, see next time!

Ciao and happy splunking

iuseppe

P.S.: Karma Points are appreciated 😉

gcusello
Esteemed Legend

hi @hungln9,

if you're sure that the maxmemory is always expressed in MB, you could run something like this:

index=your_index
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used

if instead you could also have GB, you should modify a little the search:

index=your_index
| rex "Max Memory Used: (?<max_memory_used>\d+)\s+(?<mem_unit>\w+)"
| eval max_memory_used=if(mem_unit="GB",max_memory_used*1024,max_memory_used)
| timechart max(max_memory_used) AS max_memory_used

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...