topic Re: Parsing Splunk log in All Apps and Add-ons

How to get max memory used value in each message and create time chart to show max memory used value and average?

hungln9 — Wed, 13 Apr 2022 14:54:03 GMT

Hi Teams,

I am newbie to splunk, I have log message like this:

4/5/22
6:03:22.697 PM

2022-04-05T10:03:22.697Z 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO: Insert batch 0/6 END RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 REPORT RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 Duration: 601.44 ms Billed Duration: 602 ms Memory Size: 1024 MB Max Memory Used: 97 MB

I want to get Max Memory Used value in each message and create time chart to show Max Memory Used value and the Max Memory Used average value. Can anyone help me in this!

Re: Parsing Splunk log

gcusello — Tue, 05 Apr 2022 10:22:15 GMT

hi @hungln9,

if you're sure that the maxmemory is always expressed in MB, you could run something like this:

index=your_index | rex "Max Memory Used: (?<max_memory_used>\d+)" | timechart max(max_memory_used) AS max_memory_used

if instead you could also have GB, you should modify a little the search:

index=your_index | rex "Max Memory Used: (?<max_memory_used>\d+)\s+(?<mem_unit>\w+)" | eval max_memory_used=if(mem_unit="GB",max_memory_used*1024,max_memory_used) | timechart max(max_memory_used) AS max_memory_used

Ciao.

Giuseppe

Re: Parsing Splunk log

hungln9 — Tue, 05 Apr 2022 11:04:15 GMT

Many thanks @gcusello ,

That is exactly what I need.

Can I ask you 1 more things?

I want to include the instances infomation to the chart, in this message is INFO 802cf235-b8d6-454e-bb1a-25d16f6b5f21, can you guide me:

4/5/22
6:03:22.697 PM

2022-04-05T10:03:22.697Z 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO 802cf235-b8d6-454e-bb1a-25d16f6b5f21 INFO: Insert batch 0/6 END RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 REPORT RequestId: 802cf235-b8d6-454e-bb1a-25d16f6b5f21 Duration: 601.44 ms Billed Duration: 602 ms Memory Size: 1024 MB Max Memory Used: 97 MB

Re: Parsing Splunk log

gcusello — Tue, 05 Apr 2022 12:05:40 GMT

Hi @hungln9,

in general, you have to find a rule to apply the regex: if in your case you want the string after INFo, you could use a regex like this:

| rex "INFO\s+(?<instance>[^ ]+)"

Tell me if I can help you more, otherwise, please, accept my answer for the other people of Community.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

Re: Parsing Splunk log

gcusello — Tue, 05 Apr 2022 13:36:23 GMT

Hi @hungln9,

good for you, see next time!

Ciao and happy splunking

iuseppe

P.S.: Karma Points are appreciated 😉

Re: Parsing Splunk log

hungln9 — Wed, 06 Apr 2022 03:34:11 GMT

Thanks for your support, @gcusello

I mean that I have a lot of instances(instance1, instance2....), and I want to show all of them on only 1 time chart, can you tell me, how can I do that?

Re: Parsing Splunk log

gcusello — Wed, 06 Apr 2022 06:21:50 GMT

Hi @hungln9,

it mainly depends on how many instances you have to display.

There a limit in chart but it's very high (hundreds of bars), the main limit is the readability of your chart.

maybe you could create more panels displaying a group of instances.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

Re: Parsing Splunk log

hungln9 — Wed, 06 Apr 2022 07:21:10 GMT

Many thanks @gcusello ,

I want to create send email alert when "max memory used" is greater than 1024.

I'm trying to save search results as alert, but can not find the way to set or define this condition.

Can you please guide me?

Re: Parsing Splunk log

gcusello — Wed, 06 Apr 2022 07:45:18 GMT

Hi @hungln9.,

put this condition at the end of your search and set your alert to trigger when results>0:

index=your_index | rex "INFO\s+(?<instance>[^ ]+)" | timechart max(max_memory_used) AS max_memory_used | where max_memory_used>1024

Ciao.

Giuseppe

Re: Parsing Splunk log

hungln9 — Wed, 06 Apr 2022 09:13:52 GMT

Thanks for your support @gcusello

I tried to add condition, but seem it not work, even I tried with value=10

index=my_index*
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used(MB)
| rex "INFO Done Lamda function\s+(?<D365>[^ ]+)"
|where max_memory_used>10

While without condition, It worked

index=my_index*
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used(MB)
| rex "INFO Done Lamda function\s+(?<D365>[^ ]+)"

Can you pls take a look on this!

Re: Parsing Splunk log

gcusello — Wed, 06 Apr 2022 09:35:51 GMT

Hi @hungln9,

in the timechart, you renamed "max_memory_used(MB)" whilein the where condition you used "max_memory_used" that's different!

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

Re: Parsing Splunk log

hungln9 — Thu, 07 Apr 2022 03:05:02 GMT

Many thanks @gcusello ,

My problem was solved, no issue at all.

Re: Parsing Splunk log

hungln9 — Fri, 08 Apr 2022 03:48:30 GMT

Hi @gcusello ,

Sorry for many question from me.

I have new trouble

I already created alert to send notify email to me once the max memory used is over. But I recevied a lot email notify in 1 minute, once alert was triggered. I just want to trigger action send me 1 or some notify, Could you please guide me?

Re: Parsing Splunk log

gcusello — Fri, 08 Apr 2022 06:18:20 GMT

Hi @hungln9,

no problems for your questions!

Anyway, in this case you have to configure throttle in your alert, the period, after a triggering that the alert doesn't run.

You can find it in the Alert definition.

Ciao.

Giuseppe

Re: Parsing Splunk log

hungln9 — Wed, 13 Apr 2022 06:26:42 GMT

Many thanks @gcusello ,

This is thing, what I need

Re: Parsing Splunk log

gcusello — Wed, 13 Apr 2022 07:25:30 GMT

Hi @hungln9,

you're always welcome.

Ciao and happy splunking.

Giuseppe

Re: Parsing Splunk log

hungln9 — Wed, 13 Apr 2022 09:14:44 GMT

Hi @gcusello ,

I have many sources in search result, can you guide me how can I group some resource into 1 chart?

index=my_index*
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used by source

I tried to group with this query, but seem it's incorrect:

index=my_index* (source=source1 or source=2)
| rex "Max Memory Used: (?<max_memory_used>\d+)"
| timechart max(max_memory_used) AS max_memory_used by source

Re: Parsing Splunk log

gcusello — Wed, 13 Apr 2022 10:07:32 GMT

Hi @hungln9,

it's always better to ask a new question to the Community, so more people can help you better and quicker!

Anyway, using the second search, you use the same grouping options than the first but you filter your results (in teh main search) taking only events from two sources.

if it doesn't run. check the "(source=source1 OR source=source2)" condition and check if the regex you used is correct for those events..

Anyway, probably host could be more interesting than source.

Ciao.

Giuseppe

Re: Parsing Splunk log

hungln9 — Fri, 15 Apr 2022 03:22:48 GMT

Many thanks @gcusello ,

I have other question in Comunity, if you are free please help me take a look on that:

https://community.splunk.com/t5/All-Apps-and-Add-ons/How-to-caculate-percentage-of-memory-used-value-in-each-message/m-p/593854#M76611