Dashboards & Visualizations

unable to solve "mvexpand: output will be truncated due to excessive memory usage."

saikumar009
Loves-to-Learn

Hello,
I am running into an issue with some spath and mvexpand functions in splunk. I get the following error: "output will be truncated at 3700 results due to excessive memory usage."

after searching here few previous answer worked. However it is not working out for me 

Here is my search 

index=ehub-loop
|rex "(?:((?:\[BEGIN LOGGING AT (?<Event_Timestamp>.*)\]\n)?)((?:(?P<Event_log_entry>(?s).*)\n)?)((?:\[END LOGGING])?))" offset_field=_extracted_fields_bounds
| rex field=Event_log_entry max_match=0 "^(?<single_log_entry>.+)\n*" offset_field=_extracted_fields_bounds
| mvexpand single_log_entry
| rex field=single_log_entry "(?P<log_Timestamp>\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3})\s+(?P<log_level>[^ ]+)\s+\[(?P<Thread_Number>[^ ]+)\]\s+(?P<Class_Name>[^ ]+)\s+\-\s+(?P<log_msg>(?s).*)" offset_field=_extracted_fields_bounds
|stats count(Class_Name) as Error_Count by Class_Name,log_level,log_msg

Each event will be like

[BEGIN LOGGING AT 2021-05-20 21:00:12,505]
2021-05-21 12:09:40,460 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,476 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,507 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,507 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,522 Loglevel [Threadid] Classname - logmsg
[END LOGGING]

Please help me out

Labels (1)
Tags (1)
0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@saikumar009 

Can you please try this?

index=ehub-loop
| rex "(?:((?:\[BEGIN LOGGING AT (?<Event_Timestamp>.*)\]\n)?)((?:(?P<Event_log_entry>(?s).*)\n)?)((?:\[END LOGGING])?))" offset_field=_extracted_fields_bounds
| rex field=Event_log_entry max_match=0 "^(?<single_log_entry>.+)\n*" offset_field=_extracted_fields_bounds
| stats count by single_log_entry 
| rex field=single_log_entry "(?P<log_Timestamp>\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3})\s+(?P<log_level>[^ ]+)\s+\[(?P<Thread_Number>[^ ]+)\]\s+(?P<Class_Name>[^ ]+)\s+\-\s+(?P<log_msg>(?s).*)" offset_field=_extracted_fields_bounds
| stats sum(count) as Error_Count by Class_Name,log_level,log_msg

 

My Sample Search :

| makeresults | eval _raw="[BEGIN LOGGING AT 2021-05-20 21:00:12,505]
2021-05-21 12:09:40,460 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,476 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,507 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,507 Loglevel [Threadid] Classname - logmsg
2021-05-21 12:09:40,522 Loglevel [Threadid] Classname - logmsg
[END LOGGING]
" | multikv 
| rex "(?:((?:\[BEGIN LOGGING AT (?<Event_Timestamp>.*)\]\n)?)((?:(?P<Event_log_entry>(?s).*)\n)?)((?:\[END LOGGING])?))" offset_field=_extracted_fields_bounds
| rex field=Event_log_entry max_match=0 "^(?<single_log_entry>.+)\n*" offset_field=_extracted_fields_bounds
| stats count by single_log_entry 
| rex field=single_log_entry "(?P<log_Timestamp>\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3})\s+(?P<log_level>[^ ]+)\s+\[(?P<Thread_Number>[^ ]+)\]\s+(?P<Class_Name>[^ ]+)\s+\-\s+(?P<log_msg>(?s).*)" offset_field=_extracted_fields_bounds
| stats sum(count) as Error_Count by Class_Name,log_level,log_msg

 

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...