Monitoring Splunk

Splunk reporting running out of memory on a VM server wtih 16Gig

bcarlson
New Member

Good Afternoon! I am trying to create a report that goes through a 15 Million record file and
creates a cost of roaming report based on my Users roaming on a different network. A User could have records on multiple other wireless networks. The report calculates cost based on partner's rates and data consumed by User. This report works perfect if I try and run the report on a smaller number of records. The search portion of the report is pulling stop records that have accessed Bobwireless.com. Is there a better more memory efficient way to find the same information?

thanks
Bob

Domain="Bobwireless.com" AcctType="2" | eval Roamer_Cost=case(Serving_Carrier=="JillWireless", Total_Megabytes*.055, Serving_Carrier=="Larry Wireless", Total_Megabytes*.10, Serving_Carrier=="Cowboy", Total_Megabytes*.25, Serving_Carrier=="Indains", Total_Megabytes*.40, Serving_Carrier=="KCChiefs", Total_Megabytes*.40, Serving_Carrier=="Raiders:, Total_Megabytes*.0, Serving_Carrier=="Panthers", Total_Megabytes*.40, Serving_Carrier=="Chargers", Total_Megabytes*.20, Serving_Carrier=="CellComm", Total_Megabytes*.20, Serving_Carrier=="Vikings", Total_Megabytes*.10, Serving_Carrier=="Bears", Total_Megabytes*.25, Serving_Carrier=="Cardinals", Total_Megabytes*.25, Serving_Carrier=="Jaguars",Total_Megabytes*.40, Serving_Carrier=="Oilers", Total_Megabytes*.35, Serving_Carrier=="Titans", Total_Megabytes*.25, Serving_Carrier=="Dolphins", Total_Megabytes*.35, Serving_Carrier=="Packers", Total_Megabytes*.25, Serving_Carrier=="Patriots", Total_Megabytes*.25, Serving_Carrier=="Bucaneers", Total_Megabytes*.40, Serving_Carrier=="Ravens", Total_Megabytes*.35) | table User, Serving_Carrier, Total_Megabytes, Roamer_Cost

Tags (2)
0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

You could try a lookup table.

roaming_weights.csv
Serving_Carrier,weight
JillWireless,0.055
Larry Wireless,0.10

And then your search:

Domain="Bobwireless.com" AcctType="2" | lookup roaming_weights.csv Serving_Carrier | eval Roamer_Cost = weight * Total_Megabytes | table User Serving_Carrier Total_Megabytes Roamer_Cost

If you need more carriers, just add them to the CSV file.

bcarlson
New Member

I entered a system ticket to see what Splunk support says.
thanks
Bob

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Intersting, I see that error on 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux. The Kernel Builds are the same....

0 Karma

bcarlson
New Member

2.6.32-358.2.1.e16.x86_64 #1 SMP Wed Mar 12 00:26:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

0 Karma

bcarlson
New Member

I will see if I can find that? thanks

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

What OS? if linux what does uname -a report?

0 Karma

bcarlson
New Member

alacercogitatusitatus,

Got that problem solved. It was the $ sign in the CSV field for "Weight". Your search suggests seem to work much better because Splunk is not blowing up with memory errors, but it is still ending with "[SimpleResultsTable module] Splunkd daemon is not responding: ('The read operation timed out',) I wonder if anyone has an idea on that?

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Could you paste the query just as you are running it?

0 Karma

bcarlson
New Member

alacercogitatus, Good Morning! Thanks for you help! I built the CSV table and everything seems to work accept the calculation weight*Total_Megabytes. When I look at the data in the fields, I see vaild weight values. Any ideas? thanks Bob

0 Karma

bmacias84
Champion

Also before doing any eval or computational commands you should distille to your only the fields required by using the fields command. This will increase performance by only return necessary fields.

Domain="Bobwireless.com" AcctType="2"| fields User, Servicing_Carrier, Total_Megabytes | ...

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...