Splunk Search

Multi-conditional summation of time

mjones414
Contributor

Sample Data:
09/12/2017 23:58:35;E;957690.hostname user=NameHere group=GroupHere project=_pbs_project_default jobname=SomeNameHere queue=SomeQueueNameHere ctime=1505271803 qtime=1505271803 etime=1505271803 start=1505272298 exec_host=NodeName/0*24 Resource_List.walltime=720:00:00 session=22656 end=1505278715 Exit_status=0 resources_used.cpupercent=2398 resources_used.cput=40:19:50 resources_used.mem=58593416kb resources_used.ncpus=24 resources_used.vmem=80526996kb resources_used.walltime=01:46:56 run_count=1

This is a sample event of some data that I need to abstract total runtime in hours over 90 days worth of events. There are some qualifying conditions that will change the formula needed, but each event may or may not fit the condition and there are almost a million of events in a 90 day period of time.

The basic result I'm looking for is in the field I'm creating below, PBSAWallTime. The eval for this field can be found below.. However, there are conditions where this isn't doable. When resources_used.walltime :
- List item

is a negative number
is > (end-start)
is not present

Then I need to use the runtime eval field I created as the correct time field for the given event. I haven't been able to form the correct if eval syntax which would cover all three scenarios and am hoping someone can help me out?

Current Search Parameters:

sourcetype=pbs_accounting host=ServerName E 
| eval runtime=end-start 
| convert dur2sec(resources_used_walltime) as resources_used_wallseconds 
| eval PWR=resources_used_wallseconds/(end-start)
| eval PBSAWallTime=(end-start)*PWR
Tags (3)
0 Karma

mjones414
Contributor

Unfortunately that didn't quite get it, but I had to modify it a little as it didn't account for the Ratio. Not sure what I am missing...

convert dur2sec(resources_used_walltime) as resources_used_wallseconds | eval PWR=resources_used_wallseconds/(end-start) | eval PBSAWallTime=(end-start)*PWR | eval PBSAWallTotal=if(isnull(resources_used_walltime) OR resources_used_wallseconds<0 OR resources_used_wallseconds>runtime, runtime, PBSAWallTime)

0 Karma

somesoni2
Revered Legend

Give this a try

sourcetype=pbs_accounting host=ServerName E 
 | eval runtime=end-start 
 | convert dur2sec(resources_used_walltime) as resources_used_wallseconds 
 | eval PBSAWallTime=if(isnull(resources_used_walltime) OR resources_used_wallseconds<0 OR resources_used_wallseconds>runtime, runtime, resources_used_wallseconds)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: The Agentic SOC: Trust Before Autonomy

AI is fundamentally changing security operations, but true progress requires more than just automation—it ...

Free Professional Services for .conf26 Attendees

This year at .conf26, we are doing something a little different. We are bringing the best minds from ...

Defend at Machine Speed: Your Guide to Security Sessions at .conf26

Splunk .conf26   With threats moving at machine speed and attack surfaces expanding across hybrid ...