Playbook format block not using filtered data

michael3 — Mon, 25 Mar 2024 14:25:21 GMT

I'm using the Cisco FireAMP app to return the trajectory of an endpoint, and the data includes a list of all running tasks/files. For my test there are 500 items returned, with 9 marked as 'Malicious'. I'm trying to filter for those and write the details to a note. But the note always contains all 500 items, not just the 9.

My filter block (filter_2) is this:

if get_device_trajectory_2:action_result.data.*.events.*.file.disposition == Malicious

My format block (format_3) is this:

%% File Name: {0} - File Path: {1} - Hash: {2} - Category: {4} - Parent: {3} %%

where each of the variables refer to the filter block e.g.:

0: filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.file_name 1: filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.file_path 2: filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.identity.sha256 3: filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.parent.file_name 4: filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.detection

Finally, I use a Utility block to add the note. The Utility block contents reference the format block:

format_3:formatted_data.*

The debugger shows this when running the filter block:

Mar 25, 13:52:54 : filter_2() called Mar 25, 13:52:54 : phantom.condition(): called with 1 condition(s) '[['get_device_trajectory_2:action_result.data.*.events.*.file.disposition', '==', 'Malicious']]', operator : 'or', scope: 'new' Mar 25, 13:52:54 : phantom.get_action_results() called for action name: get_device_trajectory_2 action run id: 0 app_run_id: 0 Mar 25, 13:52:54 : phantom.condition(): condition 1 to evaluate: LHS: get_device_trajectory_2:action_result.data.*.events.*.file.disposition OPERATOR: == RHS: Malicious Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'None' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'None' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'None' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'None' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'None' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False Mar 25, 13:52:54 : phantom.condition(): condition loop: condition 1, 'Clean' '==' 'Malicious' => result:False Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Malicious' '==' 'Malicious' => result:True Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'None' '==' 'Malicious' => result:False Mar 25, 13:52:55 : phantom.condition(): condition loop: condition 1, 'Unknown' '==' 'Malicious' => result:False

so it looks like it's correctly identifying the malicious files. The debugger shows this when running the format block:

Mar 25, 13:52:55 : format_3() called Mar 25, 13:52:55 : phantom.collect2(): called for datapath['filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.file_name'], scope: new and filter_artifacts: [] Mar 25, 13:52:55 : phantom.get_run_data() called for key filtered-data:filter_2:condition_1 Mar 25, 13:52:55 : phantom.collect2(): Classified datapaths as [<DatapathClassification.NAMED_FILTERED_ACTION_RESULT: 9>] Mar 25, 13:52:55 : phantom.collect2(): called for datapath['filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.file_path'], scope: new and filter_artifacts: [] Mar 25, 13:52:55 : phantom.get_run_data() called for key filtered-data:filter_2:condition_1 Mar 25, 13:52:55 : phantom.collect2(): Classified datapaths as [<DatapathClassification.NAMED_FILTERED_ACTION_RESULT: 9>] Mar 25, 13:52:55 : phantom.collect2(): called for datapath['filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.identity.sha256'], scope: new and filter_artifacts: [] Mar 25, 13:52:55 : phantom.get_run_data() called for key filtered-data:filter_2:condition_1 Mar 25, 13:52:55 : phantom.collect2(): Classified datapaths as [<DatapathClassification.NAMED_FILTERED_ACTION_RESULT: 9>] Mar 25, 13:52:55 : phantom.collect2(): called for datapath['filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.file.parent.file_name'], scope: new and filter_artifacts: [] Mar 25, 13:52:55 : phantom.get_run_data() called for key filtered-data:filter_2:condition_1 Mar 25, 13:52:55 : phantom.collect2(): Classified datapaths as [<DatapathClassification.NAMED_FILTERED_ACTION_RESULT: 9>] Mar 25, 13:52:55 : phantom.collect2(): called for datapath['filtered-data:filter_2:condition_1:get_device_trajectory_2:action_result.data.*.events.*.detection'], scope: new and filter_artifacts: [] Mar 25, 13:52:55 : phantom.get_run_data() called for key filtered-data:filter_2:condition_1 Mar 25, 13:52:56 : phantom.collect2(): Classified datapaths as [<DatapathClassification.NAMED_FILTERED_ACTION_RESULT: 9>] Mar 25, 13:52:56 : save_run_data() saving 136.29 KB with key format_3:formatted_data_ Mar 25, 13:52:56 : save_run_data() saving 140.23 KB with key format_3__as_list:formatted_data_

there are 9 malicious files and it looks like that's what it's saying in the debugger, so again it seems like it's using the filtered data correctly.

But my note always has 500 items in it. I'm not sure what I'm doing wrong. Can anyone offer any help, because I'm stuck. Thanks.

Re: Playbook format block not using filtered data

michael3 — Mon, 25 Mar 2024 14:58:24 GMT

Quick update. I changed the format block to use format_3:formatted_data instead of formatted_data.*. The note looks a lot nicer, but it's still 500 items.

Re: Playbook format block not using filtered data

SOARt_of_Lost — Tue, 26 Mar 2024 16:28:16 GMT

Working with just this example, the same applies across the board.

get_device_trajectory_2:action_result.data.*.events.*.file.parent.file_name

.data.*.events.*. is most likely your problem. Every time your filter block hits a true, you're telling your format block to pull in all of the file names in the event data from get_device_trajectory_2. You'll need to find a way to tell it to only pull in the information from the index of the item you care about. Something like

get_device_trajectory_2:action_result.data.*.events.X.file.parent.file_name

where X is the item in the list that evaluated true.

Re: Playbook format block not using filtered data

michael3 — Wed, 27 Mar 2024 17:08:32 GMT

Yes, this was it. The filter wasn't able to deal with the multiple levels in my data.

I ended up replacing the filter with a code block that ran the same conditional statement and saved the positive matches to a new list (or, in my case, 5 lists for the 5 fields I needed). Then I fed those lists into the format block instead.

Thanks for the help!

topic Playbook format block not using filtered data in Splunk SOAR

Playbook format block not using filtered data

Re: Playbook format block not using filtered data

Re: Playbook format block not using filtered data

Re: Playbook format block not using filtered data