How can I download arbitrary files from the Internet using Phantom?

[AlexBryant]

I have written a custom code block that outputs URLs of files on the Internet that I'd like Phantom to download and use later in the playbook. Given one of these lists of URLs, how can I have Phantom download those files and place them into the vault to be referenced by a later block?

[rgresham_splunk · Accepted solution]

Alex,

It would be my recommendation that you use the HTTP app to pull the file (link text
However, you will need to get the source code to modify it to get it to handle data file saves to the vault.

Currently, there is no prebuilt way to do this. You could continue your process and use the requests library to pull the files and save them using the phantom.add_vault() API call. (This would not be the best practice (as an app would be) due to the processing and delays that it puts under the decide daemon.

[cblumer_splunk]

This is a beta "wget App for Phantom" which can perform this action. Ask about it in the phantom-community.slack.com workspace.

[AlexBryant]

I got a copy off of slack and will let you know how it goes...thanks!

[AlexBryant]

The app appears to fail when wgetting an Internet resource that doesn't have a file extension. So a URL that ends in .htm or .txt or .jpg works fine, but something like pastebin[.]com/raw/dStYKpUb (a chocolate chip cookie recipe) fails. Maybe it's treating the filename as a folder since it doesn't have an extension, but I'm still researching:

{"app_name":"wget","asset_name":"wget","param":{"context": {"guid": "XXX", "artifact_id": 0, "parent_action_run": []}, "target_url": "https://pastebin.com/raw/dStYKpUb"},"status":"failed","message":"handle_action exception occurred. Error string: '[Errno 2] No such file or directory: '/opt/phantom/vault/tmp/pastebin_12-10-2019_09-16-09.com/raw/dStYKpUb''"}]