We are trying to extract from the field "url" all files transferred as part of a session. The ultimate goal is to keep track of all filenames and extensions, with the ability to sort and report on file activity available within the proxy logs.
I started with the intent to prove that a regex worked that would list all url's that included a file name and pre-identified extensions (or all extensions), and then was going to search against the results to pull out the complete filename for reporting (divide and conquer). I tested my regex using several regex sites (ex. REGEX101.com). All accepted the regex as valid but am unable to get it to run in Splunk.
Running the Splunk cmd without success, continue to receive the error "Missing closing tick mark in macro expression". Unable to find any details on what this is referring to, especially since the test on non-Splunk resources give positive results? Where or what is this referring to? Is there a better way to achieve the results?:
| regex _raw="((?:https?:\/\/|www\.)(?:[\dA-Za-z-]++\.)+[A-Za-z]{2,6}+(?:\/[]\w!"#$%&'()*+,.:;<=>?@[\\{|}~^-]+){0,9}\/[\w.-]+.(?:bat|exe|pdf|js|msi))"`
Am able to successfully run the following, but only after stripping out the capability to only return URL's with files and interesting extensions at the end of the URL:
| regex _raw="((?:https?:\/\/|www.)(?:[\dA-Za-z-]++.)+[A-Za-z]{2,6}+(?:))"
Thanks in advance for any assist
