Is it possible to load data from a url using SPL at the search line? Three uses cases, specifically:
1) Load https://server.domain.com:8000/en-US/search/inspector?sid=[sid]&namespace=search for job inspect data for a particular sid,
2) Load https://server.domain.com:8000/en-US/api/search/jobs/[sid]/search.log?outputMode=raw for search.log for a particular sid,
3) Load https://mypage.mysite.com/myfile.csv
I know this can be done with scripts somehow, and while I am interested in that process as well, this question focuses on using the search bar (even with the assistance of an app if necessary) to load the data.
The concept is extremely simple in general. At the SPL search bar, a query something like the following:
| loadurl https://mypage.mysite.com/myfile.csv
Loads the file from the webpage specified just like a normal lookup file would be loaded. If the urls from 1 or 2 were used, the data from those pages would be displayed in the Splunk search head client web UI.
If you are on a Search Head's CLI and you need to get search results back from another search head and then SPL on that data, you will have to save the output from the other search head (using
curl or similar to run the job and pull the output back) into a directory where you can then use
inputcsv on your search head to pull it in as data for a search.
Or am I misunderstanding what you are trying to do?
It seems like people are thinking that I am asking about loading the results from a previously run job, but I am not. Regarding the first two use cases, I am asking how to load the results if the Job Inspect from a previous job and how to load the results of the search.log results from a previous job.
Procedure for #1 in the OP (original post) above:
1) Run job, 2) Click Job Inspect, 3) Right click the new window top bar and choose show as tab (Chrome), 4) Compare url of new window to #1 in the OP above.
Procedure for #2 in the OP above:
1) Run job, 2) Click Job Inspect, 3) Right click the new window top bar and choose show as tab (Chrome),
4) Click search.log, 5) Compare url of window to #2 in the OP above.
Procedure for #3 in the OP above:
1) Go to any file summary page on Virustotal, 2) Look at the url.
For any of the three procedures above, load the results via the Splunk search bar (SPL).
When I try the following SPL example (found at http://blogs.splunk.com/2011/08/16/getwatchlist-getting-watchlists-into-splunk-quickly-and-easily-wi...), I receive an error:
| getwatchlist http://data.phishtank.com/data/online-valid.csv delimiter=”,” relevantFieldName=url relevantFieldCol=2 referenceCol=3 dateCol=4 categoryCol=8 ignoreFirstLine=true isbad=true | outputlookup phishtank.csv
command="getwatchlist", Error getting settings: need more than 1 value to unpack
What does that mean?
It turns out that copying text from an online source sometimes brings the wrong double quotes. In this case, the double quotes around the comma were wrong and this error is resolved by replacing them with normal double quotes.
| getwatchlist http://data.phishtank.com/data/online-valid.csv delimiter="," relevantFieldName=url relevantFieldCol=2 referenceCol=3 dateCol=4 categoryCol=8 ignoreFirstLine=true isbad=true | outputlookup phishtank.csv