Dashboards & Visualizations

Web Page Input Splunk for Json feeds from Hybrid-Analysis.com

dmenon84
Path Finder

Hi,

I added the following web-page config on Search Head.

URL - https://www.hybrid-analysis.com/feed?json
Selector-Td
Index=main
sourcetype=hybrid-feeds

I am getting the feeds but the format is not what I desire to see

Logs on Splunk


browser="integrated_client" response_size="282029" response_code="200" request_time="520.60508728" url="https://www.hybrid-analysis.com/feed?json" content_md5="b30597afe6c188d0254022546120ad58" content_sha224="ca641b20ad4acfe098e0d78901586c098948facd65c2b49d1a487c58" encoding="ascii" content="{ \"count\": 205, \"status\": \"ok\", \"data\": [ { \"md5\": \"8488817780e9de9d705e2bee0e299e44\", \"sha1\": \"80d0a0d98b60be0b8d57c3e197dc73d80d8a936f\", \"sha256\": \"fbd35e3151052bfb33f74d9083b158929220d1b47b48944fa6d0181b30cee9f4\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:45:12\", \"threatscore\": 100, \"threatlevel\": 2, \"avdetect\": 12, \"isunknown\": false, \"vxfamily\": \"QVM20.1.0000.Malware\", \"submitname\": \"ransomware.zip\", \"isurlanalysis\": false, \"size\": 725148, \"type\": \"PE32 executable (GUI) Intel 80386, for MS Windows\", \"et_alerts_total\": 4, \"et_alerts_real_total\": 15, \"domains\": [ \"api.ipify.org\", \"rd7v7mhidgrulwqg.onion.link\" ], \"hosts\": [ \"103.198.0.2\", \"23.23.167.0\" ], \"compromised_hosts\": [ \"103.198.0.2\" ], \"et_alerts\": [ { \"destip\": \"8.8.8.8\", \"destport\": \"53\", \"protocol\": \"UDP\", \"action\": { \"signatureid\": \"2022332\", \"signaturerev\": \"3\", \"severity\": \"1\", \"category\": \"A Network Trojan was detected\", \"description\": \"ET POLICY DNS Query to .onion proxy Domain (onion.link)\" } }, { \"destip\": \"23.23.167.0\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021997\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"Potential Corporate Privacy Violation\", \"description\": \"ET POLICY External IP Lookup api.ipify.org\" } }, { \"destip\": \"23.23.167.0\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021997\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"Potential Corporate Privacy Violation\", \"description\": \"ET POLICY External IP Lookup api.ipify.org\" } }, { \"destip\": \"23.23.167.0\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021997\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"Potential Corporate Privacy Violation\", \"description\": \"ET POLICY External IP Lookup api.ipify.org\" } } ], \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/fbd35e3151052bfb33f74d9083b158929220d1b47b48944fa6d0181b30cee9f4\/?environmentId=100\", \"vt_detect\": 12, \"ms_detect\": 12 }, { \"md5\": \"5f791c9ef260305a483dd28a972c96f2\", \"sha1\": \"b01641b8e3083d44c34dfa9b57c6e04a73e9405c\", \"sha256\": \"5146d4ab415390c08f30135588e5e871e54e8c774d0dd7e8949ae010ddfd6394\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:41:05\", \"threatscore\": 8, \"threatlevel\": 0, \"avdetect\": 0, \"isunknown\": false, \"submitname\": \"Normal.dotm\", \"isurlanalysis\": false, \"size\": 20635, \"type\": \"Microsoft Word 2007+\", \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/5146d4ab415390c08f30135588e5e871e54e8c774d0dd7e8949ae010ddfd6394\/?environmentId=100\", \"vt_detect\": 0, \"ms_detect\": 0 }, { \"md5\": \"0fdaa37867ca1a6b392ff5842b1ad167\", \"sha1\": \"1fbb0916e1efc68df54faf6f2e4f6524279058b1\", \"sha256\": \"9ac01ce2b88ce1c41a53ac967a8bbf434076f71c1d52ec54de404e2c7929d01f\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:38:42\", \"threatscore\": 87, \"threatlevel\": 2, \"avdetect\": 44, \"isunknown\": false, \"vxfamily\": \"Unwanted\", \"submitname\": \"Service_KMS.exe\", \"isurlanalysis\": false, \"size\": 974016, \"type\": \"PE32 executable (GUI) Intel 80386 Mono\/.Net assemb ...\", \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/9ac01ce2b88ce1c41a53ac967a8bbf434076f71c1d52ec54de404e2c7929d01f\/?environmentId=100\", \"vt_detect\": 44, \"ms_detect\": 44 }, { \"md5\": \"5d2b528ecec2b102fa5e8dc94db33316\", \"sha1\": \"d0b3349e135295dea7fe64e288caf89e90368c19\", \"sha256\": \"5b52d60f833fd2c27be55ba24c02cb91773f9a7fa0261278aa783e2fb436b8b9\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:37:06\", \"threatscore\": 100, \"threatlevel\": 2, \"avdetect\": 40, \"isunknown\": false, \"vxfamily\": \"W97M.Downloader\", \"submitname\": \"guy.mackenzie.doc\", \"isurlanalysis\": false, \"size\": 42470, \"type\": \"Microsoft Word 2007+\", \"domains\": [ \"www.maxmind.com\" ], \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": false, \"isreliable\": true, \"reporturl\": \"\/sample\/5b52d60f833fd2c27be55ba24c02cb91773f9a7fa0261278aa783e2fb436b8b9\/?environmentId=100\", \"vt_detect\": 40, \"ms_detect\": 40 }, { \"md5\": \"36280b99d6f882abbb843776a2f995ce\", \"sha1\": \"64a7bd642ecc672a9ac1420a7dd4087db31f93c4\", \"sha256\": \"c9866f3d453936bb71a84b13703dbb507f56b7b192ae2692900339295cf48f60\", \"isunknown\": true, \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:36:27\", \"threatscore\": 56, \"threatlevel\": 2, \"submitname\": \"eBILL_BritishGas.js\", \"isurlanalysis\": false, \"size\": 6793, \"type\": \"ASCII text\", \"et_alerts_total\": 2, \"et_alerts_real_total\": 2, \"domains\": [ \"www.numengo.com\" ], \"hosts\": [ \"217.70.180.131\" ], \"compromised_hosts\": [ \"217.70.180.131\" ], \"et_alerts\": [ { \"destip\": \"217.70.180.131\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021697\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"A Network Trojan was detected\", \"description\": \"ET TROJAN EXE Download Request To Wordpress Folder Likely Malicious\" } }, { \"destip\": \"217.70.180.131\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2022239\", \"signaturerev\": \"4\", \"severity\": \"1\", \"category\": \"A Network Trojan was detected\", \"description\": \"ET TROJAN EXE Download Request To Wordpress Folder Likely Malicious\" } } ], \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/c9866f3d453936bb71a84b13703dbb507f56b7b192ae2692900339295cf48f60\/?environmentId=100\" }, { \"md5\": \"5a39973622ed4230bfcf003b4ac9f18b\", \"sha1\": \"2661f82b0ffd7ac3f274a31fa564dcb785a4fe36\", \"sha256\": \"d42135cf81df795b76fe0c6d0cac61de55966efd59cb1768c561b57e49ad7ab2\", \"isunknown\": true, \"isinteresting\": true, \"analysis_start_time\": \"2016-08-26 11:35:47\", \"threatscore\": 100, \"threatlevel\": 2, \"submitname\": \"vii_pay_commission_scales.doc\", \"isurlanalysis\": false, \"size\": 1052402, \"type\": \"Rich Text Format data, v


I have the following questions -
1) Is there another way to input data from https://www.hybrid-analysis.com/feed?json (public feeds)?
2) Also I am getting duplicates with the method I am using ? How to get rid of the duplicates?
3) Also should I add this in Master instead of Search Head ?

Thanks in advance for any guidance.

Tags (1)

neelamsantosh
Path Finder

Mate have u integrated the feeds into splunk.

I have the same requirement as yours.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...