Dashboards & Visualizations

Web Page Input Splunk for Json feeds from Hybrid-Analysis.com

dmenon84
Path Finder

Hi,

I added the following web-page config on Search Head.

URL - https://www.hybrid-analysis.com/feed?json
Selector-Td
Index=main
sourcetype=hybrid-feeds

I am getting the feeds but the format is not what I desire to see

Logs on Splunk


browser="integrated_client" response_size="282029" response_code="200" request_time="520.60508728" url="https://www.hybrid-analysis.com/feed?json" content_md5="b30597afe6c188d0254022546120ad58" content_sha224="ca641b20ad4acfe098e0d78901586c098948facd65c2b49d1a487c58" encoding="ascii" content="{ \"count\": 205, \"status\": \"ok\", \"data\": [ { \"md5\": \"8488817780e9de9d705e2bee0e299e44\", \"sha1\": \"80d0a0d98b60be0b8d57c3e197dc73d80d8a936f\", \"sha256\": \"fbd35e3151052bfb33f74d9083b158929220d1b47b48944fa6d0181b30cee9f4\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:45:12\", \"threatscore\": 100, \"threatlevel\": 2, \"avdetect\": 12, \"isunknown\": false, \"vxfamily\": \"QVM20.1.0000.Malware\", \"submitname\": \"ransomware.zip\", \"isurlanalysis\": false, \"size\": 725148, \"type\": \"PE32 executable (GUI) Intel 80386, for MS Windows\", \"et_alerts_total\": 4, \"et_alerts_real_total\": 15, \"domains\": [ \"api.ipify.org\", \"rd7v7mhidgrulwqg.onion.link\" ], \"hosts\": [ \"103.198.0.2\", \"23.23.167.0\" ], \"compromised_hosts\": [ \"103.198.0.2\" ], \"et_alerts\": [ { \"destip\": \"8.8.8.8\", \"destport\": \"53\", \"protocol\": \"UDP\", \"action\": { \"signatureid\": \"2022332\", \"signaturerev\": \"3\", \"severity\": \"1\", \"category\": \"A Network Trojan was detected\", \"description\": \"ET POLICY DNS Query to .onion proxy Domain (onion.link)\" } }, { \"destip\": \"23.23.167.0\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021997\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"Potential Corporate Privacy Violation\", \"description\": \"ET POLICY External IP Lookup api.ipify.org\" } }, { \"destip\": \"23.23.167.0\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021997\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"Potential Corporate Privacy Violation\", \"description\": \"ET POLICY External IP Lookup api.ipify.org\" } }, { \"destip\": \"23.23.167.0\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021997\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"Potential Corporate Privacy Violation\", \"description\": \"ET POLICY External IP Lookup api.ipify.org\" } } ], \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/fbd35e3151052bfb33f74d9083b158929220d1b47b48944fa6d0181b30cee9f4\/?environmentId=100\", \"vt_detect\": 12, \"ms_detect\": 12 }, { \"md5\": \"5f791c9ef260305a483dd28a972c96f2\", \"sha1\": \"b01641b8e3083d44c34dfa9b57c6e04a73e9405c\", \"sha256\": \"5146d4ab415390c08f30135588e5e871e54e8c774d0dd7e8949ae010ddfd6394\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:41:05\", \"threatscore\": 8, \"threatlevel\": 0, \"avdetect\": 0, \"isunknown\": false, \"submitname\": \"Normal.dotm\", \"isurlanalysis\": false, \"size\": 20635, \"type\": \"Microsoft Word 2007+\", \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/5146d4ab415390c08f30135588e5e871e54e8c774d0dd7e8949ae010ddfd6394\/?environmentId=100\", \"vt_detect\": 0, \"ms_detect\": 0 }, { \"md5\": \"0fdaa37867ca1a6b392ff5842b1ad167\", \"sha1\": \"1fbb0916e1efc68df54faf6f2e4f6524279058b1\", \"sha256\": \"9ac01ce2b88ce1c41a53ac967a8bbf434076f71c1d52ec54de404e2c7929d01f\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:38:42\", \"threatscore\": 87, \"threatlevel\": 2, \"avdetect\": 44, \"isunknown\": false, \"vxfamily\": \"Unwanted\", \"submitname\": \"Service_KMS.exe\", \"isurlanalysis\": false, \"size\": 974016, \"type\": \"PE32 executable (GUI) Intel 80386 Mono\/.Net assemb ...\", \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/9ac01ce2b88ce1c41a53ac967a8bbf434076f71c1d52ec54de404e2c7929d01f\/?environmentId=100\", \"vt_detect\": 44, \"ms_detect\": 44 }, { \"md5\": \"5d2b528ecec2b102fa5e8dc94db33316\", \"sha1\": \"d0b3349e135295dea7fe64e288caf89e90368c19\", \"sha256\": \"5b52d60f833fd2c27be55ba24c02cb91773f9a7fa0261278aa783e2fb436b8b9\", \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:37:06\", \"threatscore\": 100, \"threatlevel\": 2, \"avdetect\": 40, \"isunknown\": false, \"vxfamily\": \"W97M.Downloader\", \"submitname\": \"guy.mackenzie.doc\", \"isurlanalysis\": false, \"size\": 42470, \"type\": \"Microsoft Word 2007+\", \"domains\": [ \"www.maxmind.com\" ], \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": false, \"isreliable\": true, \"reporturl\": \"\/sample\/5b52d60f833fd2c27be55ba24c02cb91773f9a7fa0261278aa783e2fb436b8b9\/?environmentId=100\", \"vt_detect\": 40, \"ms_detect\": 40 }, { \"md5\": \"36280b99d6f882abbb843776a2f995ce\", \"sha1\": \"64a7bd642ecc672a9ac1420a7dd4087db31f93c4\", \"sha256\": \"c9866f3d453936bb71a84b13703dbb507f56b7b192ae2692900339295cf48f60\", \"isunknown\": true, \"isinteresting\": false, \"analysis_start_time\": \"2016-08-26 11:36:27\", \"threatscore\": 56, \"threatlevel\": 2, \"submitname\": \"eBILL_BritishGas.js\", \"isurlanalysis\": false, \"size\": 6793, \"type\": \"ASCII text\", \"et_alerts_total\": 2, \"et_alerts_real_total\": 2, \"domains\": [ \"www.numengo.com\" ], \"hosts\": [ \"217.70.180.131\" ], \"compromised_hosts\": [ \"217.70.180.131\" ], \"et_alerts\": [ { \"destip\": \"217.70.180.131\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2021697\", \"signaturerev\": \"2\", \"severity\": \"1\", \"category\": \"A Network Trojan was detected\", \"description\": \"ET TROJAN EXE Download Request To Wordpress Folder Likely Malicious\" } }, { \"destip\": \"217.70.180.131\", \"destport\": \"80\", \"protocol\": \"TCP\", \"action\": { \"signatureid\": \"2022239\", \"signaturerev\": \"4\", \"severity\": \"1\", \"category\": \"A Network Trojan was detected\", \"description\": \"ET TROJAN EXE Download Request To Wordpress Folder Likely Malicious\" } } ], \"environmentId\": \"100\", \"environmentDescription\": \"Windows 7 32 bit\", \"sharedanalysis\": true, \"isreliable\": true, \"reporturl\": \"\/sample\/c9866f3d453936bb71a84b13703dbb507f56b7b192ae2692900339295cf48f60\/?environmentId=100\" }, { \"md5\": \"5a39973622ed4230bfcf003b4ac9f18b\", \"sha1\": \"2661f82b0ffd7ac3f274a31fa564dcb785a4fe36\", \"sha256\": \"d42135cf81df795b76fe0c6d0cac61de55966efd59cb1768c561b57e49ad7ab2\", \"isunknown\": true, \"isinteresting\": true, \"analysis_start_time\": \"2016-08-26 11:35:47\", \"threatscore\": 100, \"threatlevel\": 2, \"submitname\": \"vii_pay_commission_scales.doc\", \"isurlanalysis\": false, \"size\": 1052402, \"type\": \"Rich Text Format data, v


I have the following questions -
1) Is there another way to input data from https://www.hybrid-analysis.com/feed?json (public feeds)?
2) Also I am getting duplicates with the method I am using ? How to get rid of the duplicates?
3) Also should I add this in Master instead of Search Head ?

Thanks in advance for any guidance.

Tags (1)

neelamsantosh
Path Finder

Mate have u integrated the feeds into splunk.

I have the same requirement as yours.

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...