topic Re: Extraction using rex in Splunk Search

Extraction using rex

rkris — Wed, 12 Aug 2020 16:39:55 GMT

How do I use rex to extract the backdoor info and the IP addresses so that I can display this info in my splunk dashboard?

Re: Extraction using rex

isoutamo — Wed, 12 Aug 2020 17:12:28 GMT

Please send those events as text, so the community could Help you with regex.

r. Ismo

Re: Extraction using rex

rkris — Thu, 13 Aug 2020 13:15:20 GMT

@isoutamo

These are my events in text format:

BACKDOOR MISC sm4ck attempt [Classification: ] [Priority: ]: {PROTOCOL} 75.93.73.179:58287 -> 1.227.5.72:25

BACKDOOR DeepThroat 3.1 Drive Info From Server <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 31.95.30.207:80 -> 10.10.50.16:100

BACKDOOR DeepThroat 3.1 Swap Mouse Buttons Client Request [Classification: ] [Priority: ]: {PROTOCOL} 198.15.21.160:58287 -> 158.170.160.216:25

BACKDOOR BackOrifice access [Classification: ] [Priority: ]: {PROTOCOL} 155.226.107.174:58287 -> 203.153.54.84:25

BACKDOOR MISC sm4ck attempt [Classification: ] [Priority: ]: {PROTOCOL} 75.93.73.179:58287 -> 1.227.5.72:25

BACKDOOR BackOrifice access <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 191.174.4.187:80 -> 10.10.50.26:100

BACKDOOR DeepThroat 3.1 All Window List Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 178.135.115.196:80 -> 10.10.50.234:100

BACKDOOR HackAttack 1.20 Connect <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 34.114.188.191:80 -> 10.10.50.126:100

Master's Paradise98 backdoor for Windows & from 192.168.34.6 to 10.10.18.60

BACKDOOR DeepThroat 3.1 E-Mail Info Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 218.224.198.5:80 -> 10.10.50.167:100

BACKDOOR DeepThroat 3.1 Server Password Change Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 20.75.220.146:80 -> 10.10.50.41:100

BACKDOOR DeepThroat 3.1 Run Program Normal Client Request [Classification: ] [Priority: ]: {PROTOCOL} 103.63.71.129:58287 -> 155.32.118.92:25

BACKDOOR Doly 2.0 access [Classification: ] [Priority: ]: {PROTOCOL} 237.194.170.123:58287 -> 9.208.133.84:25

BACKDOOR HackAttack 1.20 Connect <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 34.114.188.191:80 -> 10.10.50.126:100

AOL Admin backdoor for Windows and AOL & from 207.125.33.2 to 10.10.18.50

Backdoor: HP OpenView hidden SNMP community & from 109.48.111.123 to 10.10.18.54

BACKDOOR DeepThroat 3.1 Swap Mouse Buttons Client Request [Classification: ] [Priority: ]: {PROTOCOL} 198.15.21.160:58287 -> 158.170.160.216:25

BACKDOOR DeepThroat 3.1 Hide/Show Start Button Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 9.83.177.135:80 -> 10.10.50.53:100

BACKDOOR DeepThroat 3.1 Server Active on Network <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 1.68.247.43:80 -> 10.10.50.218:100

Master's Paradise98 backdoor for Windows & from 192.168.34.6 to 10.10.18.6

BACKDOOR DeepThroat 3.1 Resolution Change Client Request [Classification: ] [Priority: ]: {PROTOCOL} 72.3.15.96:58287 -> 26.211.133.37:25

BACKDOOR MISC linux rootkit attempt <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 23.173.136.97:80 -> 10.10.50.33:100

BACKDOOR BackOrifice access <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 191.174.4.187:80 -> 10.10.50.26:100

BACKDOOR Doly 2.0 access <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 168.248.174.125:80 -> 10.10.50.20:100

I only need to extract the backdoor names and IP address which are highlighted in red.

Re: Extraction using rex

isoutamo — Thu, 13 Aug 2020 19:42:07 GMT

Probably this is not an optimal solution, but it seems to work with you data.

index=_internal | head 1 | eval _raw="BACKDOOR MISC sm4ck attempt [Classification: ] [Priority: ]: {PROTOCOL} 75.93.73.179:58287 -> 1.227.5.72:25 BACKDOOR DeepThroat 3.1 Drive Info From Server <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 31.95.30.207:80 -> 10.10.50.16:100 BACKDOOR DeepThroat 3.1 Swap Mouse Buttons Client Request [Classification: ] [Priority: ]: {PROTOCOL} 198.15.21.160:58287 -> 158.170.160.216:25 BACKDOOR BackOrifice access [Classification: ] [Priority: ]: {PROTOCOL} 155.226.107.174:58287 -> 203.153.54.84:25 BACKDOOR MISC sm4ck attempt [Classification: ] [Priority: ]: {PROTOCOL} 75.93.73.179:58287 -> 1.227.5.72:25 BACKDOOR BackOrifice access <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 191.174.4.187:80 -> 10.10.50.26:100 BACKDOOR DeepThroat 3.1 All Window List Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 178.135.115.196:80 -> 10.10.50.234:100 BACKDOOR HackAttack 1.20 Connect <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 34.114.188.191:80 -> 10.10.50.126:100 Master's Paradise98 backdoor for Windows & from 192.168.34.6 to 10.10.18.60 BACKDOOR DeepThroat 3.1 E-Mail Info Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 218.224.198.5:80 -> 10.10.50.167:100 BACKDOOR DeepThroat 3.1 Server Password Change Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 20.75.220.146:80 -> 10.10.50.41:100 BACKDOOR DeepThroat 3.1 Run Program Normal Client Request [Classification: ] [Priority: ]: {PROTOCOL} 103.63.71.129:58287 -> 155.32.118.92:25 BACKDOOR Doly 2.0 access [Classification: ] [Priority: ]: {PROTOCOL} 237.194.170.123:58287 -> 9.208.133.84:25 BACKDOOR HackAttack 1.20 Connect <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 34.114.188.191:80 -> 10.10.50.126:100 AOL Admin backdoor for Windows and AOL & from 207.125.33.2 to 10.10.18.50 Backdoor: HP OpenView hidden SNMP community & from 109.48.111.123 to 10.10.18.54 BACKDOOR DeepThroat 3.1 Swap Mouse Buttons Client Request [Classification: ] [Priority: ]: {PROTOCOL} 198.15.21.160:58287 -> 158.170.160.216:25 BACKDOOR DeepThroat 3.1 Hide/Show Start Button Client Request <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 9.83.177.135:80 -> 10.10.50.53:100 BACKDOOR DeepThroat 3.1 Server Active on Network <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 1.68.247.43:80 -> 10.10.50.218:100 Master's Paradise98 backdoor for Windows & from 192.168.34.6 to 10.10.18.6 BACKDOOR DeepThroat 3.1 Resolution Change Client Request [Classification: ] [Priority: ]: {PROTOCOL} 72.3.15.96:58287 -> 26.211.133.37:25 BACKDOOR MISC linux rootkit attempt <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 23.173.136.97:80 -> 10.10.50.33:100 BACKDOOR BackOrifice access <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 191.174.4.187:80 -> 10.10.50.26:100 BACKDOOR Doly 2.0 access <\Device\Packet_{D3446BC6-343C-4C0C-8A29-AC6161C9D0F3}> TCP 168.248.174.125:80 -> 10.10.50.20:100" | rename COMMENT AS "Previous prepare test data. Replace it with your query with search word backdoor" | rex max_match=0 "^(?<msg>[^\[<\&]+)\s?.*[^\d](?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:\d+)?\s+(->)?(to)?\s(?<dst>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:\d+)?" | table src dst msg

max_match=0 is probably unneeded with your real query. This expect that your base query gives only lines which contains work "backdoor".

r. Ismo

Re: Extraction using rex

rkris — Fri, 14 Aug 2020 11:37:41 GMT

@isoutamo

These are the results i get after using your code:

Is there a way to ensure that the backdoor values appear on the same line as the IP Addresses?

Re: Extraction using rex

isoutamo — Fri, 14 Aug 2020 11:42:55 GMT

You should use your base query which generate those events and then add lines after the next line

| rename COMMENTS as ...

Those first lines just generate example data based on what you have sent earlier.

r. Ismo

Re: Extraction using rex

rkris — Fri, 14 Aug 2020 11:51:58 GMT

@isoutamo

A base query like this? :

source="General-linux-sql.log" sourcetype="Linux"

Re: Extraction using rex

isoutamo — Fri, 14 Aug 2020 12:58:30 GMT

Yes and add backdoor also.

index=<YOUR INDEX HERE> source="General-linux-sql.log" sourcetype="Linux" backdoor | rex "^(?<msg>[^\[<\&]+)\s?.*[^\d](?<src>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:\d+)?\s+(->)?(to)?\s(?<dst>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(:\d+)?" | table src dst msg

If needed you can add max_match=0 to rex command, but as your event is oneliners that shouldn't needed.

r. Ismo

Re: Extraction using rex

rkris — Fri, 14 Aug 2020 13:09:57 GMT

@isoutamo

What kind of value do i put in index?

Re: Extraction using rex

isoutamo — Fri, 14 Aug 2020 15:13:29 GMT

It’s same where you are stored those log events. Could be main or something else. Use the same index where you collect those examples. If you never use that then you could drop it away, even it best practices to always add it to the queries.

r. Ismo