topic How can a regex query be written to extract source IP address? in Splunk Search

How can a regex query be written to extract source IP address?

Daksesh — Thu, 19 Jan 2023 18:17:10 GMT

The position of IP address is getting changed(appearing before or after https) in the logs, in such scenario how regex query can be written to extract source IP address.

Please guide me.

Below are the sample logs.

<14>Jan 19 04:32:59 XXXX accesslog_SIEM: Info: 1674102779.121 7 - 10.130.130.152 TCP_DENIED_SSL/403 0 POST https://activity.windows.com:443/v3/feeds/me/$batch - v3/feeds/me/$batch "Domainname\user@Domainname" - DROP_WEBCAT_7-BGC.BlockInternetAccess.DP-DOMPVM.Generalusers.ID-NONE-NONE-NONE-NONE-NONE - 61519 activity.windows.com 443 1 IW_comp 5.0 - - - - - - - - - - - - - - - - - IW_comp - "Computers and Internet" "Unknown" "Unknown" - - 0.00 0 - - - - - - - - - - - - - "SGPlatform 2.0" 21040

<14>Jan 19 04:32:59 XXXX accesslog_SIEM: Info: 1674102778.930 114 "https://www.XXX.com/English/home/default.aspx" 10.130.80.223 TCP_CLIENT_REFRESH_MISS_SSL/200 785 GET https://px.ads.linkedin.com:443/collect?v=2&fmt=js&pid=4881225&time=1674102778231&url=https%3A%2F%2Fwww.XXX.com%2FEnglish%2Fhome%2Fdefault.aspx&cookiesTest=true&liSync=true px.ads.linkedin.com collect?v=2&fmt=js&pid=4881225&time=1674102778231&url=https%3A%2F%2Fwww.XXX.com%2FEnglish%2Fhome%2Fdefault.aspx&cookiesTest=true&liSync=true - application/javascript ALLOW_CUSTOMCAT_12-ZAMLUM.FullAccess.AP-AuthByPass.ID-NONE-NONE-NONE-DefaultGroup-NONE - 54083 px.ads.linkedin.com 443 1 C_Allo 4.0 - - - - - - - - - - - - - - - - - IW_snet - "Social Networking" "Unknown" "Unknown" - - 55.09 0 - - - - - - - - - - - - - "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) browser/2020.2.6 Chrome/87.0.4280.141 Electron/11.3.99 Safari/537.36 PingdomTMS/2020.2" 1162

Re: Field extractions

ITWhisperer — Thu, 19 Jan 2023 09:13:27 GMT

Try something like this

| rex "^.+[^\d](?<ipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

Re: Field extractions

Daksesh — Thu, 19 Jan 2023 11:27:10 GMT

Hi ,

Thank you for your answer.

The output of above command gives results both source IP address(10.130.130.33) and destination IP address(192.111.4.115) in the same field ipaddress.

Would like to separate Source IP address and Destination IP address in props.conf file in Heavy forwarder. Could you please help on this.

Below is the sample log line.

<14>Jan 19 04:32:59 xxxxx accesslog_SIEM: Info: 1674102779.144 250 - 10.130.130.33 TCP_MISS_SSL/200 0 TCP_CONNECT 192.111.4.115:443 cloud-ec-asn.amp.cisco.com - - - DECRYPT_ADMIN_2-NONE-DOMPVM.Generalusers.ID-NONE-NONE-NONE-DefaultGroup-NONE - 55009 cloud-ec-asn.amp.cisco.com 443 2 IW_comp 9.4 1 - - - - - - - - - - - - - - - - IW_comp - "Computers and Internet" "Unknown" "Unknown" - - 0.00 0 - - - - - - - - - - - - - - 0

Re: Field extractions

ITWhisperer — Thu, 19 Jan 2023 13:06:34 GMT

You need to identify some additional text anchors for the pattern matching, for example, is source IP always followed by " TCP_", and, if present, is destination IP always followed by ":" (and a port number)?

| rex "^.+[^\d](?<srcipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) TCP_(.+[^\d](?<dstipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):\d)?"

Re: Field extractions

Daksesh — Fri, 20 Jan 2023 12:00:30 GMT

Hi ,

Thank you for your inputs.

I have written same regex in the props.conf file in Splunk heavy forwarder, but field extractions are not happening in search head.

Could you please guide me if there is any mistake from my end. Below is the props.conf file

[user@XXXXX local]$ cat props.conf
[cp_log1]
category = Custom
pulldown_type = 1

[wsa_test]
category = Custom
EXTRACT-src_ipaddress = .+[^\d](?<ipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
EXTRACT-dest_ipaddress = TCP_(.+[^\d](?<dstipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):\d)
EXTRACT-domain = https?:\/\/(?<domain>.*?(?:.com|.net))
EXTRACT-username = GOLDBAR\\(?<username>[\w]+)
pulldown_type = 1

Thank you

Re: Field extractions

Daksesh — Mon, 23 Jan 2023 05:35:53 GMT

Hi,

Had written regex in the props.conf file, but field extractions are not observing in Search head. Please find the below configuration. could you please guide me if anything to be included in props.conf file.

Thank you