How can a regex query be written to extract source IP address?

[Daksesh]

The position of IP address is getting changed(appearing before or after https) in the logs, in such scenario how regex query can be written to extract source IP address.

Please guide me.

Below are the sample logs.

<14>Jan 19 04:32:59 XXXX  accesslog_SIEM: Info: 1674102779.121 7 - 10.130.130.152 TCP_DENIED_SSL/403 0 POST https://activity.windows.com:443/v3/feeds/me/$batch - v3/feeds/me/$batch "Domainname\user@Domainname" - DROP_WEBCAT_7-BGC.BlockInternetAccess.DP-DOMPVM.Generalusers.ID-NONE-NONE-NONE-NONE-NONE - 61519 activity.windows.com 443 1 IW_comp 5.0 - - - - - - - - - - - - - - - - - IW_comp - "Computers and Internet" "Unknown" "Unknown" - - 0.00 0 - - - - - - - - - - - - - "SGPlatform 2.0" 21040

<14>Jan 19 04:32:59 XXXX accesslog_SIEM: Info: 1674102778.930 114 "https://www.XXX.com/English/home/default.aspx" 10.130.80.223 TCP_CLIENT_REFRESH_MISS_SSL/200 785 GET https://px.ads.linkedin.com:443/collect?v=2&fmt=js&pid=4881225&time=1674102778231&url=https%3A%2F%2F... px.ads.linkedin.com collect?v=2&fmt=js&pid=4881225&time=1674102778231&url=https%3A%2F%2Fwww.XXX.com%2FEnglish%2Fhome%2Fdefault.aspx&cookiesTest=true&liSync=true - application/javascript ALLOW_CUSTOMCAT_12-ZAMLUM.FullAccess.AP-AuthByPass.ID-NONE-NONE-NONE-DefaultGroup-NONE - 54083 px.ads.linkedin.com 443 1 C_Allo 4.0 - - - - - - - - - - - - - - - - - IW_snet - "Social Networking" "Unknown" "Unknown" - - 55.09 0 - - - - - - - - - - - - - "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) browser/2020.2.6 Chrome/87.0.4280.141 Electron/11.3.99 Safari/537.36 PingdomTMS/2020.2" 1162

[Daksesh]

Hi ,

Thank you for your answer.

The output of above command gives results both source IP address(10.130.130.33) and destination IP address(192.111.4.115) in the same field ipaddress. 

Would like to separate Source IP address and Destination IP address in props.conf file in Heavy forwarder. Could you please help on this.

Below is the sample log line.

<14>Jan 19 04:32:59 xxxxx accesslog_SIEM: Info: 1674102779.144 250 - 10.130.130.33 TCP_MISS_SSL/200 0 TCP_CONNECT 192.111.4.115:443 cloud-ec-asn.amp.cisco.com - - - DECRYPT_ADMIN_2-NONE-DOMPVM.Generalusers.ID-NONE-NONE-NONE-DefaultGroup-NONE - 55009 cloud-ec-asn.amp.cisco.com 443 2 IW_comp 9.4 1 - - - - - - - - - - - - - - - - IW_comp - "Computers and Internet" "Unknown" "Unknown" - - 0.00 0 - - - - - - - - - - - - - - 0

[ITWhisperer]

You need to identify some additional text anchors for the pattern matching, for example, is source IP always followed by " TCP_", and, if present, is destination IP always followed by ":" (and a port number)?

| rex "^.+[^\d](?<srcipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) TCP_(.+[^\d](?<dstipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):\d)?"

[ITWhisperer]

Try something like this

| rex "^.+[^\d](?<ipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

[Daksesh]

Hi ,

Thank you for your inputs.

I have written same regex in the props.conf file in Splunk heavy forwarder, but field extractions are not happening in search head. 

Could you please guide me if there is any mistake from my end. Below is the props.conf file

[user@XXXXX local]$ cat props.conf
[cp_log1]
category = Custom
pulldown_type = 1

[wsa_test]
category = Custom
EXTRACT-src_ipaddress = .+[^\d](?<ipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
EXTRACT-dest_ipaddress = TCP_(.+[^\d](?<dstipaddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):\d)
EXTRACT-domain = https?:\/\/(?<domain>.*?(?:.com|.net))
EXTRACT-username = GOLDBAR\\(?<username>[\w]+)
pulldown_type = 1

Thank you

[Daksesh]

Hi, 

Had written regex in the props.conf file, but field extractions are not observing in Search head. Please find the below configuration. could you please guide me if anything to be included in props.conf file.

Thank you