Splunk not seeing remote host Filezilla

[chrisw9808]

We have a remote windows ftp server that splunk is pulling logs off and I am not able to get it to recognize the remote host or ip as searchable fields.

The raw data looks like this

(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> TYPE I
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> 200 Type set to I
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> PORT 10,180,48,145,19,136
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> 200 Port command successful
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> STOR    /../../
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> 150 Opening data channel for file transfer.
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 226 Transfer OK
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> TYPE I
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 200 Type set to I
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> PORT 10,180,48,145,4,2

The local.conf on the windows server looks like this.

[monitor://C:\Program Files\FileZilla Server\Logs]
host = ftp01
index = hosted
sourcetype = ftp_filezilla
_whitelist = (?i).*\.log$

[monitor://C:\Program Files\FileZilla Server\Logs]
host = ftp01
index = hosted
sourcetype = ftp_filezilla
_whitelist = (?i).*\.log$

[WinEventLog:Application]
disabled = 1

[WinEventLog:Security]
disabled = 1

[WinEventLog:System]
disabled = 1

The host name acfXXX is not at all similar every time and I would need to configure it to strip it out by row number ie in linux awk '{print $6}'

[martin_mueller · Accepted solution]

(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 226 Transfer OK

Looking at the sample data, something like this should let you extract the two fields on the fly:

... | rex "-\s+(?<remote_host>\S+)\s+\((?<remote_ip>(\d+\.?)+)\)"

Once you're happy with the extraction you can move the expression to props.conf / Settings -> Fields -> Field Extractions to have the fields searchable every time.