Getting Data In

On what Splunk instance should my FIELDALIAS configurations go?

Champion

Hi,

I am processing Bluecoat logs on a heavy forwarder. I'm trying to set up some fields using FIELDALIAS, but they are not appearing. I have the following on my heavy forwarder. Should they be somewhere else?

Props.conf

[source::tcp:1918]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
REPORT-main = BlueCoatTCP_HTTPLogs
TRANSFORMS-tcp = nullPound
TIME_FORMAT = %Y-%m-%d %T
FIELDALIAS-srcip = c_ip AS src_ip
FIELDALIAS-cs_host = cs_host AS dest_host
FIELDALIAS-protocol = cs_uri_scheme AS protocol
FIELDALIAS-dest_port = cs_uri_port AS dest_port
FIELDALIAS-src_user = cs_username AS src_user
FIELDALIAS-s_ip = s_ip AS dvc_ip
FIELDALIAS-http_response = sc_status AS http_response
FIELDALIAS-bytes_out = cs_bytes AS bytes_out
FIELDALIAS-bytes_in = sc_bytes AS bytes_in
FIELDALIAS-r_ip = r_ip AS resolved_ip
FIELDALIAS-r_port = r_port AS resolved_port
FIELDALIAS-s_computername = s_computername AS proxy_server
pulldown_type = true
TZ = UTC
KV_MODE = none 

Transforms.conf

[BlueCoatTCP_HTTPLogs]
DELIMS = " "
FIELDS = "date", "time", "s-computername", "cs-username", "c-ip", "sc-filter-res
ult", "cs-method", "cs-uri", "sc-status", "cs(Referer)", "cs-categories", "cs-au
th-group", "x-exception-id", "s-action", "cs(User-Agent)", "time-taken", "sc-byt
es", "cs-bytes", "cs-uri-scheme", "cs-host", "cs-uri-port", "cs-uri-path", "cs-u
ri-query", "cs-uri-extension", "rs(Content-Type)", "r-ip", "r-port"
0 Karma

Ultra Champion

A good resource I use for identifying where configuration items should be set: http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

Splunk Employee
Splunk Employee

FIELDALIAS is a search time operation, so it needs to go on the search head.

However, if you are using bluecoat proxy, there is already an add-on for that: https://splunkbase.splunk.com/app/2758/

0 Karma

Revered Legend

Split your props.conf into two, one goes to Heavy forwarder, other on Search Head

Heavy forwarder : Props.conf

[source::tcp:1918]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
TRANSFORMS-tcp = nullPound
TIME_FORMAT = %Y-%m-%d %T
pulldown_type = true
TZ = UTC
KV_MODE = none 

Heavy Forwarder : Transforms.conf

[nullPound]
<<I thing this is the one you missed to mention>>


Search Head: Props.conf

[source::tcp:1918]
REPORT-main = BlueCoatTCP_HTTPLogs
FIELDALIAS-srcip = c_ip AS src_ip
FIELDALIAS-cs_host = cs_host AS dest_host
FIELDALIAS-protocol = cs_uri_scheme AS protocol
FIELDALIAS-dest_port = cs_uri_port AS dest_port
FIELDALIAS-src_user = cs_username AS src_user
FIELDALIAS-s_ip = s_ip AS dvc_ip
FIELDALIAS-http_response = sc_status AS http_response
FIELDALIAS-bytes_out = cs_bytes AS bytes_out
FIELDALIAS-bytes_in = sc_bytes AS bytes_in
FIELDALIAS-r_ip = r_ip AS resolved_ip
FIELDALIAS-r_port = r_port AS resolved_port
FIELDALIAS-s_computername = s_computername AS proxy_server

Search Head: Transforms.conf

[BlueCoatTCP_HTTPLogs]
DELIMS = " "
FIELDS = "date", "time", "s-computername", "cs-username", "c-ip", "sc-filter-res
ult", "cs-method", "cs-uri", "sc-status", "cs(Referer)", "cs-categories", "cs-au
th-group", "x-exception-id", "s-action", "cs(User-Agent)", "time-taken", "sc-byt
es", "cs-bytes", "cs-uri-scheme", "cs-host", "cs-uri-port", "cs-uri-path", "cs-u
ri-query", "cs-uri-extension", "rs(Content-Type)", "r-ip", "r-port"
0 Karma

Champion

What is the empty nullpound for in the transforms.conf?

0 Karma

Revered Legend

In your original props.conf, there is a line for a transform but there is no definition provided in transforms. I just added a placeholder.

TRANSFORMS-tcp = nullPound

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!