topic Re: On what Splunk instance should my FIELDALIAS configurations go? in Getting Data In

On what Splunk instance should my FIELDALIAS configurations go?

a212830 — Wed, 30 Sep 2015 13:06:33 GMT

Hi,

I am processing Bluecoat logs on a heavy forwarder. I'm trying to set up some fields using FIELDALIAS, but they are not appearing. I have the following on my heavy forwarder. Should they be somewhere else?

Props.conf

[source::tcp:1918]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
REPORT-main = BlueCoatTCP_HTTPLogs
TRANSFORMS-tcp = nullPound
TIME_FORMAT = %Y-%m-%d %T
FIELDALIAS-srcip = c_ip AS src_ip
FIELDALIAS-cs_host = cs_host AS dest_host
FIELDALIAS-protocol = cs_uri_scheme AS protocol
FIELDALIAS-dest_port = cs_uri_port AS dest_port
FIELDALIAS-src_user = cs_username AS src_user
FIELDALIAS-s_ip = s_ip AS dvc_ip
FIELDALIAS-http_response = sc_status AS http_response
FIELDALIAS-bytes_out = cs_bytes AS bytes_out
FIELDALIAS-bytes_in = sc_bytes AS bytes_in
FIELDALIAS-r_ip = r_ip AS resolved_ip
FIELDALIAS-r_port = r_port AS resolved_port
FIELDALIAS-s_computername = s_computername AS proxy_server
pulldown_type = true
TZ = UTC
KV_MODE = none

Transforms.conf

[BlueCoatTCP_HTTPLogs]
DELIMS = " "
FIELDS = "date", "time", "s-computername", "cs-username", "c-ip", "sc-filter-res
ult", "cs-method", "cs-uri", "sc-status", "cs(Referer)", "cs-categories", "cs-au
th-group", "x-exception-id", "s-action", "cs(User-Agent)", "time-taken", "sc-byt
es", "cs-bytes", "cs-uri-scheme", "cs-host", "cs-uri-port", "cs-uri-path", "cs-u
ri-query", "cs-uri-extension", "rs(Content-Type)", "r-ip", "r-port"

Re: On what Splunk instance should my FIELDALIAS configurations go?

somesoni2 — Wed, 30 Sep 2015 16:01:22 GMT

Split your props.conf into two, one goes to Heavy forwarder, other on Search Head

Heavy forwarder : Props.conf

[source::tcp:1918]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
TRANSFORMS-tcp = nullPound
TIME_FORMAT = %Y-%m-%d %T
pulldown_type = true
TZ = UTC
KV_MODE = none 

Heavy Forwarder : Transforms.conf

[nullPound]
<<I thing this is the one you missed to mention>>


Search Head: Props.conf

[source::tcp:1918]
REPORT-main = BlueCoatTCP_HTTPLogs
FIELDALIAS-srcip = c_ip AS src_ip
FIELDALIAS-cs_host = cs_host AS dest_host
FIELDALIAS-protocol = cs_uri_scheme AS protocol
FIELDALIAS-dest_port = cs_uri_port AS dest_port
FIELDALIAS-src_user = cs_username AS src_user
FIELDALIAS-s_ip = s_ip AS dvc_ip
FIELDALIAS-http_response = sc_status AS http_response
FIELDALIAS-bytes_out = cs_bytes AS bytes_out
FIELDALIAS-bytes_in = sc_bytes AS bytes_in
FIELDALIAS-r_ip = r_ip AS resolved_ip
FIELDALIAS-r_port = r_port AS resolved_port
FIELDALIAS-s_computername = s_computername AS proxy_server

Search Head: Transforms.conf

[BlueCoatTCP_HTTPLogs]
DELIMS = " "
FIELDS = "date", "time", "s-computername", "cs-username", "c-ip", "sc-filter-res
ult", "cs-method", "cs-uri", "sc-status", "cs(Referer)", "cs-categories", "cs-au
th-group", "x-exception-id", "s-action", "cs(User-Agent)", "time-taken", "sc-byt
es", "cs-bytes", "cs-uri-scheme", "cs-host", "cs-uri-port", "cs-uri-path", "cs-u
ri-query", "cs-uri-extension", "rs(Content-Type)", "r-ip", "r-port"

Re: On what Splunk instance should my FIELDALIAS configurations go?

a212830 — Wed, 30 Sep 2015 16:55:00 GMT

What is the empty nullpound for in the transforms.conf?

Re: On what Splunk instance should my FIELDALIAS configurations go?

mreynov_splunk — Wed, 30 Sep 2015 17:35:10 GMT

FIELDALIAS is a search time operation, so it needs to go on the search head.

However, if you are using bluecoat proxy, there is already an add-on for that: https://splunkbase.splunk.com/app/2758/

Re: On what Splunk instance should my FIELDALIAS configurations go?

sloshburch — Thu, 01 Oct 2015 12:44:42 GMT

A good resource I use for identifying where configuration items should be set: http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

Re: On what Splunk instance should my FIELDALIAS configurations go?

somesoni2 — Thu, 01 Oct 2015 14:23:20 GMT

In your original props.conf, there is a line for a transform but there is no definition provided in transforms. I just added a placeholder.

TRANSFORMS-tcp = nullPound