topic Props not parsing the timestamp for TCP input logs in Getting Data In

Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 12:04:26 GMT

Hi All,

I have a bluecoat proxy log source for which I am using the official splunk addon. However, I noticed that the timestamp is not being parsed for from the logs and instead the index time is being taken.

To remedy this, I added a custom props in ../etc/apps/Splunk_TA_bluecoat-proxysg/local, with the following stanza:

[bluecoat:proxysg:access:syslog] TIME_FORMAT=%Y-%m-%d %H:%M:%S TIME_PREFIX=^

Rest of the configuration is the same as it is in the base app (Splunk_TA_bluecoat-proxysg).

During testing, when I upload logs through Add Data, the the time stamp is being properly parsed. However when I start using SplunkTCP to ingest the data, the timestamp extraction stops working. Note that in both of the scenarios, the rest of the parsing configurations (field extraction and mapping is working just fine).

Troubleshooting:

1. I tried to check with btool for props .. I can see the custom stanza I added there.

2. Tried putting the props in ../etc/system/local

3. Restarted Splunk multiple times.

Any ideas that I can try to get this to work? or where should I look at?

Sample Log:

2024-12-03 07:30:06 9 172.24.126.56 - - - - "None" - policy_denied DENIED "Suspicious" - 200 TCP_ACCELERATED CONNECT - tcp beyondwords-h0e8gjgjaqe0egb7.a03.azurefd.net 443 / - - "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0" 172.29.184.14 39 294 - - - - - "none" "none" "none" 7 - - 631d69b45739e3b6-00000000df56e125-00000000674eb37e - -

Splunk Search (Streaming data):

Splunk Search (uploaded data):

Re: Props not parsing the timestamp for TCP input logs

gcusello — Wed, 04 Dec 2024 12:07:38 GMT

Hi @Utkc137 ,

there's a priority in conf files reading and in that add-on there are some tranformations, so probably the sourcetype you added isn't present when the local file is read and created after using a transformation, see the default sourcetype and try adding your configuration to this sourcetype.

Ciao.

Giuseppe

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 12:13:16 GMT

Just tested using source in the props stanza name (source is define in inputs.conf) and it's still picking up the index time as the timestamp 😞

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 12:17:07 GMT

Also, the sourcetype I used originally is also mentioned in the inputs.conf .. and remains the same until the logs are ingested

Re: Props not parsing the timestamp for TCP input logs

gcusello — Wed, 04 Dec 2024 12:18:27 GMT

Hi @Utkc137 ,

did you tried with the sourcetype "bluecoat"?

that should be the one you assigned to your input.

Ciao.

Giuseppe

Re: Props not parsing the timestamp for TCP input logs

gcusello — Wed, 04 Dec 2024 12:20:37 GMT

Hi @Utkc137 ,

then, where do you located the add-on?

it should be in the first HF data passed through or (if HFs aren't present) in the Indexers.

Ciao.

Giuseppe

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 12:23:00 GMT

Just tested with bluecoat sourcetype .. no luck.

It's a standalone splunk instance (dev env).

Re: Props not parsing the timestamp for TCP input logs

dural_yyz — Wed, 04 Dec 2024 13:26:59 GMT

Can you share the inputs stanza you have for listening to the TCP stream?

Inside the default application props is:

[bluecoat] rename=bluecoat:proxysg:access:syslog

This occurs at search time only per the instructions at:

https://docs.splunk.com/Documentation/Splunk/9.3.0/Admin/Propsconf#Sourcetype_configuration

rename = <string>
* Renames [<sourcetype>] as <string> at search time
* With renaming, you can search for the [<sourcetype>] with
  sourcetype=<string>
* To search for the original source type without renaming it, use the
  field _sourcetype.
* Data from a renamed sourcetype only uses the search-time
  configuration for the target sourcetype. Field extractions
  (REPORTS/EXTRACT) for this stanza sourcetype are ignored.
* Default: empty string

This leaves any _time extraction issues with the source type identified in the inputs.conf stanza.

Re: Props not parsing the timestamp for TCP input logs

gcusello — Wed, 04 Dec 2024 13:27:41 GMT

Hi @Utkc137 ,

sorry for the very stupid question: did you restarted your Splunk server after conf update?

Could you share the inputs.conf you are using?

Please thy this:

[bluecoat] TIME_FORMAT=%Y-%m-%d %H:%M:%S TIME_PREFIX=^ rename=bluecoat:proxysg:access:syslog [bluecoat:proxysg:access:syslog] TIME_FORMAT=%Y-%m-%d %H:%M:%S TIME_PREFIX=^ pulldown_type = true category = Network & Security description = Data from Blue Coat ProxySG in W3C ELFF format thru syslog KV_MODE = none SHOULD_LINEMERGE = false EVENT_BREAKER_ENABLE=true MAX_DAYS_AGO = 10951 TRUNCATE = 64000

in local/props.conf

Ciao.

Giuseppe

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 13:38:59 GMT

Yes, I did restart splunk after each conf change 🙂

Here's the inputs.conf

[splunktcp://9997] index = mmsproxy source = tcp.bluecoat sourcetype = bluecoat:proxysg:access:syslog disabled = false

Will check you props too and respond back in a few min

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 13:48:45 GMT

For testing, I tied the props you provided along with these inputs.conf

Test 1:

[splunktcp://9997] index = mmsproxy source = tcp.bluecoat sourcetype = bluecoat:proxysg:access:syslog disabled = false

Test 2:

[splunktcp://9997] index = mmsproxy source = tcp.bluecoat sourcetype = bluecoat disabled = false

Restarted Splunk on both these tests. Still no luck.

Re: Props not parsing the timestamp for TCP input logs

dural_yyz — Wed, 04 Dec 2024 14:12:44 GMT

Port 9997 is a reserved port for splunk - if this is an external stream from syslog or any other source please select a different port.

Example

port=2514

I selected that as 514 is syslog reserved and 1514 I have seen for TCP encrypted syslog so best to just get up and away from that. But by keeping the *514 format it will be easier for others who may inherit your setup to know instinctively that it's a syslog source.

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 14:39:48 GMT

Switched the inputs to 2154 .. still no luck.

Re: Props not parsing the timestamp for TCP input logs

Utkc137 — Wed, 04 Dec 2024 14:53:58 GMT

This the configuration I have as of now .. I am out of reasons on why this would not work. Am I missing something very basic here?

Inputs:

./splunk btool inputs list --debug splunktcp://2514 /opt/splunk/etc/system/local/inputs.conf [splunktcp://2514] /opt/splunk/etc/system/default/inputs.conf _rcvbuf = 1572864 /opt/splunk/etc/system/local/inputs.conf disabled = false /opt/splunk/etc/system/default/inputs.conf host = $decideOnStartup /opt/splunk/etc/system/local/inputs.conf index = mmsproxy /opt/splunk/etc/system/local/inputs.conf source = tcp.bluecoat /opt/splunk/etc/system/local/inputs.conf sourcetype = bluecoat:proxysg:access:syslog

Props:

./splunk btool props list --debug bluecoat | grep -ie local /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf [bluecoat] /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf TIME_FORMAT = %Y-%m-%d %H:%M:%S /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf TIME_PREFIX = ^ /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf rename = bluecoat:proxysg:access:syslog /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf [bluecoat:proxysg:access:syslog] /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf EVENT_BREAKER_ENABLE = true /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf KV_MODE = none /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf MAX_DAYS_AGO = 10951 /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf SHOULD_LINEMERGE = false /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf TIME_FORMAT = %Y-%m-%d %H:%M:%S /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf TIME_PREFIX = ^ /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf TRUNCATE = 64000 /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf category = Network & Security /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf description = Data from Blue Coat ProxySG in W3C ELFF format thru syslog /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local/props.conf pulldown_type = true

Re: Props not parsing the timestamp for TCP input logs

gcusello — Wed, 04 Dec 2024 16:34:53 GMT

Hi @Utkc137 ,

sorry, but you're receiving logs from BlueCoat using syslog or from another Splunk Forwarder? usually BlueCoat uses syslogs not a Splunk Forwarder.

splunktcp inputs is for log forwarding from another Splunk system not using syslogs!

Ciao.

Giuseppe