About dj064

dj064

We are facing a log indexing issue with the log paths mentioned below. Previously, with the same inputs.conf configuration, logs were being ingested without issues, but suddenly, it stopped sending logs. Each log file contains logs for a single day, but splunk reports that it has already read these logs and skips them. Below is the inputs.conf configuration: [monitor://C:\Ticker\out\] whitelist = .*_Mcast2Msg\\logs\\.*log$ index = rtd disabled = false followTail = 0 ignoreOlderThan = 3d recursive = true sourcetype = rtd_mcast crcSalt = <SOURCE> source path: C:\Ticker\out\Equiduct_Mcast2Msg\logs\EquiductTest-01-21-25.log C:\Ticker\out\Istanbul_Mcast2Msg\logs\Istanbul-01-16-25.log C:\Ticker\out\JSE_Mcast2Msg\logs\JSE-01-16-25.log C:\Ticker\out\JSE_Mcast2Msg\logs\JSEtst-01-17-25.log C:\Ticker\out\Warsaw_Mcast2Msg\logs\Warsaw-01-14-25.log _internal logs: 01-21-2025 14:48:20.745 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=105 for file='C:\Ticker\out\Equiduct_Mcast2Msg\logs\Equiduct-Limit-1-01-21-25.log'. 01-21-2025 14:48:13.586 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=171 for file='C:\Ticker\out\Equiduct_Mcast2Msg\logs\Equiduct-Limit-1-01-20-25.log'. 01-21-2025 14:48:06.332 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=66 for file='C:\Ticker\out\Istanbul_Mcast2Msg\logs\Istanbul-01-21-25.log'. 01-21-2025 14:47:57.650 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=66 for file='C:\Ticker\out\Istanbul_Mcast2Msg\logs\Istanbul-01-20-25.log'. 01-21-2025 14:47:51.466 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=65 for file='C:\Ticker\out\JSE_Mcast2Msg\logs\JSE-01-20-25.log'. 01-21-2025 14:47:45.271 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=65 for file='C:\Ticker\out\JSE_Mcast2Msg\logs\JSE-01-21-25.log'. 01-21-2025 14:47:39.644 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=114 for file='C:\Ticker\out\Warsaw_Mcast2Msg\logs\Warsaw-01-21-25.log'. 01-21-2025 14:47:35.855 +0000 INFO WatchedFile [708 tailreader0] - Will begin reading at offset=114 for file='C:\Ticker\out\Warsaw_Mcast2Msg\logs\Warsaw-01-20-25.log'. 01-21-2025 14:47:35.660 +0000 INFO TailingProcessor [6536 MainTailingThread] - Adding watch on path: C:\Ticker\out. 01-21-2025 14:47:35.659 +0000 INFO TailingProcessor [6536 MainTailingThread] - Parsing configuration stanza: monitor://C:\Ticker\out\. Issue Details: 1) When we update the very first line of a log file, only the updated first line is ingested by Splunk, and the rest of the content is skipped. 2) We have deleted the fishbucket, but the issue persists. 3) Even after reinstalling the Splunk forwarder (version 8.2.12), the problem continues.

dj064 · ‎10-17-2024

Hi @PickleRick, Thank you for your suggestions. After following your suggestions, the configurations are now working correctly for my use case. Here are the changes I made for [route_to_teamid_index] stanza in transforms.conf: 1) For [route_to_teamid_index] - Set FORMAT = $1 - Updated SOURCE_KEY = MetaData:Source Current working configs for my use cases: ----------------------------------------------------------------------------- props ----------------------------------------------------------------------------- #custom-props-for-starflow-logs [source::.../starflow-app-logs...] TRANSFORMS-set_new_sourcetype = new_sourcetype TRANSFORMS-set_route_to_teamid_index = route_to_teamid_index ----------------------------------------------------------------------------- transforms ----------------------------------------------------------------------------- #custom-transforms-for-starflow-logs [new_sourcetype] REGEX = .* DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::aws:kinesis:starflow WRITE_META = true [route_to_teamid_index] REGEX = .*\/starflow-app-logs(?:-[a-z]+)?\/([a-zA-Z0-9]+)\/ SOURCE_KEY = MetaData:Source FORMAT = $1 DEST_KEY = _MetaData:Index WRITE_META = true Previously, the configuration had SOURCE_KEY = source, which was causing issues. The SOURCE_KEY = <field> setting essentially tells Splunk where the regex should be applied. In my configuration, it was set to "source" but Splunk might not have been able to apply the regex to just the source field. After spending time reading through transforms.conf, I noticed that under the global settings, there was a specific mention of this. SOURCE_KEY = <string> * NOTE: This setting is valid for both index-time and search-time field extractions. * Optional. Defines the KEY that Splunk software applies the REGEX to. * For search time extractions, you can use this setting to extract one or more values from the values of another field. You can use any field that is available at the time of the execution of this field extraction * For index-time extractions use the KEYs described at the bottom of this file. * KEYs are case-sensitive, and should be used exactly as they appear in the KEYs list at the bottom of this file. (For example, you would say SOURCE_KEY = MetaData:Host, *not* SOURCE_KEY = metadata:host .) Keys MetaData:Source : The source associated with the event. Thank you sincerely for all of your genuine help!

dj064 · ‎10-16-2024

@PickleRick Thanks for suggestion. I made the following changes in transforms.conf: 1) For [new_sourcetype] - Removed SOURCE_KEY = source 2) For [route_to_teamid_index] - Updated the regex - Set WRITE_META = true After these changes, the sourcetype value successfully changed to "aws:kinesis:starflow", but the data did not route to the specified index. Instead, it went to the default index. current configs: ----------------------------------------------------------------------------- props ----------------------------------------------------------------------------- #custom-props-for-starflow-logs [source::.../starflow-app-logs...] TRANSFORMS-set_new_sourcetype = new_sourcetype TRANSFORMS-set_route_to_teamid_index = route_to_teamid_index ----------------------------------------------------------------------------- transforms ----------------------------------------------------------------------------- #custom-transforms-for-starflow-logs [new_sourcetype] REGEX = .* DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::aws:kinesis:starflow WRITE_META = true [route_to_teamid_index] REGEX = .*\/starflow-app-logs(?:-[a-z]+)?\/([a-zA-Z0-9]+)\/ SOURCE_KEY = source FORMAT = index::$1 DEST_KEY = _MetaData:Index WRITE_META = true I'm confident that both my props.conf and [new_sourcetype] stanza in transforms.conf are functioning correctly. The only issue seems to be with [route_to_teamid_index].

dj064 · ‎10-16-2024

Hi @PickleRick still it is not working.

dj064 · ‎10-15-2024

Thanks @PickleRick for suggestion. Shall I use below config? [source::.../starflow-app-logs*/...]

dj064 · ‎10-15-2024

Nope, I only included it for clarity while writing this post; it’s not part of my actual configuration. Note: I have removed that part from my post as well.

dj064 · ‎10-15-2024

I have been working on routing logs based on their source into different indexes. I configured below props.conf and transforms.conf on my HF, but it didn't worked. We currently follow the naming convention below for our CloudWatch log group names: /starflow-app-logs-<platform-name>/<team-id>/<app-name>/<app-environment-name> -------------------------------------------------------------------------- Example sources: -------------------------------------------------------------------------- us-east-1:/starflow-app-logs/sandbox/test/prod us-east-1:/starflow-app-logs-dev/sandbox/test/dev us-east-1:/starflow-app-logs-stage/sandbox/test/stage Note: We are currently receiving log data for the above use case from the us-east-1 region. -------------------------------------------------------------------------- Condition: -------------------------------------------------------------------------- If the source path contains <team-id>, logs should be routed to the respective index in Splunk. If the source path contains any <team-id>, its logs will be routed to the same <team-id>-based index, which already exists in our Splunk environment. -------------------------------------------------------------------------- props.conf -------------------------------------------------------------------------- [source::us-east-1:/starflow-app-logs*] TRANSFORMS-set_starflow_logging = new_sourcetype, route_to_teamid_index -------------------------------------------------------------------------- transforms.conf -------------------------------------------------------------------------- [new_sourcetype] REGEX = .* SOURCE_KEY = source DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::aws:kinesis:starflow WRITE_META = true [route_to_teamid_index] REGEX = us-east-1:\/starflow-app-logs(?:-[a-z]+)?\/([a-zA-Z0-9]+)\/ SOURCE_KEY = source FORMAT = index::$1 DEST_KEY = _MetaData:Index I’d be grateful for any feedback or suggestions to improve this configuration. Thanks in advance!

Posts	7
Solutions	1
Karma Given	7
Karma Received	0
Member Since	‎10-15-2024

Online Status	Offline
Date Last Visited	6 hours ago

Logging issues

Routing log data to different indexes based on the...

Logging issues

Re: Routing log data to different indexes based on...

Re: Routing log data to different indexes based on...

Re: Routing log data to different indexes based on...

Re: Routing log data to different indexes based on...

Re: Routing log data to different indexes based on...

Routing log data to different indexes based on the...