Hi All, We are working in Splunk Cloud environment, I want to deploy custom the TIME_PREFIX configuration for one of the log source, we have tested the configuration in Splunk stand alone box, it works fine, we have deployed the same configuration on Splunk Cloud indexers and IDM node, but it's not working. Below is my props configuration. TIME_PREFIX = "time":\s*" MAX_TIMESTAMP_LOOKAHEAD = 128  sample value: , "time": "2022-12-24T02:36:55.9183013Z", My question is do I need to deploy this configuration on search heads as well.   Thanks, Bhaskar     ... View more

Hi All, We are currently in-progress of onboarding the okta identity cloud logs, we are using Splunk built add-on for okta identity cloud. when we configure input for test instance of okta cloud it works perfectly fine, but we are configuring the input for okta cloud production logs are not coming in. we have tried below steps. Disabling and re-enabling the input. Deleting and re-creating the input. Creating a new API input in Okta. Changing the configuration items to high and low values. Changed the interval to higher values. Reviewed internal logs for errors. Testing the API key locally (which was successful). Configured the API key in different heavy forwarder   While checking on okta side it shows rate limit warning.  Any help would be very appreciated.   Thanks, Bhaskar Chourasiya ... View more

Hi All, we are facing a issue in Splunk Add-on for Microsoft Cloud Services event hub input, there are multiple inputs we have created and almost all the inputs are collecting partial logs. We are checking the count of event at Azure Log Analytics workspace and at the same time checking events on Splunk there is random difference in event collection. There are no errors in internal logs, although we can see some warning messages, we tried increasing the ingestion pipeline to 4. tried disabling all the inputs but kept only one to check if that's making any issue.  Splunk deployment is single instance test environment where  32vCPU and 64 GB memory is assigned, storage is more than 800 IOPS. Not much of the application are installed.  Splunk support case is also opened but till now they haven't able to find any root cause. Need suggestions and inputs if someone else has faced such issue. Little back ground on architecture, we have multiple data sources (Azure Activity & AD) sending logs to one event hub and we are segregating the sourcetypes in splunk by transforming data based on category and resourceId.  Please help to resolve this issue.    Thanks Bhaskar   ... View more

Hi All, I am facing an issue related to time zone interpretation, one server which is configured with CET and sending log splunk cloud (in my best knowledge indexers are placed in GMT timezone). This server sends syslogs to SC4S servers configured with GMT time zone. Event Time value in splunk is being picked as per the raw event time. Since splunk indexers are GMT, SC4S is in GMT, I am getting time difference between event time (server time/ CET time zone) and index time (GMT time zone). please help, how can I resolve this issue of huge time difference in event time and index time.   Thanks, Bhaskar ... View more

Hi All, we have onboarded windows DHCP servers on splunk cloud by installing UFs on each server. DHCP server writes logs on local log file and universal forwarder sends logs direct to splunk cloud. The problem is logs are getting ingested to splunk with varied time difference. See the screenshot below, first log generated at 00:38 and indexed at 5:38 exact 5 hours difference where second log generated at  19:58 but indexed at 00:59 which has exact 7 hours of difference in event time (_time) and time in raw event but indexed at 00:59 and _time has been picked 00:58.   Please help to understand what can be the problem.   Thanks, Bhaskar ... View more