EVENT Time 3/18/24 12:58:45.880 PM { "Timestamp": "2024-03-18T10:58:45.880+00:00", "Level": "ERR", "Message": "The HTTP status code of the response was not expected (404).\n\nStatus: 404\nResponse: \n{\"IsError\":false,\"IsValidationError\":false,\"IsNotFound\":true,\"IsDisplayError\":false,\"Type\":\"http://glc-api-integration-tst.cs.sanlam.co.za/intermediary-accreditations\",\"Status\":404,\"Title\":\"Not found error\",\"Detail\":\"Intermediary accreditation Product not found. Search parameters: WhiteLabel: GLC, ProductCode: SANFRP01, GuaranteedIncome: False, GuaranteedCapital: False, GuaranteedAnnuitySingleLife: False, GuaranteedAnnuityJointLife: False \",\"Errors\":[],\"ValidationErrors\":{}}", "Properties": { "RequestId": "0HN271KO7M0H2:00000007", "RequestPath": "/products/fit-and-proper", "CorrelationId": "1e5975b4-d3e3-42c5-9bed-48760f4002d8", "ConnectionId": "0HN271KO7M0H2", "MachineName": "k8s-glc-api-bff-portal-tst-7b67f985b5-xqspv", "SolutionName": "PortalBFF", "Environment": "tst", "LoggerName": "PortalBFF", "ApplicationName": "PortalBFF", "ThreadId": "1", "ProcessId": "1", "ProcessUserId": "root", "SiteName": "PortalBFF" }, "Exception": { "ExceptionSource": "API.Client.3rdParty.Integration", "ExceptionType": "Api.Client.ThirdParty.Integration.V1.ApiException", "ExceptionMessage": "The HTTP status code of the response was not expected (404).\n\nStatus: 404\nResponse: \n{\"IsError\":false,\"IsValidationError\":false,\"IsNotFound\":true,\"IsDisplayError\":false,\"Type\":\"http://glc-api-integration-tst.cs.sanlam.co.za/intermediary-accreditations\",\"Status\":404,\"Title\":\"Not found error\",\"Detail\":\"Intermediary accreditation Product not found. Search parameters: WhiteLabel: GLC, ProductCode: SANFRP01, GuaranteedIncome: False, GuaranteedCapital: False, GuaranteedAnnuitySingleLife: False, GuaranteedAnnuityJointLife: False \",\"Errors\":[],\"ValidationErrors\":{}}", "StackTrace": " at Api.Client.ThirdParty.Integration.V1.IntegrationClient.IntermediaryAccreditationAsync(IntermediaryAccreditationProductRequestModel model, CancellationToken cancellationToken)\n at Providers.ExternalIntegrations.FitAndProper.V1.FitAndProperProvider.IntermediaryAccreditation(IntermediaryAccreditationProductRequestModel model, CancellationToken cancellationToken) in /src/Providers/ExternalIntegrations/FitAndProper/V1/FitAndProperProvider.cs:line 14\n at Services.ExternalIntegrations.FitAndProper.Queries.V1.GetFitAndProperQueryHandler.Handle(GetFitAndProperQuery request, CancellationToken cancellationToken) in /src/Services/ExternalIntegrations/FitAndProper/Queries/V1/GetFitAndProperQuery.cs:line 37\n at Api.Utilities.Behaviours.PerformanceBehaviour`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken)\n at Api.Utilities.Behaviours.ValidationBehavior`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken)\n at Api.Controller.Products.V1.ProductsController.GetFitAndProper(GetFitAndProperQuery request, CancellationToken cancellationToken) in /src/Api/Controller/Products/V1/ProductsController.cs:line 53\n at lambda_method954(Closure , Object )\n at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)\n at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\n at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.ClaimsMiddleware.Invoke(HttpContext context, IUserService userService)\n at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.SwaggerUiIndexMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.RedirectToIndexMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.OpenApiDocumentMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.SwaggerBasicAuthMiddleware.InvokeAsync(HttpContext context)\n at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)\n at Api.Utilities.Middleware.RequestBodyMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.LoggingMiddleware.Invoke(HttpContext context)", "FileName": null, "MethodName": "Api.Client.ThirdParty.Integration.V1.IntegrationClient+<IntermediaryAccreditationAsync>d__15", "Line": 0, "Data": null }, "RequestBody": null, "Additional": null } { "Timestamp": "2024-03-18T10:58:46.979+00:00", "Level": "ERR", "Message": "The HTTP status code of the response was not expected (404).\n\nStatus: 404\nResponse: \n{\"IsError\":false,\"IsValidationError\":false,\"IsNotFound\":true,\"IsDisplayError\":false,\"Type\":\"http://glc-api-integration-tst.cs.sanlam.co.za/intermediary-accreditations\",\"Status\":404,\"Title\":\"Not found error\",\"Detail\":\"Intermediary accreditation Product not found. Search parameters: WhiteLabel: GLC, ProductCode: SANFRP01, GuaranteedIncome: False, GuaranteedCapital: False, GuaranteedAnnuitySingleLife: False, GuaranteedAnnuityJointLife: False \",\"Errors\":[],\"ValidationErrors\":{}}", "Properties": { "RequestId": "0HN271KO7M0H6:00000002", "RequestPath": "/products/fit-and-proper", "CorrelationId": "31069097-572e-42cd-b47d-688b63260705", "ConnectionId": "0HN271KO7M0H6", "MachineName": "k8s-glc-api-bff-portal-tst-7b67f985b5-xqspv", "SolutionName": "PortalBFF", "Environment": "tst", "LoggerName": "PortalBFF", "ApplicationName": "PortalBFF", "ThreadId": "1", "ProcessId": "1", "ProcessUserId": "root", "SiteName": "PortalBFF" }, "Exception": { "ExceptionSource": "API.Client.3rdParty.Integration", "ExceptionType": "Api.Client.ThirdParty.Integration.V1.ApiException", "ExceptionMessage": "The HTTP status code of the response was not expected (404).\n\nStatus: 404\nResponse: \n{\"IsError\":false,\"IsValidationError\":false,\"IsNotFound\":true,\"IsDisplayError\":false,\"Type\":\"http://glc-api-integration-tst.cs.sanlam.co.za/intermediary-accreditations\",\"Status\":404,\"Title\":\"Not found error\",\"Detail\":\"Intermediary accreditation Product not found. Search parameters: WhiteLabel: GLC, ProductCode: SANFRP01, GuaranteedIncome: False, GuaranteedCapital: False, GuaranteedAnnuitySingleLife: False, GuaranteedAnnuityJointLife: False \",\"Errors\":[],\"ValidationErrors\":{}}", "StackTrace": " at Api.Client.ThirdParty.Integration.V1.IntegrationClient.IntermediaryAccreditationAsync(IntermediaryAccreditationProductRequestModel model, CancellationToken cancellationToken)\n at Providers.ExternalIntegrations.FitAndProper.V1.FitAndProperProvider.IntermediaryAccreditation(IntermediaryAccreditationProductRequestModel model, CancellationToken cancellationToken) in /src/Providers/ExternalIntegrations/FitAndProper/V1/FitAndProperProvider.cs:line 14\n at Services.ExternalIntegrations.FitAndProper.Queries.V1.GetFitAndProperQueryHandler.Handle(GetFitAndProperQuery request, CancellationToken cancellationToken) in /src/Services/ExternalIntegrations/FitAndProper/Queries/V1/GetFitAndProperQuery.cs:line 37\n at Api.Utilities.Behaviours.PerformanceBehaviour`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken)\n at Api.Utilities.Behaviours.ValidationBehavior`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken)\n at Api.Controller.Products.V1.ProductsController.GetFitAndProper(GetFitAndProperQuery request, CancellationToken cancellationToken) in /src/Api/Controller/Products/V1/ProductsController.cs:line 53\n at lambda_method954(Closure , Object )\n at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)\n at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\n at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.ClaimsMiddleware.Invoke(HttpContext context, IUserService userService)\n at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.SwaggerUiIndexMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.RedirectToIndexMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.OpenApiDocumentMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.SwaggerBasicAuthMiddleware.InvokeAsync(HttpContext context)\n at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)\n at Api.Utilities.Middleware.RequestBodyMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.LoggingMiddleware.Invoke(HttpContext context)", "FileName": null, "MethodName": "Api.Client.ThirdParty.Integration.V1.IntegrationClient+<IntermediaryAccreditationAsync>d__15", "Line": 0, "Data": null }, "RequestBody": null, "Additional": null } { "Timestamp": "2024-03-18T10:58:49.080+00:00", "Level": "ERR", "Message": "The HTTP status code of the response was not expected (404).\n\nStatus: 404\nResponse: \n{\"IsError\":false,\"IsValidationError\":false,\"IsNotFound\":true,\"IsDisplayError\":false,\"Type\":\"http://glc-api-integration-tst.cs.sanlam.co.za/intermediary-accreditations\",\"Status\":404,\"Title\":\"Not found error\",\"Detail\":\"Intermediary accreditation Product not found. Search parameters: WhiteLabel: GLC, ProductCode: SANFRP01, GuaranteedIncome: False, GuaranteedCapital: False, GuaranteedAnnuitySingleLife: False, GuaranteedAnnuityJointLife: False \",\"Errors\":[],\"ValidationErrors\":{}}", "Properties": { "RequestId": "0HN271KO7M0H8:00000002", "RequestPath": "/products/fit-and-proper", "CorrelationId": "d7ccaaaa-1a6d-4dbe-9b80-ad63fc69a1c5", "ConnectionId": "0HN271KO7M0H8", "MachineName": "k8s-glc-api-bff-portal-tst-7b67f985b5-xqspv", "SolutionName": "PortalBFF", "Environment": "tst", "LoggerName": "PortalBFF", "ApplicationName": "PortalBFF", "ThreadId": "1", "ProcessId": "1", "ProcessUserId": "root", "SiteName": "PortalBFF" }, "Exception": { "ExceptionSource": "API.Client.3rdParty.Integration", "ExceptionType": "Api.Client.ThirdParty.Integration.V1.ApiException", "ExceptionMessage": "The HTTP status code of the response was not expected (404).\n\nStatus: 404\nResponse: \n{\"IsError\":false,\"IsValidationError\":false,\"IsNotFound\":true,\"IsDisplayError\":false,\"Type\":\"http://glc-api-integration-tst.cs.sanlam.co.za/intermediary-accreditations\",\"Status\":404,\"Title\":\"Not found error\",\"Detail\":\"Intermediary accreditation Product not found. Search parameters: WhiteLabel: GLC, ProductCode: SANFRP01, GuaranteedIncome: False, GuaranteedCapital: False, GuaranteedAnnuitySingleLife: False, GuaranteedAnnuityJointLife: False \",\"Errors\":[],\"ValidationErrors\":{}}", "StackTrace": " at Api.Client.ThirdParty.Integration.V1.IntegrationClient.IntermediaryAccreditationAsync(IntermediaryAccreditationProductRequestModel model, CancellationToken cancellationToken)\n at Providers.ExternalIntegrations.FitAndProper.V1.FitAndProperProvider.IntermediaryAccreditation(IntermediaryAccreditationProductRequestModel model, CancellationToken cancellationToken) in /src/Providers/ExternalIntegrations/FitAndProper/V1/FitAndProperProvider.cs:line 14\n at Services.ExternalIntegrations.FitAndProper.Queries.V1.GetFitAndProperQueryHandler.Handle(GetFitAndProperQuery request, CancellationToken cancellationToken) in /src/Services/ExternalIntegrations/FitAndProper/Queries/V1/GetFitAndProperQuery.cs:line 37\n at Api.Utilities.Behaviours.PerformanceBehaviour`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken)\n at Api.Utilities.Behaviours.ValidationBehavior`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken)\n at Api.Controller.Products.V1.ProductsController.GetFitAndProper(GetFitAndProperQuery request, CancellationToken cancellationToken) in /src/Api/Controller/Products/V1/ProductsController.cs:line 53\n at lambda_method954(Closure , Object )\n at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)\n at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\n at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.ClaimsMiddleware.Invoke(HttpContext context, IUserService userService)\n at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.SwaggerUiIndexMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.RedirectToIndexMiddleware.Invoke(HttpContext context)\n at NSwag.AspNetCore.Middlewares.OpenApiDocumentMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.SwaggerBasicAuthMiddleware.InvokeAsync(HttpContext context)\n at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)\n at Api.Utilities.Middleware.RequestBodyMiddleware.Invoke(HttpContext context)\n at Api.Utilities.Middleware.LoggingMiddleware.Invoke(HttpContext context)", "FileName": null, "MethodName": "Api.Client.ThirdParty.Integration.V1.IntegrationClient+<IntermediaryAccreditationAsync>d__15", "Line": 0, "Data": null }, "RequestBody": null, "Additional": null } { ... View more

Yes, this is specific to one sourcetype. I have confirmed it has been edited as follows: [glc:api:bff:tst] CHARSET=UTF-8 LINE_BREAKER=([\n\r]+)\{[\n\r]*"Timestamp" MAX_TIMESTAMP_LOOKAHEAD=32 NO_BINARY_CHECK=true SHOULD_LINEMERGE=false category=Custom description=test disabled=false pulldown_type=true TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3Q%:z TIME_PREFIX="Timestamp":\s+"   I have attached a screenshot of the events in Splunk and also a .txt with the content of one of the events. You will notice more than one entry in this single event. ... View more

Thanks, it seems to be breaking at the right place, but I'm still getting multiple entries as one event. Doesn't seem to be splitting them into separate events for each of the following entries:  { "Timestamp": "2024-03-18T10:57:17.096+00:00", } ... View more

I tried a different server in the same IP range now to test and the method you mentioned worked just fine.  So was the port being blocked. Thanks for the help. ... View more

Thanks for the response. I have tried the following, but it times out. I assume its a port issue. [license] manager_uri = https://servername:8089 ... View more

Hi Guiseppe, My initial response was to create dedicated dashboards as you mentioned as well. But thought perhaps someone had another idea. Basically all users belong to the same role, they can see all events for index=api. But the admin would like to limit access to the org field. So some users can only see org=x and some can only see org=y   ... View more

Hi, I selected all time and still nothing. The props.conf is as follows as per manjunathmeti   [sanport:dcm] SHOULD_LINEMERGE = true INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false TIMESTAMP_FIELDS = time ... View more

Thanks, I'll give that a try. ... View more

Thanks, I've tried that but it doesnt seem to work. Maybe it's related to the search query? Basically there are 2 events that show up every hour (polling cycle seems to be 60 minutes for these events).  I need the initial event that shows those 2 events, and then only again if there is a new entry/event with a different user. See attached screenshots of search and alert settings. I have set it to supress for 24 hours for now just to stop the alerts continuously being emailed. ... View more

@woodcock There are only 30 logs in this directory. I have enabled the crcSalt now as well. Lets see if that makes a difference. ... View more

Below is the content of the inputs.conf The whole log directory is specified, but its always just picked up the original .log file which is fine. [monitor:///WebSphere8/applications/dev/psiberworks/logs] disabled = false whitelist = .log$ crcSalt = SOURCE index = ibm_was_app_psi-was8-dev-01 time_before_close = 1 ... View more

I found a similar issue here : https://answers.splunk.com/answers/680732/splunk-skips-or-delays-indexing-of-the-log-file-du.html Made the change as specified : time_before_close = 1 But doesn't look like it helped. Forwarder version is 7.0.3 Unless I need to wait until the log rolls again at midnight tonight? ... View more