Hello, I have a simple query that run on the last 10 days of month, around 300k events something like: index=myindex RESPCODE=0 |bin span=1mon _time |eval length=len(POS_ENTRY_CODE) |search POS_ENTRY_CODE=8* AND length=2 |stats count The schedule return 6, which a little bit low than what I expected, so I hit Run to run the same query in ad-hoc, and it return 12, which is right. The time it took to run the schedule was 17 min and the time it took to run the query in adhoc was 19 min. My Splunk version is 7.2.6. So what wrong with my schedule alert and what should I do to mitigate this problem?   ... View more

This happen sometime on this DB and the only one that I experience this. The view, fields and value are pretty simple, my input also. But sometime it don't index certain fields that make it difficult to make statistical report. I checked the view and they certainly have those fields and value but the events are lacking them Another weird thing is some events are duplicate and some aren't. The events that are not duplicate is the one lacking fields. I never seen this before and don't know how to resovle this. ... View more

Hello, I have a sourcetype that have a default LINE_BREAKING and SHOULD_LINEMERGE=false, like so: Per my understanding, this mean it automatically extract each line as one event. But the indexed data is like this: The red event is correct with linecount=1, but most of the events have linecount=2, some have event more without line breaking. So what should I fix? ... View more

Hello, Our Splunk Enterprise structure are 1 Master, 2 Search Head and 4 Indexer Cluster. The Master will configure Forwarder Management and the deployment apps stay there. Now I want to index some logs file from a server (which already installed an UF) with csv structure, but not have csv extension. The log file is like this       api_key,api_method_name,bytes,cache_hit,client_transfer_time,connect_time,endpoint_name,http_method,http_status_code,http_version,oauth_access_token,package_name,package_uuid,plan_name,plan_uuid,pre_transfer_time,qps_throttle_value,quota_value,referrer,remote_total_time,request_host_name,request_id,request_time,request_uuid,response_string,service_definition_endpoint_uuid,service_id,service_name,src_ip,ssl_enabled,total_request_exec_time,traffic_manager,traffic_manager_error_code,uri,user_agent,org_name,org_uuid,sub_org_name,sub_org_uuid unknown,-,30,0,0.0,0.0,-,POST,596,HTTP/1.1,-,-,-,-,-,0.0,0,0,-,0.0,developer.napas.com.vn,1675641598.598_unknown_unknown,2023-02-05T23:59:58,dafeac38-123d-4bb7-aa1c-59680afbc0b2,596 Service Not Found (Proxy),-,unknown,-,10.244.1.0,1,0.0,tm-deploy-0-97674db57-smcdv,ERR_596_SERVICE_NOT_FOUND,/healthcheck,-,-,-,-,- unknown,-,30,0,0.0,0.0,-,POST,596,HTTP/1.1,-,-,-,-,-,0.0,0,0,-,0.0,developer.napas.com.vn,1675641608.030_unknown_unknown,2023-02-06T00:00:08,e4cd645a-5471-4097-baf0-67f90f4d2cee,596 Service Not Found (Proxy),-,unknown,-,10.244.1.0,1,0.001,tm-deploy-0-97674db57-smcdv,ERR_596_SERVICE_NOT_FOUND,/healthcheck,-,-,-,-,- unknown,-,30,0,0.0,0.0,-,POST,596,HTTP/1.1,-,-,-,-,-,0.0,0,0,-,0.0,developer.napas.com.vn,1675641618.607_unknown_unknown,2023-02-06T00:00:18,ee18e506-2ea5-4792-a586-f0274e6c823b,596 Service Not Found (Proxy),-,unknown,-,10.244.1.0,1,0.0,tm-deploy-0-97674db57-smcdv,ERR_596_SERVICE_NOT_FOUND,/healthcheck,-,-,-,-,- unknown,-,30,0,0.0,0.0,-,POST,596,HTTP/1.1,-,-,-,-,-,0.0,0,0,-,0.0,developer.napas.com.vn,1675641627.988_unknown_unknown,2023-02-06T00:00:27,5cc9f704-61a3-443c-b670-26373afe5502,596 Service Not Found (Proxy),-,unknown,-,10.244.1.0,1,0.0,tm-deploy-0-97674db57-smcdv,ERR_596_SERVICE_NOT_FOUND,/healthcheck,-,-,-,-,-       The log file is named as: access_worker5_2023_2_5.log or access_worker5_2023_2_5.log.1 Before config input.conf in my deployment I config props.conf and transfroms.conf in my Search Head in /splunk/etc/apps/search/local as props.conf       [mllog.new] CHARSET = UTF-8 INDEXED_EXTRACTIONS = csv KV_MODE = none LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true SHOULD_LINEMERGE = false category = Structured description = sourcetype for index csv disabled = false pulldown_type = true FIELD_NAMES = api_key,api_method_name,bytes,cache_hit,client_transfer_time,connect_time,endpoint_name,http_method,http_status_code,http_version,oauth_access_token,package_name,package_uuid,plan_name,plan_uuid,pre_transfer_time,qps_throttle_value,quota_value,referrer,remote_total_time,request_host_name,request_id,request_time,request_uuid,response_string,service_definition_endpoint_uuid,service_id,service_name,src_ip,ssl_enabled,total_request_exec_time,traffic_manager,traffic_manager_error_code,uri,user_agent,org_name,org_uuid,sub_org_name,sub_org_uuid TIMESTAMP_FIELDS = request_time REPORT-tibco-mllog-new = REPORT-tibco-mllog-new DATETIME_CONFIG = HEADER_FIELD_LINE_NUMBER = 1 FIELD_DELIMITER = , HEADER_FIELD_DELIMITER = , ###THIS IS FOR TESTING### [mllog.new2] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true category = Custom disalbe = false pulldown_type = 1         transfroms.conf       [REPORT-tibco-mllog-new] DELIMS = "," FIELDS = "api_key","api_method_name","bytes","cache_hit","client_transfer_time","connect_time","endpoint_name","http_method","http_status_code","http_version","oauth_access_token","package_name","package_uuid","plan_name","plan_uuid","pre_transfer_time","qps_throttle_value","quota_value","referrer","remote_total_time","request_host_name","request_id","request_time","request_uuid","response_string","service_definition_endpoint_uuid","service_id","service_name","src_ip","ssl_enabled","total_request_exec_time","traffic_manager","traffic_manager_error_code","uri","user_agent","org_name","org_uuid","sub_org_name","sub_org_uuid"         Then I config deployment apps like normal, with input.conf like this input.conf       [monitor:///u01/pv/log-1/data/trafficmanager/enriched/access/*] disabled = 0 index = myindex sourcetype = mllog.new ###THIS IS FOR TESTING### [monitor:///u01/pv/log-1/data/trafficmanager/enriched/access/*] disabled = 0 index = myindex sourcetype = mllog.new2         After configuration and restart. I ran 2 query index = myindex sourcetype = mllog.new -> 0 events index = myindex sourcetype = mllog.new2 -> Have events, but not with correct line breaking, some event have 1 line (correct) some events have 2 lines or even 257 lines (which clearly wrong), indexed header and don't have fields seperation.   So clearly I have config wrong somewhere, can someone point me to the right direction. ... View more

Hello, I have an deployment app that monitor log file from an external server that work fine since last year. But suddenly, since 26/1/2023 untill now, it can't index anything. Nothing changed from the server side or on my side either, the host still produce log file on a daily basis. I also request to check the connection and restart deployment client but no improvement. My input.config is: [monitor:///u01/pv/log-1/data/trafficmanager/enriched/access/*.log] disabled = 0 index = my index sourcetype = my sourcetype The example log file name is: access_worker_6_2023_01_26.log  I like to resolve this problem, even redo every step if I have to because this is urgent. And I like to know how to troubleshoot step by step to know where is the problem, and how to prevent this in the future.   ... View more

Hello, I have a csv file that have some summary stats from an index, but the requirement  is to show an sample event with all the info in that index. The CSV file have a hash number (from a account number), some calculated status. For example: PAN total_trans total_amount HASHPAN 1234******5678 15 15000 ABC123   The index have all the transaction and detail. We need to take an sample in the index from the csv and output to a new csv that have sample detail. Something like this: PAN total_trans total_amount HASHPAN _time TRACE TRANSACTIONID 1234******5678 15 15000 ABC123 xxxxxxx xxxxxx xxxxxxxxx   I don't really like to use join, because the index have a lot of events (around 32 mils events). Are there any elegants way to get the data? ... View more

Hello, I have a DBInput that have a database with a list of user with email and phone number, and people can make change to that DB, which include delete a row. The problem that I encounter is, the data that already indexed retain the deleted row in the DB, and thus the alert still send to that already deleted contact. So I want to find a way to reindex that db on a daily basis and delete the last indexed data. I have 3 solotions that I think of: 1. Setup an alert that run |delete daily and DBConnect can reindex (but I have to manually set the rising column check point to 0) 2. Batch input them daily and setup my search (it's a join in an alert) to search for -1d since it's not big of a data. 3. Join the table directly within SQL query when I indexing them, that way it'll always have the updated DB (but it'll tank on the DB server side) Which of these 3 solutions do you think is good? Or can you offer me an alternate, better solutions? ... View more

Hello, So I have a forwarder installed on a server and it show up on Clients in Forwarder Management. Then I create new app in depployment-apps, with local/input.conf like this   [monitor:///home/cnttm/Vibus/logTransit/application.log] crcSalt = <SOURCE> disable = false index = mynewindex [monitor:///home/cnttm/Vibus/logTransit/*.log] crcSalt = <SOURCE> disable = false index = mynewindex [monitor:///home/cnttm/Vibus/logTransit/*] crcSalt = <SOURCE> disable = false index = mynewindex [monitor:///home/cnttm/*] crcSalt = <SOURCE> disable = false index = mynewindex   The log directory is: /home/cnttm/Vibus/logTransit/application.log Then I create a server classes and apps, enable and restart it. But when I search index=mynewindex, I dont have any result, and I'm pretty sure we have log in that directory. Does anyone know anything wrong with my syntax? And how do I know/check if my deployment apps is working or not? ... View more