Hi,
I've installed the ServiceNow add-on v2.5.0 on a Search Head that is part of a distributed setup with 6 Indexers. I've configured the add-on with a service account to let it communicate with our ServiceNow installation and pull in incident, change and CMDB information. Normally, this search head uses a proxy and SSO to connect to Service Now but I've had that disabled to prevent issues from arising due to network complexity. I haven't updated the ServiceNow installation with the provided Dublin/Calgary/Eureka XML file since I'm only looking for pulling data in, not sending incidents/tickets/events back.
One of the database table names that I want to index contains 1059 rows so I've configured this database table name as a modular input. I configured collection at a 60 second interval, set "since when" to 2014-01-01 00:00:00 and I enabled the modular input. I can see in the logging that the URL it retrieves from (https://mycustomer.service-now.com/mytable.do?JSONv2&sysparm_query=sys_updated_on%3E=2014-01-01+00:00:00^ORDERBYsys_updated_on&sysparm_record_count=5000) picks up on 1059, but a seach in Splunk gives me only 1013 events. I've verified that if I manually curl the above URL from the search head that I do indeed get everything.
This is one of the events that were part of the JSON datastream but wasn't picked up by Splunk. (data is partially anonymized)
{"u_config_admin_group":"a738fecc1c56a1003615a9c3415190d0","checked_in":"","po_number":"","correlation_id":"","supported_by":"","u_responsible_vendor":"31ef66841c56a1003615a9c34151904e","u_layer_group":"compute","u_supply_offering_count":"2","first_discovered":"","owned_by":"","gl_account":"","managed_by":"","asset":"","u_standard":"true","maintenance_schedule":"","u_warranty_start":"","u_business_chain_count":"0","category":"","delivery_date":"","install_status":"7","u_status_updated":"2015-02-27 10:14:56","u_row_position":"","dns_domain":"","u_audit_comments":"","u_repair_contract_id":"02c798bc1c5ea1003615a9c341519003","u_cabinet_position":"","change_control":"","checked_out":"","purchase_date":"","order_date":"","u_maintenance_vendor":"31ef66841c56a1003615a9c34151904e","__status":"success","skip_sync":"false","lease_id":"","vendor":"","sys_id":"e94538bc1c9ea1003615a9c3415190a0","u_cabinet":"","u_active":"true","u_function_category":"myserverA","u_originating_vendor":"","sys_created_by":"john.smith","u_row_number":"","subcategory":"","u_support_offering_count":"5","u_audit_executed_datetime":"2012-12-04 16:56:17","start_date":"","comments":"","unverified":"false","location":"","u_cname":"","justification":"","u_rack_position_bottom":"","sys_domain":"global","u_configuration_item_count":"0","sys_mod_count":"3","cost_cc":"USD","u_tech_supported_by":"","u_service_offering_count":"0","monitor":"false","sys_updated_on":"2015-02-27 10:14:56","warranty_expiration":"","invoice_number":"","u_rack_position_top":"","cost":"","fqdn":"","u_system_category":"production","ip_address":"","u_business_service_count":"0","last_discovered":"","model_id":"","manufacturer":"","company":"","due":"","u_audit_ok_datetime":"2012-12-04 16:56:17","asset_tag":"LH200551","discovery_source":"","u_audit_status":"Executed OK","can_print":"false","u_standard_function":"","department":"","support_group":"","u_platform":"linux","sys_created_on":"2014-07-31 09:16:23","u_system_environment":"single-server","cost_center":"","short_description":"","sys_updated_by":"jsmith","name":"serverA","due_in":"","install_date":"2012-12-03 23:00:00","u_replaced_by":"","u_os_version":"rehel6-64 bit","assigned":"","u_os":"","u_audit_status_by":"471677c81c1aa1003615a9c3415190a9","serial_number":"","mac_address":"","assigned_to":"","model_number":"","u_audit_needed_datetime":"","schedule":"","sys_class_name":"u_cmdb_ci_logical_host","u_relation_log":"","attributes":"","fault_count":"0","operational_status":"1"},
Any idea on what's causing this and how to troubleshoot? DEBUG logging doesn't help much here.
... View more