All Apps and Add-ons

Duplications from ServiceNow into Splunk

cardanid
New Member

It appears when trying to pull the sys_transaction table into Splunk (still looking at other tables), I am getting duplications.

1st issue I see is that the "sys_created_on" field from the Splunk ServiceNow app is not matching what is in the actual rest call to ServiceNow via the logs. Also it pulling the same records based on the SN search.

See screen shot attached.

[snow://syslog_transaction]
account = ServiceNow SAND2
duration = 60
filter_data = sysparm_query=sys_created_byCONTAINSsys_rest
id_field = sys_id
index = be03_service_now
since_when = 2019-09-19 00:00:00
table = syslog_transaction
timefield = sys_created_on
disabled = 0

2019-09-19 15:04:38,199 INFO pid=27829 tid=Thread-5 file=snow_job_factory.py:__call__:50 | End collecting data from table syslog_transaction for input syslog_transaction
2019-09-19 15:05:30,685 INFO pid=27829 tid=Thread-2 file=snow_job_factory.py:__call__:34 | Start collecting data from table syslog_transaction for input syslog_transaction
2019-09-19 15:05:30,685 INFO pid=27829 tid=Thread-2 file=snow_data_loader.py:_do_collect:160 | Initiating request to https://allstatesand2.service-now.com/api/now/table/syslog_transaction?sysparm_query=sys_created_byCONTAINSsys_rest&sysparm_display_value=all&sysparm_limit=1000&sysparm_exclude_reference_link=true&sysparm_query=sys_created_on>=2019-08-15+09:05:28^ORDERBYsys_created_on
2019-09-19 15:05:40,093 INFO pid=27829 tid=Thread-2 file=snow_data_loader.py:_do_collect:178 | Ending request to https://allstatesand2.service-now.com/api/now/table/syslog_transaction?sysparm_query=sys_created_byCONTAINSsys_rest&sysparm_display_value=all&sysparm_limit=1000&sysparm_exclude_reference_link=true&sysparm_query=sys_created_on>=2019-08-15+09:05:28^ORDERBYsys_created_on
2019-09-19 15:05:40,307 INFO pid=27829 tid=Thread-2 file=snow_data_loader.py:collect_data:150 | Data collection completed for input syslog_transaction. Got 1000 records from https://allstatesand2.service-now.com/syslog_transaction.
2019-09-19 15:05:40,476 INFO pid=27829 tid=Thread-2 file=snow_job_factory.py:__call__:50 | End collecting data from table syslog_transaction for input syslog_transaction
2019-09-19 15:06:30,686 INFO pid=27829 tid=Thread-1 file=snow_job_factory.py:__call__:34 | Start collecting data from table syslog_transaction for input syslog_transaction
2019-09-19 15:06:30,686 INFO pid=27829 tid=Thread-1 file=snow_data_loader.py:_do_collect:160 | Initiating request to https://allstatesand2.service-now.com/api/now/table/syslog_transaction?sysparm_query=sys_created_byCONTAINSsys_rest&sysparm_display_value=all&sysparm_limit=1000&sysparm_exclude_reference_link=true&sysparm_query=sys_created_on>=2019-08-15+09:05:28^ORDERBYsys_created_on
2019-09-19 15:06:37,861 INFO pid=27829 tid=Thread-1 file=snow_data_loader.py:_do_collect:178 | Ending request to https://allstatesand2.service-now.com/api/now/table/syslog_transaction?sysparm_query=sys_created_byCONTAINSsys_rest&sysparm_display_value=all&sysparm_limit=1000&sysparm_exclude_reference_link=true&sysparm_query=sys_created_on>=2019-08-15+09:05:28^ORDERBYsys_created_on
2019-09-19 15:06:38,077 INFO pid=27829 tid=Thread-1 file=snow_data_loader.py:collect_data:150 | Data collection completed for input syslog_transaction. Got 1000 records from https://allstatesand2.service-now.com/syslog_transaction.
2019-09-19 15:06:38,233 INFO pid=27829 tid=Thread-1 file=snow_job_factory.py:__call__:50 | End collecting data from table syslog_transaction for input syslog_transaction
0 Karma

cardanid
New Member

I am not able to upload screen shots

0 Karma

cardanid
New Member

This is interesting, its changing the times of the pull from somewhere.

** start_process_at>=2019-09-20+11:00:00^ORDERBYstart_process_at

2019-09-20 14:27:01,450 INFO pid=3635 tid=Thread-7 file=snow_data_loader.py:_do_collect:178 | Ending request to https://allstatesand2.service-now.com/api/now/table/syslog_transaction?sysparm_query=sys_created_byC...
2019-09-20 14:27:01,650 INFO pid=3635 tid=Thread-7 file=snow_data_loader.py:collect_data:150 | Data collection completed for input syslog_transaction. Got 1000 records from https://allstatesand2.service-now.com/syslog_transaction.

**start_process_at>=2019-08-15+09:05:28^ORDERBYstart_process_at
2019-09-20 14:28:02,114 INFO pid=3635 tid=Thread-1 file=snow_data_loader.py:_do_collect:178 | Ending request to https://allstatesand2.service-now.com/api/now/table/syslog_transaction?sysparm_query=sys_created_byC...

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...