About cdstealer

cdstealer · ‎05-21-2023

Hi @richgalloway, Apologies for the late reply. It's a single instance, which is why I'm stuck. I don't have anything special setup, most things are at default. It's not a file permission thing from what I can see.

cdstealer · ‎05-20-2023

Me too. Nothing regarding errors are logged, but it does product over 1000 lines for cron schedules. I don't know if the OP has this, but no indices are detected on the setup page either. I'm running splunk 9.0.3 and nix 8.9.0.

cdstealer · ‎10-27-2022

Hi @joshiro, Apologies for the delay in replying. The issue looks to have stopped and I have my suspicions that something was changed on the DB server (quite recently) as nothing has changed from a Splunk POV. What has changed I couldn't tell you. If I do uncover it, I'll update here. The only thing I could suggest if updating your version of the app. I couldn't get 3.10 to work, so stuck with 3.9.0. Sorry it's not a fix. Steve

cdstealer · ‎04-18-2022

Thanks @tscroggins I see there is an update to db connect. I will get that done and see what happens 🙂

cdstealer · ‎04-05-2022

Hi, I'm getting these errors in splunkd.log each time the query is executed. 04-05-2022 18:01:48.750 +0100 ERROR ExecProcessor [8917 ExecProcessorSchedulerThread] - message from "/opt/splunk/etc/apps/splunk_app_db_connect/linux_x86_64/bin/dbxquery.sh" 17:01:48.750 [metrics-logger-reporter-1-thread-1] INFO com.splunk.dbx.connector.health.impl.ConnectionPoolMetricsLogReporter - type=TIMER, name=unnamed_pool_-382175356_jdbc__jtds__sqlserver__//servername__212/table;useCursors__true;domain__xxx.com;useNTLMv2__true.pool.Wait, count=12, min=0.120249, max=36.824436, mean=1.0705702234360484, stddev=0.028345392065423972, p50=1.06918, p75=1.06918, p95=1.06918, p98=1.06918, p99=1.06918, p999=1.648507, m1_rate=2.79081711035706E-30, m5_rate=1.1687825901066073E-8, m15_rate=2.6601992470705972E-5, mean_rate=5.566605761092861E-4, rate_unit=events/second, duration_unit=milliseconds 04-05-2022 18:01:48.750 +0100 ERROR ExecProcessor [8917 ExecProcessorSchedulerThread] - message from "/opt/splunk/etc/apps/splunk_app_db_connect/linux_x86_64/bin/dbxquery.sh" 17:01:48.750 [metrics-logger-reporter-1-thread-1] INFO c.s.d.c.h.i.ConnectionPoolMetricsLogReporter - type=TIMER, name=unnamed_pool_-382175356_jdbc__jtds__sqlserver__//servername__212/mantis;useCursors__true;domain__xxx.com;useNTLMv2__true.pool.Wait, count=12, min=0.120249, max=36.824436, mean=1.0705702234360484, stddev=0.028345392065423972, p50=1.06918, p75=1.06918, p95=1.06918, p98=1.06918, p99=1.06918, p999=1.648507, m1_rate=2.79081711035706E-30, m5_rate=1.1687825901066073E-8, m15_rate=2.6601992470705972E-5, mean_rate=5.566605761092861E-4, rate_unit=events/second, duration_unit=milliseconds Unfortunately I can see nothing pertaining to what the actual error is. If I use SQL Explorer, I can connect and pull data back without issue. However, the data that is collected is very sporadic if at all. We have a second DB connection running the same query etc without issue. We're using Splunk 8.2.3.2 and db_connect 3.7.0 TIA Steve

cdstealer · ‎06-25-2021

We're running 8.1.3 currently.

cdstealer · ‎06-25-2021

Hi Richard, The regex is defined in transforms and called by props. Apologies, that would be useful 🙂 Added to the top

cdstealer · ‎06-25-2021

When using PCRE regex to split a field into components, I find it frustrating. I know my regex works as I've validated this in both regex101 and debuggex. (?:.*?)(?P<ClientIP>(?:\d{1,3}\.){3}\d{1,3}|(?:(?:[0-9a-f]{1,4}(?::+)?){0,7}:+[0-9a-f]+))[,\n\r]+(?:(?:[\+](?P<LB_IP>[^:](?:\d{1,3}\.){3}\d{1,3}):(?P<LB_Port>\d+)))? So this is extracting details from the IIS X_Forwarded_For field. The supplied log data that was parsed extracted perfectly on both platforms and even using "grep -P". But in Splunk, I only get a full extraction when the following format is observed. 123.123.123.123,+123.123.123.123,+123.123.123.123:12345 If the final ip:port is missing from the event, only the first IP is captured 123.123.123.123,+123.123.123.123 I've had similar experiences over the years with Splunk, so I'm wondering if my regex fu is rubbish, regex validators are wrong or splunk has a bug that's never been fixed. TIA Steve

cdstealer · ‎06-09-2021

So after a little more reading, this is a known issue with DBConnect. The official work around is to use KV_MODE and then apply a search time extraction for the problematic fields. auditMessage="(?P<auditMessage>.+?)",

cdstealer · ‎05-25-2021

HI Rich, Thanks for the response. I had already done that, works in reg101 but not in splunk I've also come up with (?:([\w+]+)=(?:\")(.+?)(?:\",\s|\"$)) which does exactly what I want on reg101 , but again doesn't work the same in Splunk.

cdstealer · ‎05-25-2021

Hi, I'm struggling to get a complete extraction on any fields that contain double quotes. The payload: 2021-05-25 07:59:04.000, auditId="17864172953", groups_groupId="4639", groupName="some group name", people_personId="625841", users_userId="152321", userLogin="field-removed", userStaffFlag="false", auditIP="111.222.333.444", auditMod="Module", auditMessage="Module: "mod1" is not present in a check, Module: "mod2" is not present in a check, Module: "mod3" is not present in a check, Module: "mod4" is not present in a check, Module: "mod5" is not present in a check, Module: "mod6" is not present in a check, Module: "mod7" is not present in a check, Module: "mod8" is not present in a check, Module: "mod9" is not present in a check, Module: "mod10" is not present in a check", auditDate="2021-05-25 07:59:04.0", auditType="info", auditRID="88827e1f-d157-46d5-b1b4-20b91d4440a4", auditMicroSeconds="0.0000" In this example, it's the "auditMessage" key that will not extract completely and stops at the first "Module: "<-- The REGEX (\w+)="(.*?)" Gets me most of the way there (regx101), but doesn't work in splunk. My Fu has failed me.

cdstealer · ‎12-09-2020

Hi, I searched and found several tickets regarding my situation, but all lead to nowhere. So, my situation... Unfortunately we have a few logs that mix formats eg starts in plain text and then contains a json payload. The events are <4000 chars, so I can't see where the truncation is happening. I've also tried specifying the following in props/transforms with no difference.: TRUNCATE = 100000 MAXEVENTS = 500 MAXCHARS = 100000 DEPTH_LIMIT = 5000 I'm probably missing something obvious. 🙂 My current props/transforms for this sourcetype are: [opt:gateway] KV_MODE = none TIME_PREFIX = ^ TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE_DATE = true TRANSFORMS-opt_json2 = optimus_dll1,optimus_dll2 SEDCMD-eol = s/\\r\\n//g LINE_BREAKER=([\r\n]+) [optimus_dll1] REGEX = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s+\[\S+\]\s+(?P<LogLevel>[^ ]\w+)\s+(?P<OptimusDLL>[^ ]+) WRITE_META = true REPEAT_MATCH = false FORMAT = LogLevel::$1 OptimusDLL::$2 [optimus_dll2] REGEX = ^(?:[^$]*)\s-\s(?P<json>.+) FORMAT = json::$1 WRITE_META = true REPEAT_MATCH = false SOURCE_KEY = _raw DEPTH_LIMIT = 5000 optimus_dll1 is extracted as expected, optimus_dll2 only grabs the first 1000 chars of the match. Using the regex inline via rex, via regex101 or cmdline all extract the full field. TIA Steve

cdstealer · ‎10-08-2020

Solving my own question 🙂 Base search is now: index=production sourcetype="audit" environ::LV | table * and panel searches are: | search (field=value OR field=value OR field=value) field=value | appendpipe [stats count AS total by _time] | appendpipe [search auditType="error" | stats count AS error by _time] | appendpipe [where auditMicroSeconds>3 | stats count AS over BY _time] | appendpipe [search ("data retrieval" AND "failed") | stats count AS failed BY _time] | timechart span=30s count(error) AS Error count(over) AS Over3s count(failed) AS Failed count(total) AS Total Data is now being presented as expected 🙂

cdstealer · ‎10-07-2020

Hey, Thanks for the suggestion 🙂 Unfortunately it doesn't work 😞 I had to remove the OR X and AND X from the searchmatch and then it just returned the same results as the original. Appreciated though.

cdstealer · ‎10-07-2020

Hi, I'm trying to populate a dashboard using a base search and then pulling multiple stats from those results. base search: index=production sourcetype="audit" environ::LV inline search: | appendpipe [ stats count AS Total by _time] | appendpipe [ search ("Error:" OR auditType="error") | stats count AS error by _time] | appendpipe [ where auditMicroSeconds>3 | stats count AS Over BY _time] | appendpipe [ search ("data retrieval" AND "failed") | stats count AS failed BY _time] | timechart span=30s count(Total) AS Total count(error) AS Error count(Over) AS Over But it just doesn't work. Hope this makes sense. TIA Steve

cdstealer · ‎09-08-2020

Hi, It looks like some weird file permission issue. Although everything in /opt/splunk is readable by splunk, it still couldn't see props.conf. Which of course makes no sense when the working searchhead has the exact same perms and works without issue. On one of the affected search heads, I did a chmod -R 755 on /opt/splunk/etc/apps/* and it started working immediately on both affected servers??? Very strange. I'll mark this as resolved 🙂 Thanks Steve

cdstealer · ‎09-08-2020

Will do 🙂

cdstealer · ‎09-08-2020

OK, I fixed the field extractions by converting to indextime rather than searchtime. However, any searchtime functions eg calculated fields are still being ignored by the 2 searchheads. I'll keep digging.

cdstealer · ‎09-07-2020

Hi Soutamo, We don't run a multisite due to its scale and function. So I have the deployer and a search head in one location and then 2 search heads in the other location. These are then setup behind a load balancer which will fail to the single searchhead should the primary location fail.

cdstealer · ‎09-07-2020

Hi Nisha, Thanks for the query.. this is the output on both a working and non-working searchhead. id = https://127.0.0.1:8089/servicesNS/nobody/app_name/configs/conf-props/sourcetype title = sourcetype updated = 1970-01-01T01:00:00+01:00 disabled = false ADD_EXTRA_TIME_FIELDS = true ANNOTATE_PUNCT = true AUTO_KV_JSON = true BREAK_ONLY_BEFORE_DATE = true CHARSET = UTF-8 DATETIME_CONFIG = /etc/datetime.xml DEPTH_LIMIT = 1000 EVAL-duration = hour * 3600 + minute * 60 + second + precise/10000000 EXTRACT-fields = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} \[\d+\] \[(?P<ClientIP>[^ ]*) $\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$\] INFO\s+(?P<GingerClass>[^ ]*) - Start: (?P<JobStartTime>[^;]*);Reseller:\s(?P<ResellerName>[^;]*);ResellerUnit: (?P<ResellerUnit>[^;]*);(?P<JobGUID>[^;]*);(?P<UserGUID>[^;]*)?;(?P<ApiMethod>[^;]*);Duration: (?P<hour>[^:]\d+):(?P<minute>[^:]\d+):(?P<second>[^\.]\d+)\.(?P<precise>[^;]\d+);(?P<RegOutcome>[^;]*)?;(?P<FailReason>[^$]*)? LEARN_MODEL = true LEARN_SOURCETYPE = true LINE_BREAKER_LOOKBEHIND = 100 MATCH_LIMIT = 100000 MAX_DAYS_AGO = 2000 MAX_DAYS_HENCE = 2 MAX_DIFF_SECS_AGO = 3600 MAX_DIFF_SECS_HENCE = 604800 MAX_EVENTS = 256 MAX_TIMESTAMP_LOOKAHEAD = 128 SHOULD_LINEMERGE = true TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N TIME_PREFIX = ^ TRUNCATE = 10000 app = app_name author = nobody detect_trailing_nulls = false maxDist = 100 splunk_server = searchhead1 The actual props.conf is: [sourcetype] TIME_PREFIX = ^ TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N EXTRACT-fields = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} \[\d+\] \[(?P<ClientIP>[^ ]*) $\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$\] INFO\s+(?P<GingerClass>[^ ]*) - Start: (?P<JobStartTime>[^;]*);Reseller:\s(?P<ResellerName>[^;]*);ResellerUnit: (?P<ResellerUnit>[^;]*);(?P<JobGUID>[^;]*);(?P<UserGUID>[^;]*)?;(?P<ApiMethod>[^;]*);Duration: (?P<hour>[^:]\d+):(?P<minute>[^:]\d+):(?P<second>[^\.]\d+)\.(?P<precise>[^;]\d+);(?P<RegOutcome>[^;]*)?;(?P<FailReason>[^$]*)? EVAL-duration = hour * 3600 + minute * 60 + second + precise/10000000

cdstealer · ‎09-03-2020

Hi, A bit of a strange one that I can't workout. I have a deployer server and a search head in one DC and 2 searchheads in another DC. They are all part of the searchhead cluster and all share the same configs. My problem is that the searchhead app has been deployed to all the searchheads. The 2 searchheads located in the same DC have the app and correct configs, but don't perform any field extractions. Interestingly, if I open an event and select extract field, the parser sees the fields? The searchhead on its own performs as expected. I can see no errors. Running btool confirms the file is also correct. It's the first time I've ever come across this. TIA Steve

cdstealer · ‎11-15-2019

Hi, I'm generating a stats (csv) file that is updated every second. The log has no errors/skips, but I've found that if I don't specify an interval within inputs.conf it will miss randomly and/or part read the last event as it's written. If I set an interval, say 10 seconds, then I get a missed event every 10 seconds without fail. I'm using collectl to populate the file. So I guess my question is; is it possible to skip the last event, but read it on the next run? Here are my inputs and props for completeness. inputs.conf [monitor:///var/tmp] whitelist = sysRes-log-\d{8}\.tab disabled = false index = os host = logserver sourcetype = sysStats #multiline_event_extra_waittime = true recursive = false props.conf [sysStats] DATETIME_CONFIG = FIELD_DELIMITER = space INDEXED_EXTRACTIONS = csv KV_MODE = none LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true PREAMBLE_REGEX = (^##|^#\s) SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = Date,Time TIME_FORMAT = %Y%m%d %H:%M:%S BREAK_ONLY_BEFORE_DATE = true category = Structured description = Space value format. Set header and other settings in "Delimited Settings" disabled = false pulldown_type = true

cdstealer · ‎04-16-2019

Hi daluoc, You can't specify the extracted field name more than once. So you would need to specify each condition followed by the field extraction. eg (no\sIKE\sconfig\sfound\sfor\s\d+\.\d+\.\d+\.\d+\.\.\.|received\sinvalid\sIKE\sheader\sfrom\s)(?P<unauthorized>[^(,|\s)]+) (no\sIKE\sconfig\sfound\sfor\s\d+\.\d+\.\d+\.\d+\.\.\. is the first condition. Then add a "|" as a separator and then received\sinvalid\sIKE\sheader\sfrom\s) as the second then followed by the extraction (?P<unauthorized>[^(,|\s)]+) You can | numerous conditions together and they will populate the same field. Hope that makes sense. Also apologies for the necropost 😉

cdstealer · ‎08-30-2018

Hi, Sorry, I probably should have explained that I know python and about out of index errors 🙂 But I need to know what the issue is. Cheers

cdstealer · ‎08-30-2018

Hi, I've installed splunk-add-on-for-tenable both 5.1.2 & 5.1.4, but neither work. Al I see in the ta_nessus.log is: 2018-08-30 13:20:31,950 INFO pid=6088 tid=MainThread file=nessus.py:main:260 | Start nessus TA 2018-08-30 13:20:32,039 INFO pid=6088 tid=MainThread file=nessus_config.py:get_nessus_conf:71 | Try to get encrypted proxy username & password 2018-08-30 13:20:32,040 INFO pid=6088 tid=MainThread file=nessus.py:get_nessus_modinput_configs:142 | Set loglevel to ERROR 2018-08-30 13:20:34,036 ERROR pid=6088 tid=MainThread file=nessus_data_collector.py:_collect_scan_data:300 | Traceback (most recent call last): File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 294, in _collect_scan_data page_size) File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 245, in _collect_scan_data_of_one_scan scan_info) File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 217, in _collect_one_host_scan_info if port_items[2]: IndexError: list index out of range 2018-08-30 13:20:35,851 ERROR pid=6088 tid=MainThread file=nessus_data_collector.py:_collect_scan_data:300 | Traceback (most recent call last): File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 294, in _collect_scan_data page_size) File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 245, in _collect_scan_data_of_one_scan scan_info) File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 217, in _collect_one_host_scan_info if port_items[2]: IndexError: list index out of range 2018-08-30 13:20:37,080 ERROR pid=6088 tid=MainThread file=nessus_data_collector.py:_collect_scan_data:300 | Traceback (most recent call last): File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 294, in _collect_scan_data page_size) File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 245, in _collect_scan_data_of_one_scan scan_info) File "/opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus_data_collector.py", line 217, in _collect_one_host_scan_info if port_items[2]: IndexError: list index out of range Any Ideas? T.I.A.

Posts	111
Solutions	10
Karma Given	17
Karma Received	28
Member Since	‎08-28-2011

Online Status	Offline
Date Last Visited	‎08-10-2023 02:55 PM

Why am I getting Splunk DB Connect ERROR Without E...

Splunkism Regex?

DB Connect KEY=VALUE Extraction Fail

Regex Truncation vs Rex

Multiple Stats from Base Search

props deployed on SHCluster but no extractions

Log event skipped on read

Splunk Tenable Add-on: Anyone know why I'm getting...

Ciphering services not available with snmp_ta?

Is there any way to completely disable compression...

Re: Splunk Add-on for Unix and linux

Re: Splunk Add-on for Unix and linux

Re: Splunk DB Connect ERROR Without Error

Re: Splunk DB Connect ERROR Without Error

Why am I getting Splunk DB Connect ERROR Without E...

Re: Splunkism Regex?

Re: Splunkism Regex?

Splunkism Regex?

Re: DB Connect KEY=VALUE Extraction Fail

Re: DB Connect KEY=VALUE Extraction Fail

DB Connect KEY=VALUE Extraction Fail

Regex Truncation vs Rex

Re: Multiple Stats from Base Search

Re: Multiple Stats from Base Search

Multiple Stats from Base Search

Re: props deployed on SHCluster but no extractions

Re: props deployed on SHCluster but no extractions

Re: props deployed on SHCluster but no extractions

Re: props deployed on SHCluster but no extractions

Re: props deployed on SHCluster but no extractions

props deployed on SHCluster but no extractions

Log event skipped on read

Re: error regex (PCRE2_DUPNAMES not set)

Re: Splunk Tenable Add-on: Anyone know why I'm get...

Splunk Tenable Add-on: Anyone know why I'm getting...