About KendallW

KendallW · ‎09-29-2024

Hi @mayurr98, Is it ingesting exactly 10,000 events per hour or is that figure rounded? The query is set to run four times per hour. Does it return 2,500 events per run, or does the first run return 10,000 and the rest 0? Are there any error messages in the internal logs? index=_internal sourcetype=dbx_* Also, how much more than 10,000 events are we talking here? There could be a few different causes depending on the answers to the above questions. First check there are no other random db_inputs.conf files overwriting your above settings. Perhaps use btool to check this. Next, as DB Connect v3 and above uses HEC under the hood, use btool to check limits.conf to see if the max_content_length setting is too restrictive. Depending on what you find in the logs, you might also consider reducing the max_rows and/or fetch_size settings and/or decreasing the interval. Counterintuitive as it sounds, this might actually improve number of events returned, if there is some performance bottleneck preventing the query from completing. Regards, K

KendallW · ‎09-29-2024

Hi @JWGOV , As per the documentation here, for Oracle database inputs you may use any of varchar, int, float, real, bigint, number, timestamp, datetime, or date for the rising column. You shouldn't really have to do any other formatting of data so long as the rising column is in one of those types, but... Without seeing the actual data causing the ORA-01843 errors it's hard to say where the root cause lies, but consider using TO_NUMBER to convert the EVENT_TIMESTAMP field from a timestamp to a number, which would at least avoid this specific error. Although, you still might have problems if some bad data lands in the EVENT_TIMESTAMP field and can't be converted to number, or if it is converted to a number from a different timestamp format which might result in a checksum error. Are you able to show an example when one of those errors occurred?

KendallW · ‎09-29-2024

Hi @isoutamo thanks for your reply, Unfortunately the TA is not supported by Splunk, and Splunk support told me as much when I raised a case with them and suggested I try the forum. I did find this post which seems to be a very similar issue, so we are going to try the solution there of using the Splunk Add-on for Microsoft Cloud Services (which IS Splunk supported!) to ingest the events with HEC via Event Hub. I will reply again with any updates

KendallW · ‎09-26-2024

Hey gang, I'm using the Splunk Add on for Microsoft Azure to ingest AAD signin logs to Splunk under the azure:aad:signin sourcetype, however there seems to be a gap between the number of events visible in EntraID versus what is visible from Splunk. There are always slightly more events in EntraID. The gap seems to worsen the higher the volume of events becomes. See this table: Time Splunk Entra ID Difference 1st hour 3265 3305 40 2nd hour 3085 4804 1719 3rd hour 3264 6309 3045 4th hour 2274 3841 1567 5th hour 1659 2632 973 6th hour 2168 3442 1274 7th hour 6236 8923 2687 8th hour 22716 35901 13185 9th hour 63186 101602 38416 10th hour 88607 145503 56896 11th hour 68407 140095 71688 12th hour 76866 124423 47557 13th hour 68717 122355 53638 14th hour 81310 144880 63570 15th hour 50849 140876 90027 16th hour 42972 124040 81068 17th hour 33693 91792 58099 18th hour 13683 50408 36725 19th hour 13973 38695 24722 20th hour 12182 29645 17463 21st hour 9734 24187 14453 22nd hour 8037 16935 8898 23rd hour 5869 11994 6125 24th hour 5631 8837 3206 Total 688383 1385424 697041 Percentage difference 50.31% - This gap appears even when searching historical logs i.e. time slots over the last two weeks. - The retention period of the index is 90 days, so the events should not have expired yet. - There are no line breaking, event breaking, aggregation, timestamp, or other parsing errors for the sourcetype. - The gap is still present when searching over all time. - The internal logs from the Splunk Add on for Microsoft Azure only show the following two error messages which don't seem relevant, and only appeared a few times over the last month or so: "File "/opt/splunk/etc/apps/TA-MS-AAD/lib/splunklib/binding.py", line 1337, in request raise HTTPError(response) splunklib.binding.HTTPError: HTTP 503 Service Unavailable -- KV Store is in maintenance mode." "File "/opt/splunk/etc/apps/TA-MS-AAD/lib/splunklib/modularinput/event.py", line 111, in write_to stream.flush() BrokenPipeError: [Errno 32] Broken pipe" I have updated the polling interval of the Microsoft Entra ID Interactive Sign-ins input to 900 seconds, but still the issue persists. What other explanation could there be for the gap? Thanks, K

KendallW · ‎09-01-2024

In short, download the codebase from Github as a zip, then you can either install it from the GUI or extract the zip to $SPLUNK_HOME/etc/apps and restart Splunk.

KendallW · ‎09-01-2024

@ta1 There are installation instructions in the README.md file in the Github repo: https://github.com/plusserver/collectd/blob/master/README.md#installation

KendallW · ‎09-01-2024

Yes - see example here: https://docs.splunk.com/Documentation/SplunkCloud/latest/RESTTUT/RESTsearches#Example:_Create_a_search

KendallW · ‎08-26-2024

@guillermomolina There is a link at the bottom of every one of their emails - https://discover.splunk.com/subscription.html

KendallW · ‎08-26-2024

@irkey Put them in a search macro - https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/Usesearchmacros

KendallW · ‎08-25-2024

Hi @VijaySrrie assuming you are collecting the logs on syslog server then forwarding to Splunk with a UF? You can check if the UF is reaching its thruput limit which could cause indexing lag: index=_internal sourcetype=splunkd component=ThruputProcessor "has reached maxKBps"

KendallW · ‎08-25-2024

Hi @nawab123 most likely there is an issue with either your authentication.conf settings, or an issue connecting to the LDAP/SAML system. Check splunkd.log for those kind of errors

KendallW · ‎08-21-2024

Hi @Iris_Pi , yes, as per the documentation: For settings that are specified in multiple categories of matching [<spec>] stanzas, [host::<host>] settings override [<sourcetype>] settings. Additionally, [source::<source>] settings override both [host::<host>] and [<sourcetype>] settings.

KendallW · ‎08-19-2024

That error can have a few different causes. Check this post https://community.splunk.com/t5/Security/Why-does-Saml-response-not-contain-group-information/m-p/417494/highlight/true#M13278

KendallW · ‎08-19-2024

Hi @Iris_Pi would it be feasible to specify the time zone using source stanzas in props.conf instead of sourcetype in this case? [source::] takes precedence over [<sourcetype>] in props.conf.

KendallW · ‎08-19-2024

Hi @Muthu_Vinith could you please clarify the question.. Do you want to on-board an SSO error file to Splunk?

KendallW · ‎08-12-2024

As per @ITWhisperer 's comment, yes it is case sensitive. Use eval upper or lower to convert them all to the same case

KendallW · ‎08-12-2024

Hi @Joshua2 I won't judge the solution design, but the intended use of the Universal Forwarder is to forward logs, not store them locally for later manual transfer. You can do this with the UF by setting up local indexing on each machine, however you would have to pay for license usage as the data is indexed at the UF tier, and then again pay for license usage when it is transferred and indexed to your Splunk Enterprise instance later. So you'd be paying twice to index the same data. Also note there are performance implications for local indexing, and there are very limited parsing options on the UF, so you'd need to set up parsing later at the indexer anyway. If you're ok with that option, you can do it by setting the indexAndForward setting in ouptuts.conf: [tcpout] defaultGroup = local_indexing [tcpout:local_indexing] indexAndForward = true A better option to store the logs locally would be to use a third party log collection tool like Fluentd or LogStash, or write your own Powershell scripts. Ideally you would use Splunk for its intended purpose by directly forwarding the logs from the 60 UFs to a Splunk indexer (or HF), however I understand that may not be possible in this case.

KendallW · ‎08-12-2024

Hi @Viral_G, Try adding "align": "center" in the columnFormat tag of your tables, e.g. "columnFormat": { "identity": { "data": "> table | seriesByName(\"identity\") | formatByType(identityColumnFormatEditorConfig)", "align": "center" }, "usernames": { "data": "> table | seriesByName(\"usernames\") | formatByType(usernamesColumnFormatEditorConfig)", "align": "center" } }

KendallW · ‎08-12-2024

Hi @FeatureCreeep try setting the 'param.is_use_event_time' in alert_actions.conf as discussed in this doc: https://lantern.splunk.com/Observability/Product_Tips/IT_Service_Intelligence/Configuring_notable_event_timestamps_to_match_raw_data

KendallW · ‎08-12-2024

Hi @arunsoni, For this data you can break the events before each timestamp appears. Splunk probably won't be able to handle the JSON field extraction because of the preamble line, so just extract the fields at search time. props.conf : SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z) TIME_PREFIX = ^ TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3NZ MAX_TIMESTAMP_LOOKAHEAD = 25 search-time field extraction regex: (?<abrMode>[^"]+)"abrMode":"(?<abrMode>[^"]+)","abrProto":"(?<abrProto>[^"]+)","event":"(?<event>[^"]+)","sUrlMap":"(?<sUrlMap>[^"]+)","sc":\{"Host":"(?<Host>[^"]+)","OriginMedia":"(?<OriginMedia>[^"]+)","URL":"(?<URL>[^"]+)"\},"sm":\{"ActiveReqs":(?<ActiveReqs>\d+),"ActiveSecs":(?<ActiveSecs>\d+),"AliveSecs":(?<AliveSecs>\d+),"MediaSecs":(?<MediaSecs>\d+),"SpanReqs":(?<SpanReqs>\d+),"SpanSecs":(?<SpanSecs>\d+)\},"swnId":"(?<swnId>[^"]+)","wflow":"(?<wflow>[^"]+)"

KendallW · ‎08-12-2024

Hi @Cheng2Ready Yes, you just have to split each line of the field as a separate event, then you can use stats last to grab the last line: index=example "House*" Message=* | makemv Message | mvexpand Message | stats last(Message) as last_line

KendallW · ‎08-11-2024

It shouldn't matter what is contained in the host field in the 'cmdb' index as we are overwriting it. There is no problem with overwriting default fields in a search. Regardless, I still can't see why your original query didn't work. - There may be some whitespace or other strange characters in some of the field values from one of the indexes causing them to not match with the other index. Are you able to check this?

KendallW · ‎08-11-2024

Hi @neerajs_81 try just renaming the dv_name field instead of creating a new field with coalesce, e.g.: (index=cmdb sourcetye=server) OR (index=windows) | rename dv_name as host | stats dc(index) as idx_count, values(index) values(dv_os), values(dv_install_status) by host

KendallW · ‎08-11-2024

Hi @sidnakvee, Welcome! I highly suggest checking out some of the free training offered by Splunk, especially this one about getting data into Splunk: https://education.splunk.com/Saba/Web_spf/NA10P2PRD105/guestapp/ledetail/cours000000000003373 To answer your question, it sounds like you would like to send data from your local Windows machine to Splunk Cloud using the UF. To do this, you will indeed need to edit the inputs.conf file, for example: [WinEventLog:Security] disabled = 0 [WinEventLog:Application] disabled = 0 [WinEventLog:System] disabled = 0 [monitor://C:\Path\To\Sysmon\Logs] disabled = 0 Make sure to restart Splunk on the UF after making any changes, so that the changes are applied. Next, check that the UF is actually connected to your Splunk Cloud instance and forwarding its internal logs (index=_internal). If not, check the Splunk logs on the UF itself for any connectivity issues. The log files you want to check are "splunkd.log" and "metrics.log" located in ...\splunkforwarder\var\log\splunk\.

KendallW · ‎08-05-2024

@LearningGuy use inputlookup to grab the whole csv file into the search, then append the data from your index, then get common values using stats. E.g. | inputlookup host.csv | rename ip_address as ip | append [search index=owner | fields ip, owner] | stats values(host) as host values(owner) as owner by ip

Posts	109
Solutions	8
Karma Given	9
Karma Received	16
Member Since	‎12/06/2018
Email	kendall@epicon.com.au

Online Status	Offline
Date Last Visited	‎10-01-2024 01:27 AM

TA-MS-AAD Signin logs missing - help!

Splunk Cloud DDAS / delete?

Re: Splunk DB connect indexing only 10k events per...

Re: DB Connect: Intermittent ORA-01843 errors when...

Re: TA-MS-AAD Signin logs missing - help!

TA-MS-AAD Signin logs missing - help!

Re: Splunk Bots V2

Re: Splunk Bots V2

Re: can you create searches using the REST API in ...

Re: How to unsubscribe to splunk commercial emails...

Re: Reference multiple fields into a single name

Re: Log latency

Re: Error connecting to /services/authentication/u...

Re: How to handle multiple timezone when getting t...

Re: How to update an HTML file in Splunk

Re: How to handle multiple timezone when getting t...

Re: How to update an HTML file in Splunk

Re: Search query to combine data from 2 different ...

Re: UF on a standalone > USB transfer > indexed in...

Re: Need to center align the table values in Dashb...

Re: Normal/Clearing event getting same timestamp a...

Re: Props.conf for log entry with multiple formats

Re: Is there a way to capture the last line/senten...

Re: Search query to combine data from 2 different ...

Re: Search query to combine data from 2 different ...

Re: Splunk Cloud Not seeing Windows logs

Re: "left join" lookup from CSV to an index

Are you a member of the Splunk Community?