In our env, we've had a high value for remote.s3.multipart_upload.part_size to fix a bug present in versions prior to 8.1. When I recently reverted the setting to its default, these logs started popping up from time to time:   WARN S3Client [34628 cachemanagerUploadExecutorWorker-0] - command=multipart-upload command=put transactionId=.... rTxnId=...status=completed success=N uri=... statusCode=500 statusDescription="Internal Server Error" payload="<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>...</RequestId><HostId>...</HostId></Error>" ERROR S3Client [103868 cachemanagerUploadExecutorWorker-0] - multipart-upload command=end transactionId=... rTxnId=... parts=5 uploadId="..." status=completed success=N payload="<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>...</RequestId><HostId>...</HostId></Error>" I checked, the uploaded file is the same size as the local one. I was also able to confirm the remote and local files have the same CRC, so it appears the upload worked. The file modtime in the S3 console is three seconds before the failed upload error message is recorded. My guess is there was a temporary issue, and the retry succeeded. I can't find any follow-up logs about the bucket id after the error message. Does anyone else observe this, should I just accept these error messages? Is there something I can do about it?

Hi, all! I came across with an issue with extracting fields from syslog. Here are some samples of the value which is "Call_Session_ID" I want to extract: JKYFnxBdcIIiBImIMsoJm67 tMKtr5WNYa2e9PqC1cBhswf YoKwDKa_K9m4SS1qzbecNbl hGpydwuxLF_iYw5AE0pe81g F440sxU_Ntqg2zswAXgt-lW Here's the regular expression generated by Splunk: ^[^\|\n]*\|(?P<Call_Session_ID>\w+) Here's some sample events: 2022-01-25 12:08:04,925|F440sxU_Ntqg2zswAXgt-lW|INFO|com.hsbc.amh.civr.fallout.node.AmhCivrGenesysXferNode|execute()|***End Call*** 2022-01-25 12:11:49,229|pbDdnF8QT6Bku0odJ4SL_Q8|INFO|com.hsbc.amh.civr.endcall.node.AmhCivrExitNode|execute()|***End Call*** 2022-01-25 12:27:03,958|42dHIbXvXBKqG20u_m3kU5R|INFO|com.ibm._jsp._xfer_5F_genesys:svf.nodename|_jspService()|Contact Data Sent: UD_DIALLED_SERVICE:OneNumber_Jade~UD_IVR_STARTCALL_REF:0~UD_IS_HANGUP:N~UD_CUSTOMER_TYPE:Jade~UD_FALLOUT_SECTION:BankPaymentTransfer~UD_PROPOSITION:Jade~UD_SUBPROPOSITION:GeneralBanking~UD_LANGUAGE:Cantonese~UD_FALLOUT_QUEUE:Default~UD_COUNTRY_CODE:HKCC~UD_FALLOUT_REASON:Agent Some facts about the log files: Call_Session_ID is followed by the everytime.  But there's some errors occurring when the results come out: Firstly, there's some null value in the result: Secondly, the result only shows part of the value like this: When checking back to the event, it shows that this Call_Session_ID contains a hyphen. 2022-01-25 11:59:18,032|Yih9YAueLZSJ-va5ZAVllOc|INFO|com.hsbc.amh.civr.endcall.node.AmhCivrExitNode|execute()|***End Call***  How could I solve the problem?        

Estoy tratando de desintalar el agente Splunl en un servidor y me sale este mensaje intente tambien hacerlo por linea de comando y me muestra un error parecido pero con el mismo mensaje como puedo hacer para resolver este problema.

My query after finalizing for some time , gives me,  The search processs with sid= was forcefully terminated because its physical memory usage  has exceeded the 'search_process_memory_usage_threshold'  setting in limits.conf. I am not allowed to increase memory... Any suggestion how to tweak the query  to avoid forceful termination? ================= (index=bsa) sourcetype=wf:esetext:user_banks:db OR sourcetype=wf:esetext:soc_sare_data:db au!="0*" | stats values(bank_name) as bank_name , values(bank_type) as type , values(pwd_expires) as pwd_expires , values(is_interactive) as is_interactive , values(au_owner_name) as au_owner_name , values(au_owner_email) as au_owner_email , values(service_bank_name) as service_bank_name , values(owner_elid) as owner_elid, , values(manager_name) as manager_name BY au | eval bank_name=coalesce(bank_name,service_bank_name) | eval user=lower(bank_name) | dedup user | rex field=user "[^:]+:(?<user>[^\s]+)" | fields - bank_name | stats values(au_owner_email) as au_owner_email , values(au_owner_name) as au_owner_name , values(owner_elid) as owner_elid , max(manager_name) as manager_name BY user ,service_bank_name ,type ,pwd_expires ,is_interactive

Hello,  I see following in _raw.  However, when I run search with table or fields it does not display text within double quote despite its in _raw.  "ptp-slave" does not get displayed as a value for field title. it only displays PTP State is ptp-listeining, should be.  2022-01-25 21:47:57.047, id="12342ee7c-757e-71ec-1090-777c841f0000", version="15119", title="PTP State is ptp-listening, should be "ptp-slave"", state="OPEN", severity="CRITICAL", priority="MEDIUM", --> there are more fields after this.  How can get entire test to be displayed. 

Hello, I have this ldapsearch that returns  10's of thousands of records.      | ldapsearch search=(&(objectClass=User)(!(objectClass=computer)))     I want to filter on the whenCreated attribute to return new users in the past 7 days, sliding window. Is it possible to perform filtering by one or more attributes on the ldapsearch command line? I know I can use Splunk evals after  the ldapsearch command to do this. Thanks and God bless, Genesius

We use Okta to authenticate and grant access to our Splunk Cloud instance. The groups and roles are already mapped.  When I have a team member leave the company, we deactivate their Okta account, so in theory, preventing them from accessing any of our apps. The SSO integration is great at creating SAML users on Splunk Cloud, but to get those accounts removed on Splunk Cloud usually requires a Splunk Support ticket and a 3-5 day turnaround. I can't use the Splunk REST API to do it because we're on cloud.  Does anyone know anything about deprovisioning automagically or getting Splunk Cloud to start working on this?  Fezzes, Swarm!

I'm trying to setup this image: mltk-container-golden-image-cpu:3.8.0 I'm able to access localhost but unable to get past the authentication page for Jupyter. It keeps giving me invalid credentials. I changed the password using:        jupyter notebook password        It still gives me invalid credentials.

  Need better option to get user id from first search to populate results using the subsearch.  thought join would work but its not.....suggestions? index=“something”  | rex field=userName "\'(?P<userName>.+)\'" | rex "\/ROOT\/[^\s']+\/(?P<Environment>[^\/]+)\/[^\s\/']+(\s|')" | search NOT (Environment=eu-central-1 OR Environment=BOLCOM) | rename commandTime as "Date/Time" | join userName     [ search index=okta sourcetype="OktaIM2:user" AND profile.department=*     | rename profile.employeeNumber as userName, profile.division as Department, profile.title as Title, profile.countryCode AS Country     | table userName, Department, Title, Country] | dedup Department, Title, Environment | table "Date/Time", Department, Title, Environment, Country

Hello friends. We are in the process of moving the collection of o365 events which we currently do on an on-prem HF via "Splunk_TA_microsoft-cloudservices" to SplunkCloud IDM using "splunk_ta_o365". Using the same Client ID, Client Secret, and Tenant ID, we seem to be getting similar workloads:  Aip, AzureActiveDirectory, CRM, Exchange, MicrosoftForms, MicrosoftStream, MicrosoftTeams, OneDrive, PowerApps, PowerBI, PublicEndpoint, SecurityComplianceCenter, SharePoint, SkypeForBusiness, Yammer   But if we perform a comparison of number of events, we seem to get lower amount of data using the `splunk_ta_o365` in SplunkCloud versus the `Splunk_TA_microsoft-cloudservices` in on-prem.   What seems to be the problem?

I found that the format of a sourcetype had changed some time ago. Now I need to extract the data correctly for both cases.   2022-01-11 17:40:59.000, SEVERITY="123", DESCRIPTION="ooops" 2018-01-24 16:35:05 SEVERITY="112", DESCRIPTION="blabla"   Extraction for the first type of entries works with this regex that was build with splunk field extraction   ^(?P<dt>[^,]+)[^"\n]*"(?P<SEVERITY>\d+)[^=\n]*="(?P<DESCRIPTION>[^"]+)   How can the regex be expanded to split either at "," or at the second space, if the comma is missing? An idea is to capture always at the second space and remove the comma or split before SEVERITY and remove the comma. I didn't get either working. You can find the regex at https://regex101.com/r/mxdAyx/1  Thanks

I am in the middle of configuring a standalone Splunk installation.  I am getting confused about the different attributes that can be set for overall storage and per index.  It is a very small installation with only about 30 assets connected and about 2.3TB of storage to store data for a year.  I have the following configuration so far: frozenTimePeriodinSecs = 31536000 #365 days [volume:hotwarm] path = <directory to hotwarm location> maxVolumeDataSizeMB = 178176 #174GB [volume:cold] path = <directory to cold location> maxVolumeDataSizeMB = 1970176 #1924GB [network] homePath = volume:hotwarm/network/db coldPath = volume:cold/network/colddb thawedPath = $SPLUNK_DB/network/thaweddb [windows] homePath = volume:hotwarm/windows/db coldPath = volume:cold/windows/colddb thawedPath = $SPLUNK_DB/windows/thaweddb I'm not sure how to use the maxTotalDataSizeMB with maxVolumeDataSizeMB to keep from maxTotalDataSizeMB triggering a roll to frozen before the 365 days is up.  We currently do not have any idea how much data will be coming in.  Is it good practice to set maxTotalDataSizeMB for each index to the same size as maxVolumeDataSizeMB? I have seen this practice before...   And if so, is it the maxVolumeDataSize of the cold storage, or hot/warm/cold storage combined?