Activity Feed
- Posted Re: Will deleting audit events be logged in the internal index? on Getting Data In. 09-28-2023 11:48 AM
- Posted Re: AWS Addon - SNS Signature on Getting Data In. 01-10-2022 06:41 PM
- Posted Re: Distributed search from a SH Cluster to multiple Indexer Clusters on Deployment Architecture. 06-30-2021 01:09 PM
- Posted Distributed search from a SH Cluster to multiple Indexer Clusters on Deployment Architecture. 06-30-2021 12:33 PM
- Posted Re: How to SEDCMD nested json calculated as string? on Getting Data In. 01-27-2021 07:50 AM
- Karma Re: How to SEDCMD nested json calculated as string? for to4kawa. 01-27-2021 07:50 AM
- Posted Re: aws:sqs:securityhub splunk sourcetype jason format on All Apps and Add-ons. 01-26-2021 08:52 PM
- Posted How to SEDCMD nested json calculated as string? on Getting Data In. 01-26-2021 01:49 PM
- Tagged How to SEDCMD nested json calculated as string? on Getting Data In. 01-26-2021 01:49 PM
- Karma Re: Spath calculated field limitations for to4kawa. 01-25-2021 09:29 PM
- Posted Re: Spath calculated field limitations on Knowledge Management. 01-19-2021 12:38 PM
- Karma Re: MaxMind Database Update does not affect searches results for lenorxav. 01-14-2021 12:44 PM
- Posted Re: MaxMind Database Update does not affect searches results on Deployment Architecture. 01-13-2021 02:55 PM
- Posted Spath calculated field limitations on Knowledge Management. 01-13-2021 02:37 PM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 |
09-28-2023
11:48 AM
Unfortunately, we are in the same situation. I don't see an easy solution but here are a few searches that can help. If anyone has a better solution, it would be great to hear. (This will tell you if someone attempts to delete data w/o the permissions) index=_internal orig_component="StreamingDeleteOperator" sourcetype=splunk_search_messages | stats count by app message _time | mvcombine message (This will tell you when someone pipes the delete command into a search) index=_audit "| delete" search!="'search index=_audit \"| delete\"'" I have yet to see an audit log for successful deletion. Always be careful when searching the delete term, as best practice you should only apply the "Can Delete" capability for the period it is needed, and the search results should always be tested before attempting to use.
... View more
01-10-2022
06:41 PM
seeing the same thing on Splunk V8.1.5 Add-on V5.2.1: 2022-01-11 02:29:48,836 level=WARNING pid=2971768 tid=Thread-2 logger=splunk_ta_aws.modinputs.sqs_based_s3.handler pos=handler.py:_process:299 | datainput="<foo-bar>" start_time=1641868103, message_id="68a1a0a5-64bf-492c-a47d-96f1c3be0fb6" created=1641868188.579236 ttl=300 job_id=c3799c53-fcb9-4150-87f4-913ade22a58b | message="Warning: This message does not have a valid SNS Signature None None doesn't match required format '^https://sns\\.[-a-z0-9]+\\.amazonaws\\.com(?:\\.cn)?/'" We copied the config from a working V8.1.5 Add-on V5.2.0 system so we can build resiliency using the SQS Queues, but nothing is coming through from the new HF.
... View more
06-30-2021
01:09 PM
Thanks @burwell , we know we are currently peered, and am familiar with both the options you provided, but I am trying to go deeper. What I would like to find is an output that shows actual results (bytes, buckets, meta, etc) returned from the Indexer Cluster we want to de-peer from, based on the outgoing queries from the SHC.
... View more
06-30-2021
12:33 PM
Hey Splunkers! We have multiple IDX/SH clusters that are peered based on regulatory/compliance/operational reasons. We have a specific SHC that we would like to de-peer from an older IDX cluster. Indexes are reused and migrated across different IDX clusters frequently. What is the fastest and most accurate way to see what data is being fetched from the IDX clusters by a SHC? Thanks in Advance!
... View more
Labels
- Labels:
-
distributed search
01-27-2021
07:50 AM
@to4kawa Thank you for the response! I know the community appreciates all your contributions! That is an option, but this feed has upwards of 1400 dynamic fields across several data sources that aggregate into SecurityHub and then are pooled from AWS SQS, some fields are even in nested json lists. That would be alot of fields to index, but it may be our only realistic option. Any additional ideas to fix the parsing behind the scenes? Currently the user has been provided a macro with spath and lots of renames. Thanks!!
... View more
01-26-2021
08:52 PM
@Splunk_rocks Hey, I am having issues with SecurityHub parsing as well. Did you ever find a solution? This is my issue. Let me know if you found a solution. https://community.splunk.com/t5/Getting-Data-In/How-to-SEDCMD-nested-json-calculated-as-string/m-p/537320#M90059 Thanks!
... View more
01-26-2021
01:49 PM
Hey Splunkers!
We have a large json event that has a Body Message, and BodyJson Message, a little redundant but this is what was provided. The immediate issue is the BodyJson.Message doesnt auto extract the JSON fields and it appears to be due to the doublequote before/after the curly brace in the Message object, and also the backslash escaping the doublequotes in the KV pairs. If I remove them from the upload the data extracts completely, but I havent found a good sedcmd to modify just this section of the event, without breaking the rest of the event. Please Help!
"Message": "{\"version\":\"0\",\"id\":\"5d3f\"...]}}" (need to sedcmd)
_raw (obfuscated)
{"MessageId": "eff1", "ReceiptHandle": "gw6", "MD5OfBody": "41a8a", "Body": "{\n \"Type\" : \"Notification\",\n \"MessageId\" : \"dafe\",\n \"TopicArn\" : \"arn:aws:sns:us-east\",\n \"Message\" : \"{\\\"version\\\":\\\"0\\\"}\",\n \"Timestamp\" : \"2021-01-26T04:30:22.756Z\",\n \"SignatureVersion\" : \"1\",\n \"Signature\" : \"Eqaf90pc\",\n \"SigningCertURL\" : \"https://sns.us-east-1.amazonaws.com\",\n \"UnsubscribeURL\" : \"https://sns.us-east-1.amazonaws.com\"\n}", "Attributes": {"SenderId": "AID", "ApproximateFirstReceiveTimestamp": "1611635422813", "ApproximateReceiveCount": "1", "SentTimestamp": "1611635422812"}, "BodyJson": {"Type": "Notification", "MessageId": "dafe", "TopicArn": "arn:aws:sns", "Message": "{\"version\":\"0\",\"id\":\"5d3f\",\"detail-type\":\"Findings\",\"source\":\"aws\",\"account\":\"54\",\"time\":\"2021-01-26T04:30:22Z\",\"region\":\"us-east-1\",\"resources\":[\"arn:aws\"]}}", "Timestamp": "2021-01-26T04:30:22.756Z", "SignatureVersion": "1", "Signature": "Eqaf90pcXJtL425k7", "SigningCertURL": "https://sns.us-east-1.amazonaws.com", "UnsubscribeURL": "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east"}}
SEDCMD-test1 = s/ "Message": "{/ "Message": {/g
SEDCMD-test2 = s/}", "Timestamp/}, "Timestamp/g
SEDCMD-test3 = s/\\//g
... View more
Labels
- Labels:
-
JSON
01-19-2021
12:38 PM
Thanks for the quick reply @to4kawa . I think this is part of the issue... The _raw output has the nested JSON objects escaping quotes with a backslash under Message. Is my best bet to setup a props/transforms on the SH to replace \" with " ? Are there any working examples you could point me towards? Thanks! "BodyJson": {"Type": "Notification", "MessageId": "4f8b9202e", "TopicArn": "arn:aws:sns:ap-south-1:6679786758:events-ap-south-1", "Message": "{\"version\":\"0\",\"id\":\"0a880\",\"detail-type\":\"Findings - Imported\",\"source\":\"aws\",\"account\":\"56565\",\"time\":\"2021-01-19T20:26:38Z\",\"region\":\"ap-south-1\",\"resources\":[\"arn:aws:ap-south-1::product/aws/arn:aws:securityhub:ap-south-1:102707:subscription/v/1.2.0/1.6/finding/cb7ac3afd\"],\"detail\":....
... View more
01-13-2021
02:55 PM
@lenorxav Did you find a resolution to this issue? I am having a similar problem where I updated the mmdb but it is not reflecting on the SH. Thanks!
... View more
01-13-2021
02:37 PM
Hey Splunkers! We are running into an issue with an on-prem distributed deployment where the AWS feed is not extracting nested JSON fields at search time without the use of spath. We get first level and partial second level auto extraction, but it stops there. We need to normalize this data for functionality with friendly name alias's, and would like to avoid end users having to use spath with a long rename macro. yes, KV_MODE is set to JSON on the SH, IDX, and HF. no, we'd rather not perform indexed extractions. We've upped several limits and are unsure why it wouldn't just auto extract at searchtime. Please halp! Here is the issue with using spath in calculated fields as a work around. I can calculate version and id consistently, but next level nested values with lists do not calculate to return fields at search-time. works -> aws : EVAL-version version spath('BodyJson.Message', "version") works -> aws : EVAL-id id spath('BodyJson.Message', "id") doesn't work -> aws : EVAL-resources resources spath('BodyJson.Message', 'resources{}') BodyJson: { Message: {"version":"0","id":"-e154-88b-c","detail-type":"Findings - Imported","source":"aws.","account":"4724","time":"2021-01-13T20:09:26Z","region":"ca-central-1","resources":["arn:aws:ca"],"detail":{"findings":[{"ProductArn":"arn:aws:"... What am I doing wrong here? Also, is there a known limitation on how many cycles of spath calculations the system will run on a specific field? Thanks in advance!
... View more
Labels
- Labels:
-
calculated field
