About D2SI

D2SI · ‎01-09-2020

Alright I have tested it and it does not work, I mean if DATETIME_CONFIG is set to CURRENT, it overrides any other TIME_* parameter and set timestamp to indextime. Thanks anyhow for the suggestion.

D2SI · ‎01-09-2020

Thanks for your answer. Your are saying that with TIME_* parameters configured and DATETIME_CONFIG set to current, events with a timestamp will get timestamped according to TIME_* parameters while events from the same sourcetype without timestamp will have the indextime as the timestamp. To my understanding setting DATETIME_CONFIG to current overrides all the TIME_* parameters and set the indextime as timestamp but I will test this.

D2SI · ‎01-09-2020

Hello, Thanks a lot for the App! I have some alerts configured the way I need them but I got this splunkd WARN: 01-09-2020 07:21:33.275 +0000 WARN sendmodalert - action=slack STDERR - Validation warning: Parameter attachment must be ether "alert_link" or "message" while the configuration is 'Attachment = none', and it says the setting is optional. The alert output is fine. I guess the WARN could just be ignored but I wanted to post here so it can be 'fixed' maybe in the next version. Thanks

D2SI · ‎01-09-2020

Hello there, I had an "Invalid key in stanza" in the version of 1.0.14 of the App: Invalid key in stanza [victorops] in /opt/splunk/etc/apps/victorops_app/default/alert_actions.conf, line 13: param.record_id (value: ). Invalid key in stanza [victorops] in /opt/splunk/etc/apps/victorops_app/default/alert_actions.conf, line 22: param._cam (value: { "category": ["Information Conveyance"], "task": ["create"], "subject": ["splunk.event"], "technology": [{"vendor": "Victorops", "product": "API", "version": "20131114"}], "supports_adhoc": true }). So I added the following lines to $SPLUNK_HOME/victorops_app/README/alert_actions.conf.spec: param.record_id = Record ID. param._cam = Cam. I have raised a support case to notify devs but it still present in version 1.0.18 so I thought I might as well post here.

D2SI · ‎01-08-2020

Hello there, I got the following error a lot: "ERROR HttpInputDataHandler - Parsing error : No data" I guess it is related to HEC but I don't understand it nor find info about it. Would anyone know more about this error?

D2SI · ‎01-08-2020

Hello there, For a particular sourcetype there are events with a timestamp and events without timestamp. As Splunk cannot detect a timestamp in the events without timestamp, it generates plenty of errors saying 'Defaulting to timestamp of previous event'. I am trying to clean this up. I see that I can configure a local datetime.xml to take into account various timestamp formats. What I would like to do is : for event with a timestamp -> assign the correct timestamp for events without timestamp -> assign the indexing time as the timestamp and don't throw errors Is there a way in datetime.xml to say : for this format, apply indextime timestamp? Thanks in advance for any hint!

D2SI · ‎12-18-2019

Thanks @starcher ! It worked like a charm indeed!

D2SI · ‎12-10-2019

Hello there, We are trying to automate KV Store backup/restore but there is something I do not understand yet. Prior 6.5.3, you had to stop the Search Head, then backup the KV Store directory. (https://docs.splunk.com/Documentation/Splunk/6.5.3/Admin/BackupKVstore) Now, there is an handy dedicated command for that which is great. (https://docs.splunk.com/Documentation/Splunk/7.2.8/Admin/BackupKVstore I am lost with the size difference thought. On our Search Head, KV Store directory weight ~1 GB. When I check our KV Store collections from the Monitoring Console, it weights less than 20 MBs total. When I try to backup using the backup command, the output weight a few MBs, which fits well with what I see from the Monitoring COnsole. So I am wondering why the size difference is so big between both methods. What would I loose if I restore the KV Store with the lightweight few MBs backup to a reconstructed Search Head? Any clue ? Thanks in advance!

D2SI · ‎11-05-2019

Interesting, let me check on that! Screenshots are tiny I know but it's the last 15 min not months 🙂 And yes, if I do any transforming command it works, I mean the 'Statistics' & 'Vizualisations' works, it just the raw data part that gets blank! Thanks

D2SI · ‎11-05-2019

admin user, happens on any index, yes I have filed a case! thanks

D2SI · ‎11-05-2019

I kinda did ! but thanks 🙂

D2SI · ‎10-30-2019

Hello Ana, Open logging should be enabled on the gateway, with care (maybe not on prod right away, to evaluate the impact). Then, when logging is enabled, logs are being written to local files as documented here: link text Then you got to put this data into Splunk. And there are several ways to do so. The simpler: You have a Splunk Universal Forwarder in the gateway, and it just monitor the file with a simple configuration like: [monitor:///<INSTALL_DIR>/apigateway/logs/group-*_instance-*_traffic.log] sourcetype = axway:apigateway:traffic:json index = <index> the configuration needed to route data to your Splunk (outputs.conf). Our AWS scenario, not the simpler, but we had to do it that way: You have a CloudWatch agent on the Gateway that forwards the open logging data to a CloudWatch Log Group, which is pushed to Splunk HTTP Event Collector via Kinesis Firehose. In that case, yes, you need to configure a token on the Splunk side with the HEC input: [http://<input name>] index = <index> indexes = <index> sourcetype = axway:apigateway:traffic:json token = <token> useACK = 1 And the token should also be configured on the Kinesis Firehose stream. There is some documentation on how to create a Kinesis Forehose stream (i.e. link text), except you do not have to use a Lambda function. I hope it will help!

D2SI · ‎10-29-2019

Hello there, Since a few months, Splunk randomly displays a blank screen where the raw data is usually shown. It does not seem to depend on browser add-ons as I am experiencing the same issue with 3 different browsers: As a temporary workaround, I use '| table _time _raw' to see the raw data but it is not convenient. Out of the blue, it just comes back to normal a moment later. The issue is very random but happens everyday. Anyone would have a clue? Thanks in advance for any hint!

D2SI · ‎10-21-2019

Finally we went back to SQS mode and created a separate SQS queue for config notifications. I do not have the config anymore and I do not remember having issue with event breaking but after having checked another props for events also coming form CW Event rule and the sample I still had, this might help: [aws:config:notification] LINE_BREAKER = }}(){\"MessageId SHOULD_LINEMERGE = false TIME_PREFIX = SentTimestamp\":\s\" MAX_TIMESTAMP_LOOKAHEAD = 13 TIME_FORMAT = %s

D2SI · ‎08-29-2019

Great thanks!

D2SI · ‎08-29-2019

Hello there! I am looking for a way to adjust multivalue choice box length to keep them on one line. I have already adjusted multivalue input box : <html> <style> #instance_id { width: 50%; } </style> </html> But I do have long string choices that get displayed on two lines (#2), so I am trying to adjust box choices length to have something like this (#3): Is there a way using css / simple xml? Thanks in advance for any hint!

D2SI · ‎08-29-2019

Alright, great!

D2SI · ‎08-27-2019

Could you post a sample of your modsec logs?

D2SI · ‎07-22-2019

Wow, no clue why I did not think of that! I ended up using '| makemv delim="\r\n"' instead but following the same logic and got rid of the standalone transforms.conf stanza! Thanks a lot!

D2SI · ‎07-19-2019

Hello, In a particular TA, I had to use a standalone transforms.conf stanza : [standalone_stanza] REGEX = (.+?)\:\s(.+?)(?:\\r\\n|$) FORMAT = $1::$2 It is needed because I needed dynamic field name extraction (hence, FORMAT = $1::$2). The stanza is called directly in the search queries using the extract command and with the target passed as the _raw value : [...] | eval _raw=... | table _raw | extract standalone_stanza limit=1 clean_keys=false | fields - _raw [...] Everything is fine except I got the following warning in the splunkd.log : WARN SearchOperator:kv - buildRegexList provided empty conf key, ignoring. I believe it is due to the fact that the transforms stanza is not called in props.conf. Thing is I do not want it to be called automatically because of the way it works. So I guess I can just ignore the warning log. I am just wondering if there is a cleaner way. Maybe it is somehow possible to reference the transforms stanza in props and configure it to not be launched automatically. I have checked to documentation without luck so far. Thanks in advance for any hint!

D2SI · ‎07-17-2019

Yes, just upload it is the /appserver/static folder of your app as a *.js file after having removed '

D2SI · ‎07-17-2019

Alright there is a feature request for this : ADDON-20112 In the meantime, we are successfully indexing config change notifications using CloudWatch Events rule + Kinesis Firehose + HEC : https://aws.amazon.com/fr/blogs/mt/ingest-aws-config-data-into-splunk-with-ease/ It just need a little tweaking at the indexing level to correctly split json events.

D2SI · ‎07-15-2019

We are indexing AWS data into Splunk using Splunk Add-on for AWS. We have configured inputs to retrieve data from AWS Config. AWS Config data should go in the sourcetypes 'aws:config' & 'aws:config:notification'. While we do get data in 'aws:config' we do not get any data under 'aws:config:notification'. The documentation (https://docs.splunk.com/Documentation/AddOns/released/AWS/ConfigureInputs) states that 'SQS-based S3' input type is supported for 'aws:config:notification'. However, we spotted the following message in the logs : 2019-06-07 07:55:02,631 level=INFO pid=26766 tid=Thread-5 logger=splunk_ta_aws.modinputs.sqs_based_s3.handler pos=handler.py:parse:149 | start_time=1559893993 datainput="config-sqs_s3", created=1559894102.63 message_id="58cee5d5-0b6b-46b7-af16-2e3ee2a2d22f" ttl=300 job_id=73693ed4-de47-4c5d-bd0f-3dbf11bcb5b6 | message="Ingnoring this config message." message_type="ConfigurationItemChangeNotification" And handler.py seems pretty clear about it: class ConfigNoticeParser(object): """ Wrapper class for easy accessing config dict based notifications. """ _SUPPORTED_MESSAGE_TYPE = [ 'ConfigurationHistoryDeliveryCompleted', 'ConfigurationSnapshotDeliveryCompleted', ] _UNSUPPORTED_MESSAGE_TYPE = [ 'ConfigurationItemChangeNotification', 'ConfigurationSnapshotDeliveryStarted', 'ComplianceChangeNotification', 'ConfigRulesEvaluationStarted', 'OversizedConfigurationItemChangeNotification', 'OversizedConfigurationItemChangeDeliveryFailed' ] def __init__(self, message, region_cache): self._message = message self._region_cache = region_cache def parse(self): message = self._message message_type = message['messageType'] if message_type in self._UNSUPPORTED_MESSAGE_TYPE: logger.info('Ingnoring this config message.', message_type=message_type) return [] if message_type not in self._SUPPORTED_MESSAGE_TYPE: raise TypeError('Unknown config message.') # for supported message types bucket = message['s3Bucket'] region = self._region_cache.get_region(bucket) key = message['s3ObjectKey'] if not isinstance(key, unicode): raise TypeError('s3ObjectKey is expected to be an unicode object.') return [self._make(region, bucket, key)] def _make(self, region, bucket, key): return S3Notice(region, bucket, key, None, None) inputs.conf: [aws_sqs_based_s3://config-sqs_s3] aws_account = <assume_role_name> aws_iam_role = <aws_account_name> disabled = 0 host = <host> index = main interval = 300 s3_file_decoder = config sourcetype = aws:config sqs_batch_size = 10 sqs_queue_region = <region> sqs_queue_url = https://sqs.eu-west-1.amazonaws.com/<aws_account_id>/<sqs_name>; Are we missing something here ? Thanks in advance for any hint!

D2SI · ‎07-12-2019

Thanks David! I did know spath but thought I did not need it since data was all correctly extracted, and I have found a way using it 🙂 ! Something like index=... | spath output=level_1 path=circuitPath{} | fields level_1 | mvexpand level_1 | rex field=level_1 "^{\s\"policy\":\s\"(?.+?)\"" | fields policy | append [search index=... | spath output=level_2 path=... ... Thanks!

D2SI · ‎06-06-2019

Wow, I did not think of that! Thanks

Posts	78
Solutions	1
Karma Given	29
Karma Received	12
Member Since	‎05-18-2018

Online Status	Offline
Date Last Visited	‎10-21-2020 05:05 AM

Convert earliest_time latest_time notation to epoc...

Adjust top margin on a panel with different inputs...

Center a mix of text and link inputs using CSS

Why does dark theme switch fail when using emojis ...

Adjust width of input element via CSS

Highlight clicked row from a table and keep the ro...

Highlight clicked row from a table using javascrip...

Tooltips and Popovers js not working in 7.3

Event without timestamp

Attachment must be ether alert_link or message

Re: Custom timestamp detection for a sourcetype wi...

Re: Custom timestamp detection for a sourcetype wi...

Attachment must be ether alert_link or message

Invalid key in stanza

HttpInputDataHandler - Parsing error : No data

Custom timestamp detection for a sourcetype with s...

Re: KV Store size question

KV Store size question

Re: Blank screen instead of raw data

Re: Blank screen instead of raw data

Re: Blank screen instead of raw data

Re: Axway API Gateway Add-0n

Blank screen instead of raw data

Re: Splunk Add-on for Amazon Web Services: Cannot ...

Re: Multivalue input choice box length css

Multivalue input choice box length css

Re: Why is this event splitting into three single ...

Re: Why is this event splitting into three single ...

Re: Standalone transforms.conf stanza (not called ...

Standalone transforms.conf stanza (not called in p...

Re: Expand json messages by default

Re: Splunk Add-on for Amazon Web Services: Cannot ...

Splunk Add-on for Amazon Web Services: Cannot inde...

Re: Dealing with complex variable level / depth JS...

Re: Splunk Cloud configs API endpoint using '| res...