About pkeller

pkeller · ‎05-24-2017

When I use the Job Inspector to view the Search Log of a completed search, I find hundreds of entries tagged: SearchOperator:kv that seem to have absolutely nothing to do with the sourcetype or datasource of the search ... ie: I'll see hundreds of regexes that are from EXTRACT-blah-blah ... that are explicitly for other sourcetypes. I'm concerned that I have a configuration issue. Lately we've had some complaints about search performance and I'm currently wondering why all this unrelated noise in the search log? Is Splunk actually trying to perform search time extractions for data that doesn't match the sourcetype the extraction was intended for?

pkeller · ‎04-13-2017

As others point out here, instead of running realtime searches, run the search every five minutes looking back at the events over the last five minutes. You don't have the search constantly eating resources and you're getting your alerts within 5-6 minutes of the trapped event.

pkeller · ‎04-07-2017

Thank you. This makes things very clear. - Cheers

pkeller · ‎04-06-2017

[monitor:///home/paul/training_status/] whitelist = (.csv$|.CSV$) blacklist = .filepart$ index=training_index sourcetype=training_status crcSalt = &ltSOURCE&gt The file gets updated once per week. In many cases, the file is not being fully consumed. The most recent update missed 19 records (which were consumed the last time the file was updated ) Splunkd.log shows: 04-06-2017 07:39:19.584 -0700 INFO WatchedFile - Will begin reading at offset=4234 for file='/home/paul/training_status/filename.csv So, my uneducated guess would be that splunkd is seeing data that it's already consumed and thus ignoring those 19 records before it starts ingesting. How do I prevent this? I thought setting crcSalt=&ltSOURCE&gt was supposed to handle this. Thank you.

pkeller · ‎03-20-2017

Thank you. I updated all searchbnf.conf files that had sections not containing a 'syntax' property with "syntax = foo", and deployed it to the cluster members. All is working fine now.

pkeller · ‎03-20-2017

Does the searchbnf bug imply that if I add the entire 'highlight' section to ever non-standard searchbnf config file that this may solve the issue in the interim? Thanks very much.

pkeller · ‎03-20-2017

ahah. Thank you. Yes. Quite a few searchbnf files. The one that splunkd.log is barking about appears to be from a Splunkbase app called Visual SPL. I may remove that one to see if it resolves the issue. And also .. the Uncaught TypeError: does show up under the inspect console.

pkeller · ‎03-20-2017

Yes. btool shows that the setting is toggled to true. /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf [general] /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf datasets:showInstallDialog = 1 /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf infodelivery_enabled = 0 /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf infodelivery_show_ad_modal = 1 /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf infodelivery_show_configure_modal = 1 /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf search_assistant = compact /opt/splunk/etc/apps/user-prefs/default/user-prefs.conf search_syntax_highlighting = 1 As does | rest /servicesNS/user/user-prefs/data/user-prefs/general | stats values(search_syntax_highlighting) Which shows "true"

pkeller · ‎03-20-2017

I'm finding that in all my 6.5.2 infrastructure that syntax highlighting is working fine, with the exception of my Search Head Cluster Members. This is pervasive across both my production and my test clusters. Is this perhaps a known issue? Under user-prefs.conf, search_syntax_highlighting shows a value of 1 Thank you.

pkeller · ‎02-24-2017

So your example is a multi-line event? .... This might work: ^.[^)]+)\s+(?P.+)\n?

pkeller · ‎02-24-2017

... This worked [WinEventLog://Adaxes] disabled = 0 Thank you. Paul

pkeller · ‎02-24-2017

This seems to work at: https://www.regex101.com/ ^.[^)]+)\s+(?P.+)

pkeller · ‎02-23-2017

I have some folks that want me to ingest Adaxes events under: Application and Services Logs -> Adaxes I'm not quite sure how to construct the inputs. The example below would just be a shot in the dark. [WinEventLog://Applications and Services/Adaxes] Thank you

pkeller · ‎02-15-2017

I want to grant access to a non-admin role to: | rest /services/authentication/users , but am not sure what capability that requires. I've granted the role rest_apps_management but that didn't seem to do the trick.

pkeller · ‎02-07-2017

Looking at the Daily License Usage panel on the "Previous 30 Days" tab under Licensing, I see that the base search is: index=_internal [ set_local_host ] source=license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(b) AS b latest(stacksz) AS stacksz by slave, pool, _time | stats sum(b) AS volumeB max(stacksz) AS stacksz by _time | eval pctused=round(volumeB/stacksz*100,2) | timechart span=1d max(pctused) AS "% used" fixedrange=false What is the idea behind setting _time 1/2 day behind? Thank you.

pkeller · ‎02-07-2017

Just a question regarding the resetting of _time. What's the purpose of setting time backwards by 1/2 day?

pkeller · ‎01-31-2017

Thank you. Works perfectly.

pkeller · ‎01-31-2017

My search produces output containing 5 license pool names, their configured pool size, and the volume that was indexed at the time of the rollover. My search: eventtype=evt-rollover-summary earliest=-1d latest=@d | stats latest(poolsz) as "PoolSize" sum(b) as "PoolUsage" by pool | rename pool AS "Pool Name" | eval "Pool Size (Gb)" = round(PoolSize/1024/1024,0) | eval "Pool Usage (Gb)" = round(PoolUsage/1024/1024,0) | fields - PoolSize, PoolUsage I'd like the column chart to just produce a 'dot' above the "Pool Usage (Gb)" column containing the "Pool Size (Gb)" field, but all I can seem to produce is a solid line for the overlay with nothing marking the line where the Pool Size Value is met. Not sure what I need to choose in the Overlay section to highlight the associated Pool Size value above the Pool Usage for a given Pool.

pkeller · ‎01-25-2017

The event I want to break on looks like this: 25/Jan/17:10:23:00:069+0000 DEBUG Evaluation of condition [188:FTP Mastering Users] took 0 ms props.conf looks like this: LINE_BREAKER = ([\r\n]+)(\d+/\w{3}/\d+:\d{2}:\d{2}:\d{2}:\d{3}) I've also tried this: LINE_BREAKER = ([\r\n]+)(\d+\/\w{3}\/\d+:\d{2}:\d{2}:\d{2}:\d{3}) TIME_FORMAT = %-m/%b/%y:%H:%M:%S:%3N%z TRUNCATE = 0 I'm still finding that my indexers are now combining every event matching the REGEX into a single event ( until the max events boundary is reached ) I figure I'm getting hung up on the "forward slash" in the date vs what I have in the REGEX, but have not been able to ingest this particular datasource accurately. So do I need to go a step further with regards to

pkeller · ‎01-18-2017

Using the Splunk Add-On for Microsoft Cloud Services to pull Azure data ... I'm having some difficulty indexing Azure Storage Blobs ... Anyone have any suggestions for props.conf on the sourcetype? I'm currently doing this, but events are still being broken in random places. [mscs:storage:blob] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)\{ TRUNCATE = 0 KV_MODE = json I'd welcome any better suggestions. props.conf has been deployed to the heavy forwarder, all indexers, and the search head cluster (although I'm not sure that it's necessary there) Splunk version HF - 6.4.3, Indexers - 6.4.3, Search 6.4.3

pkeller · ‎01-05-2017

This is now resolved. I did a fresh install of the Add-On and went through the storage account provisioning again and everything worked fine. This likely was due to my having copied an already configured app from our test environment to the production environment and that decryption was probably expecting a different secret key.

pkeller · ‎01-05-2017

When using the app on a heavy forwarder to configure a Storage Account with a SAS token we're getting the following error. We've tried generating 2 different SAS keys and yet we still get the same Splunk rejection. The token is valid, yet Splunk appears to be incapable of handling it. We're trying to first set this up under Configuration -> Add Azure Storage Account The token generated is formatted like the string below ... ?sv=YYYY-MM-DD&sig=xx9xx22%2xX1x0xXxXxx%2xxXXXx0XXXxXxxxXXx00xxXX%3D&se=YYYY-YY-YYTNN%3x00%3c00x&srt=sco&ss=bfqt&sp=rl

pkeller · ‎12-16-2016

I have the Qualys Technology Add-on (TA) for Splunk configured and it is collecting data from Qualys ... but that data, in xml format, is just sitting in TA-QualysCloudPlatform/tmp and I'm not seeing it forwarded to Splunk So ... assuming that the add-on has some built-in mechanism for parsing the collected XML and then forwarding to Splunk ... Does that forwarding/parsing process run at scheduled intervals? Does it remove the collected files once they've been forwarded? What process is actually reading and forwarding the collected data? Is there a way to configure a higher level of logging for this application? There's nothing very useful being written to the logs under var/log/splunk with regards to this TA. (so we didn't find out that our authentication was failing until long after it started) Thank you.

pkeller · ‎12-13-2016

The answer I received back from our SME's related to Azure are [inline] A quick question, what's your practice to configure SAS and how would you like the add-on to support that? We would like to configure one read only SAS token for each Access Key. As the Access Keys are rotated, the associated SAS token becomes invalid. If the add-on could support a pair of SAS tokens we would be able to rotate one key/token pair per maintenance cycle and not risk a service interruption. Simply put if the add-on could support 2 tokens and had the logic build in to switch to the second key if the primary fails that would be optimal. Is this the idea? 1. configure multiple SAS with different expiration date at one time in azure portal No, one token per key. 2. store all SAS tokens generated and hope the add-on can rotate them at different time? Yes, but they should rotate based on failure to connect/pull. Many thanks for your help, pkeller

pkeller · ‎12-07-2016

With regards to Azure Storage Accounts using SAS key, will the Splunk Add-on for Microsoft Cloud Services support situations where the SAS key is being rotated? I see only a field for a single SAS token.

Posts	158
Solutions	2
Karma Given	7
Karma Received	34
Member Since	‎01-09-2012

Online Status	Offline
Date Last Visited	‎12-07-2020 08:32 PM

Files under kvstore directory

dbConnect version 3 - db_inputs.conf - input_times...

Suggestions for filtering nodejs logs

Universal Forwarder Upgrades and our Deployment Se...

Splunk Enterprise Security -> Incident Review -> W...

Splunk_TA_aws - guardduty Time parsing.

Index rebalance - How to rebalance only Warm Bucke...

How do you do an automatic extraction based on the...

Consolidation of tstats results.

How can I ensure that my upgrade of Splunk Enterpr...

Splunk Search Log SearchOperator:kv

Re: Is splunk is best tool monitoring and alerting

Re: CSV Monitoring issues

CSV Monitoring issues

Re: Is Syntax Highlighting broken on a Splunk 6.5....

Re: Is Syntax Highlighting broken on a Splunk 6.5....

Re: Is Syntax Highlighting broken on a Splunk 6.5....

Re: Is Syntax Highlighting broken on a Splunk 6.5....

Is Syntax Highlighting broken on a Splunk 6.5.2 Se...

Re: How to generate a regular expression that extr...

Re: How to add Windows Event Log Adaxes.evtx as an...

Re: How to generate a regular expression that extr...

How to add Windows Event Log Adaxes.evtx as an inp...

What capability allows you to search /services/aut...

Why would a bucket/bin be set to a one day span on...

Re: Simple daily volume tracking?

Re: Column Chart

Column Chart

LINE_BREAKER not working correctly

Splunk Add-on for Microsoft Cloud Services: How to...

Re: SAS Token Encyryption Errors

SAS Token Encyryption Errors

Qualys Technology Add-on (TA) for Splunk: Does the...

Re: Splunk Add-on for Microsoft Cloud Services: Wi...

Splunk Add-on for Microsoft Cloud Services: Will t...

Join the Conversation