Hi Rakzskull, what is the maxKBps set to in limits.conf on the Forwarder where you see this message? By default it is 256. If possible, you could try increasing thruput. Also, this doc may be helpful: https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Troubleshootingeventsindexingdela... It might be worth investigating why the file is so large, whether the data being logged is excessive, and if there are opportunities to optimize its size, rotation frequency, etc. Hope this helps! ... View more

You could also run the following search: index=_internal sourcetype=splunkd component=LicenseUsage type=Usage | stats sum(eval(round(b/1024/1024/1024,2))) as indexed_gb by splunk_server Thanks! Greg ... View more

Hello -  This looks like valid JSON. Have you tried setting KV_MODE=json in props.conf? https://docs.splunk.com/Documentation/Splunk/8.0.5/Knowledge/Automatickey-valuefieldextractionsatsearch-time  Thanks! ... View more

Hey There - Which app is this?  Is it this one?  https://splunkbase.splunk.com/app/3161/ Which version of Splunk are you using?  Do you know if the app runs accelerated reports?  What do you see when you go to Search Activity in the Monitoring Console (Search->Activity-> Search Activity: Deployment) and check to see if there are searches running with high CPU. I also see high memory usage from a couple of other apps in that list.   If there is a search consuming high CPU, you can see if there is a way to modify or delete it if you don't care about it.  Also, you may need to review your Splunk infrastructure specs to determine if you need to scale or add additional compute/ storage.  There is a lot of great information here:  https://docs.splunk.com/Documentation/Splunk/latest/Capacity/Referencehardware  Let's start with what you're seeing in the Monitoring Console and go from there.   Thanks! ... View more

Could you please provide a small dummy-sample of your data to review and. test the regex?  On a side note, you may want to normalize the naming conventions.   This can be done at the source (maybe?), at index time, or at search time (e.g. create a lookup that outputs friendly names for server assets  and endpoint PCs, or use the coalesce function of the eval command).   Thanks! ... View more

Hello - You could route the sensitive data to a new index using props.conf and transforms.conf and create a new role.  This way the sensitive data is viewed only by users who need to see it.  The routing configurations would look something like this (e.g. sourcetype): Props: TRANSFORMS-foo_name = bar_sourcetype Transforms: [bar_sourcetype] REGEX = . DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::bar You can mask older data that lives in the previous index by creating and saving regular expressions that. match the field(s) containing sensitive data.   This post has good example: https://community.splunk.com/t5/Getting-Data-In/How-can-I-mask-data-both-at-index-time-and-search-time/td-p/26754  However,  the sensitive data should be removed at the source or stored in a new index that is secure to the folks who should be viewing it.  Hope this helps!   ... View more

I assume you're working with dashboards here? Without seeing the source code, it's hard to say what the root cause is. Can you provide the Simple XML and a dummy of the example search that you're using? These docs may be helpful in your case: https://docs.splunk.com/Documentation/Splunk/8.0.2/Viz/tokens https://docs.splunk.com/Documentation/Splunk/8.0.2/Viz/FormEditor Also, I recommend downloading the Dashboard Examples app from Splunkbase where you can copy example source code for dropdown inputs (and more!): https://splunkbase.splunk.com/app/1603/ ... View more

Great answer. Thank you! ... View more

My top two go-to's for this would be: 1) Use the Machine Learning Toolkit and let it do the work for you! 2) Try using the stats command and the perc function to compare current vs historical values. This is a timeless blog post that always comes in handy: https://www.splunk.com/blog/2016/01/29/writing-actionable-alerts.html Here is an example: index=_internal sourcetype=splunkd component=LicenseUsage type=Usage | bin span=5m _time | stats sum(b) as bytes by _time, idx | eval indexed_mb = ceiling(bytes/1048576) | stats sum(indexed_mb) as indexed_mb by _time, idx | stats perc95(indexed_mb) as perc95_indexed_mb, latest(indexed_mb) as current_indexed_mb by idx | where current_indexed_mb > perc95_indexed_mb This way you aren't hardcoding values, and the search adjusts to environment changes on-the-fly. You can adjust the percentiles as you see fit, and even go more granular by breaking things out by index, sourcetype (whatever makes sense for your environment). Play around with this and let me know how it works for you. ... View more

Thanks - I thought you could define admin in this same manner. Do what @SloshBurch mentioned, I think he's right 🙂 ... View more

Use password lockout (enabled by default): Password lockout temporarily locks users out of the system after a certain number of failed attempts. Password lockout is on by default and occurs five attempts. You can specify how many attempts the user has, and how long the lockout period must last. The default is 30 minutes. In less secure environments, you can set the lockout time for longer to be sure that an unverified user cannot access the system. Here is more on Password best practices for administrators: https://docs.splunk.com/Documentation/Splunk/7.2.5/Security/Passwordbestpracticesforadministrators ... View more

Hey There, My suggestions are rooted from core Splunk functionality first rather than adding it externally. First thing you can try is adding some functionality in your search that orders the values accordingly. Using commands like eval may help with adding logic and the flexibility to output your desired field name/ values. Here is a simple example using date_wday that you could use for your Month - Year values (To test, I looked at the Previous Week using the Time Range Picker in my local dev instance): index=_internal | stats count by date_wday | eval week_day = case(date_wday="sunday","0",date_wday="monday", "1", date_wday="tuesday", "2", date_wday="wednesday", "3", date_wday="thursday", "4", date_wday="friday", "5", date_wday="saturday", "6") | sort week_day | fields date_wday' Another option might be to use a lookup file in your search to output field name/ values you want. The first column header would be the original field name with its list of values, and the second header would be your output field. Let me know if this is heading towards the right direction or not we'll go from there. When in doubt I recommend using Splunk first before JS, etc. Thanks! Greg ... View more

Hi! The answer should be yes to both of your questions. ... View more

Does something like this help for starters? Since you're always looking for UserID , I included it in the initial stats . Ultimately, my logic is trying to transform like-events to a singular view by using stats and grouping by some unique identifier. By using earliest and latest functions, we can calculate duration before calculating average. Stats is also less taxing if you're able to fit it in: index=index_name (scenariotype="scenario_1" OR scenariotype="scenario_2") | stats earliest(_time) as start_time, latest(_time) as end_time, latest(userID) as userID by service_name, scenario_type | eval duration = end_time-start_time From there you can roll this up to more of a summary view with avg durations, etc.: | stats avg(duration) as avg_duration count(userID) as user_id by service_name You might need to play around with the second stats depending on how your data looks, but try this out for now and let me know how it looks. ... View more

Hi, Out of the box there may only be a \default directory - I'll see what I can find. It looks like if you use a database connection name other than sqlserver_default_connection, then you will want to replace all instances of the string sqlserver_default_connection in %SPLUNK_HOME%\etc\apps\dbx\local\inputs.conf with the name you select instead. You may want to create a new \local directory to add your new inputs.conf in so it overrides anything in \default. ... View more

Hey - did you happen to hear back from anyone on this? ... View more

Thank you! ... View more

Nah, aint no thang. This page is helpful but doesn't quite get me there with customizing: http://docs.splunk.com/Documentation/AddOns/latest/ServiceNow/Usecustomalertactions What I have listed above is almost there. The behavior I see after adding to the above scripts and files is - Incidents are created, but seem to be stored behind the scenes. What I mean by this is after I revert back to the orig scripts, all of the INC that were created using the new ones appear in Service-Now. I poked around in the Splunk App for ServiceNow, but I don't see anything that appears to need updating for populating custom fields, although I may have overlooked something. ... View more

I just ran into a similar issue with post-processing using a summary index. The summary index uses | fields rather than | table . In my base search prior to post-processing, I added | table * , and that helped to force the fields down for post-processing. Thanks for the tips! ... View more

rjthibod, Interesting! I'll play around with this and let you know how it goes. Thanks! ... View more