All Posts

Top

All Posts

Probably you could do this with INGEST_EVAL? Just test if you can do this with "eval ......" in one line adding several those one after one with suitable if etc. If/when you get it working in SPL then... See more...
Probably you could do this with INGEST_EVAL? Just test if you can do this with "eval ......" in one line adding several those one after one with suitable if etc. If/when you get it working in SPL then just copy that into transforms.conf into one INGEST_EVAL expression.
@Eldemallawy  In Splunk, EPS (Events Per Second) is a metric used to measure the rate at which events are ingested into the Splunk indexer. The formula to calculate EPS is relatively straightforward... See more...
@Eldemallawy  In Splunk, EPS (Events Per Second) is a metric used to measure the rate at which events are ingested into the Splunk indexer. The formula to calculate EPS is relatively straightforward: EPS = (Total Number of Events) / (Time Duration in Seconds) To calculate EPS, you need to count the total number of events that were indexed within a specific time duration (usually one second) and then divide that count by the duration in seconds. For example, if you want to calculate the EPS over a 1-minute window (60 seconds) and you have indexed 3,000 events during that time: EPS = 3000 / 60 = 50 events per second. This means you are indexing, on average, 50 events per second during that 1-minute period.
Many thanks @kiran_panchavat, much appreciated. Cheers, Ahmed.
@Eldemallawy  1. Identify and prioritize the data types within the environment. 2. Install the free license version of Splunk. 3. Take the highest priority data type and start ingesting its data ... See more...
@Eldemallawy  1. Identify and prioritize the data types within the environment. 2. Install the free license version of Splunk. 3. Take the highest priority data type and start ingesting its data into Splunk, making sure to start adding servers/devices slowly so the data volume does not exceed the license.  If data volumes are too high, pick a couple of servers/devices from the different types, areas, or locations to get a good representation of the servers/devices. 4. Review the data to ensure that the correct data is coming in. If there is unnecessary data being ingested, that data can be dropped to further optimize the Splunk implementation. 5. Make any adjustments to the Splunk configurations needed, and then watch the data volume over the next week to see the high, low, and average size of the data per server/device. 6. Take these numbers and calculate them against the total number of servers/devices to find the total data volume for this data type 7. Repeat this process for the other data types listed until you are completed.
@Eldemallawy  Estimating the Splunk data volume within an environment is not an easy task due to several factors: number of devices, logging level set on devices, data types collected per device, u... See more...
@Eldemallawy  Estimating the Splunk data volume within an environment is not an easy task due to several factors: number of devices, logging level set on devices, data types collected per device, user levels on devices, load volumes on devices, volatility of all data sources, not knowing what the end logging level will be, not knowing which events can be discarded.   Estimate Indexing Volume 1. Verify raw log sizes. 2. Daily, Peak, retained, future volume. 3. Total number of data sources and hosts. 4. Add volume estimates to data source inventory/spreadsheet. Estimate index volume size: 1. For syslog type data, index occupies ~50% of original size. 2. 15% of raw data ( compression ) 3. 35% for associated index files.
I would respectfully disagree.  Ours is an observability platform.  And to turn on anomaly detection and predictive analyst.  We need a constant flow of 90 days metrics data.  This is not a static nu... See more...
I would respectfully disagree.  Ours is an observability platform.  And to turn on anomaly detection and predictive analyst.  We need a constant flow of 90 days metrics data.  This is not a static number as customers add and remove CI's from their platforms.   We are also an MSP with 100's of customers.  And for data solvency we create many indexes per customer.  Typically each of our platforms has between 200-300 customer indexes. I am trying to automate out the TIOL of constantly having to review and update the config to ensure we keep the right data ingestion to feed and water the anomaly detection and predictive analyst.   With out over provisioning our storage.
Thanks Kiran,   I was looking for a way to estimate the volume of data that will be ingested into Splunk before installing it. This will help me calculate the License cost.    Therefore, is there... See more...
Thanks Kiran,   I was looking for a way to estimate the volume of data that will be ingested into Splunk before installing it. This will help me calculate the License cost.    Therefore, is there a way to estimate the volume of DNA-C metrics based on number of LAN / WLAN devices?   Cheers, Ahmed.
My configuration has not changed. I have verified that buckets are being created, and I have verified that a hot_quar_v1 bucket is being created. Why is it being created and how do I remove it?
You have earliest and latest explicitly set in your searches so they should be used. Again - check the job inspect and job logs to see what searches are being run and how. (The search contents, the t... See more...
You have earliest and latest explicitly set in your searches so they should be used. Again - check the job inspect and job logs to see what searches are being run and how. (The search contents, the time ranges, the expanded subsearch...).
Ok. So you have a very unusual border case. Typically systems need to handle constant amount of storage and that's what Splunk does.
I have an event like this:   ~01~20241009-100922;899~19700101-000029;578~ASDF~QWER~YXCV   There are two timestamps in this. I have setup my stanza to extract the second one. But in this particula... See more...
I have an event like this:   ~01~20241009-100922;899~19700101-000029;578~ASDF~QWER~YXCV   There are two timestamps in this. I have setup my stanza to extract the second one. But in this particular case, the second one is what I consider "bad". For the record, here is my props.conf:   [QWERTY] SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE_DATE = true MAX_TIMESTAMP_LOOKAHEAD = 43 TIME_FORMAT = %Y%m%d-%H%M%S;%3N TIME_PREFIX = ^\#\d{2}\#.{0,19}\# MAX_DAYS_AGO = 10951 REPORT-1 = some-report-1 REPORT-2 = some-report-2   The consequence of this seems to be that splunk indexes the entire file as a single event, which is something i absolutely want to avoid. Also, I do need to use linemerging as the same file may contain xml dumps. So what I need is something that implements the following logic:   if second_timestamp_is_bad: extract_first_timestamp() else: extract_second_timestamp()   Any tips / hints on how to mitigate this scenario using only options / functionality provided by splunk are greatly appreciated.
Split yes sorry it was 3am (one of those arr moments). So as part of this automation I need to build to grow / shrink the disk, part of this.  Is key. In an ideal world Splunk would inform the auto... See more...
Split yes sorry it was 3am (one of those arr moments). So as part of this automation I need to build to grow / shrink the disk, part of this.  Is key. In an ideal world Splunk would inform the automation I need to grow / shrink the volume on the cluster nodes. the update the automation would update splunk .conf files to set maxTotalDataSizeMB < the total disk now available in each cluster. node.  And then adjust the .conf for each index. Key to this is scan for all indexes.  Get the daily compression ration of the TXIDX.  The compression ration of the RAW data.  And the Daily data through put per index. For me I need 90 days data.  So into this build in a safety factor.
Please share some actual (anonymised) events so we can see what you are actually dealing with. Also, provide an example of the type of output you are looking for.
Hello everyone!  I'm trying to create a dashboard and set some tokens through javascript. I have some html text inputs and I want that, on the click of a button, they set the corresponding tokens t... See more...
Hello everyone!  I'm trying to create a dashboard and set some tokens through javascript. I have some html text inputs and I want that, on the click of a button, they set the corresponding tokens to the inputted value.  However, when I try to click again the button, the click event doesn't trigger. Can you help me?   require([ 'underscore', 'jquery', 'splunkjs/mvc', 'splunkjs/mvc/simplexml/ready!' ], function (_, $, mvc) { function setToken(name, value) { mvc.Components.get("default").set(name, value); mvc.Components.get('submitted', {create: true}).set(name, value); } /* ----------------------- */ let prefix = mvc.Components.get("default").get('personal_input_prefix') ?? "personal_"; // Setting tokens for Inputs with prefix ${prefix} $('#personal_submit').on('click', function(e){ e.preventDefault(); console.log("CLICKED"); let input_text = $("input[type=text]"); for (let element of input_text) { let id = element.id; if (id !== undefined && id.startsWith(prefix)){ let value = element.value; setToken(`${id}_token`, value); // <--- document.getElementById(`${id}_token_id`).innerHTML = value; // Set token ${id}_token to value ${value} } } }); });      DASHBOARD EXAMPLE:   <form version="1.1" theme="light" script="test.js"> <label>Dashboard test</label> <row> <panel> <html> <input id="personal_valueA" type="text"/> <input id="personal_valueB" type="text"/> <button id="personal_submit" class="primary-btn">Click</button> <br/> Show: <p id="personal_valueA_token_id">$personal_valueA_token$</p> <p id="personal_valueA_token_id">$personal_valueB_token$</p> </html> </panel> </row> </form>    
Hi there are several reasons which can cause to switch a new bucket event it's max size is reached.  When you are looking how your configuration has done. you should always use btool instead of loo... See more...
Hi there are several reasons which can cause to switch a new bucket event it's max size is reached.  When you are looking how your configuration has done. you should always use btool instead of looking those from file. Btool tolds you how splunk see those configurations as usually those are combined from several files. You both should use  splunk btool indexes list --debug lotte to see what is actual configuration for index lotte.  One reason for small bucket can be source events which contains events which have time stamps from past and future. Basically those haven't continuous increasing timestamps. When I look those smaller buckets there seem to be this kind of behavior based on those epoch times in bucket names. r. Ismo
Thank you very much for your feedback! I apologize for the anonymized query; I realize some parts were trimmed incorrectly. Regarding Point 3: I aim to have both the main search and the subsearch u... See more...
Thank you very much for your feedback! I apologize for the anonymized query; I realize some parts were trimmed incorrectly. Regarding Point 3: I aim to have both the main search and the subsearch use the same earliest and latest time fields. The idea is that the tstats command serves as a pre-filter, while the base search is used to retrieve the raw events. The query I wrote generally works as expected, but sometimes it fails to correctly use the specified earliest and latest. For instance, during one test, it returned the correct time range, but when tested an hour later, it didn’t align with the specified time. Interestingly, I noticed that tweaking the search command sometimes resolves this issue and ensures it searches within the correct time range. 
As usual this depends on your environment how you should do it.  Personally I prefer separate LM if you have distributed environment and especially if you have several clusters etc. I also avoid to ... See more...
As usual this depends on your environment how you should do it.  Personally I prefer separate LM if you have distributed environment and especially if you have several clusters etc. I also avoid to use any CM as LM. You could easily run LM as virtual node and it doesn't need almost any resources (2-4vCPU, 2-4GB memory etc.) Of course if you have lot of indexers then it should be bigger. Just configure it as an individual node and/or node which send its internal logs into some indexers/cluster (I prefer this). Usually I do this with conf files not with those commands. There is no need to configure it as a search head in cluster. Especially if you have MC on another node where you check license status. If you haven't and you have forwarded internal logs into cluster then just add it as an individual SH should installed into cluster environment.  r. Ismo Probably most important thing is that its general stanza's pass4symmKey is same than all those nodes which are connected to it (or at least that was earlier required). 
Hi @Lockie , the LM is a Splunk instance and all the Splunk Servers that need a licence point to it. the Servers that need to point to the LM are IDXs, SHs, CM, MC, SHC_D, and  HFs, UFs don't need ... See more...
Hi @Lockie , the LM is a Splunk instance and all the Splunk Servers that need a licence point to it. the Servers that need to point to the LM are IDXs, SHs, CM, MC, SHC_D, and  HFs, UFs don't need a connection. You can configurethis link manually by GUI or creating an add-on to deploy using the CM on the IDXs, the SHC-D for the SHs and the DS for the other roles, Ciao. Giuseppe  
Thank you for your reply. I understand. I tried to do this today but couldn't find a way. Is there a way to separate the license-manager? If the software itself does not support it, I won't bother wi... See more...
Thank you for your reply. I understand. I tried to do this today but couldn't find a way. Is there a way to separate the license-manager? If the software itself does not support it, I won't bother with it. In addition, please tell me how to separate the mc and how to configure it