I am sorry , is till can not figure this out. You have 49391 in your sample 'would be' event but I can not see that number anywhere in your raw event, where is this number coming from? Also you have 61 , 56 etc, in your screen shot , I can not see them anywhere in your raw event... ... View more

hmm R^2 as 1 ? that means your linear regression model is able to 100% account for the independent variable? That is very rare, are you sure the model is right? What is the equation your linear regression model builds? Also, how does the predicted vs actuals compare when your use LLP? Is there a big difference? ... View more

Hi, What you have done stats wise is ok but I do doubt on your initial regression model, however if you feel it indeed works and have experimented with all the 'methods' under the kalman filter it is still remotely possible. Remember the kalman prediction in this case is based on your linear regression and will be as good and as bad as your linear model fit. Cab you validate the prediction after applying the kalman filter and see how it works? Essentially , have r^ 2 as 1 does mean that you have been able to explain all the variation perfectly, however it DOES NOT mean that ONLY the selected dependent variables are the ones your independent variable depends on... So whatever dependent variables you have chosen to predict cpu utilization explains all of the variance given that you can always get R2=1 if you have a number of predicting variables equal to the number of observations, or if you've estimated an intercept the number of observations - 1. the proof of the pudding is in eating, and you should validate your prediction against actuals. Please read the below excerpt on having R^2 as 1. I still think your linear regression model might be missing something.... 'An R2=1 indicates perfect fit. That is, you've explained all of the variance that there is to explain. In ordinary least squares (OLS) regression (the most typical type), your coefficients are already optimized to maximize the degree of model fit (R2) for your variables and all linear transforms of your variables. Your model appears to be a little odd in that x is being raised to a particular exponent, so your mileage may vary. But in response to your general question, you can always get R2=1 if you have a number of predicting variables equal to the number of observations, or if you've estimated an intercept the number of observations - 1. Either way, 20 parameters perfectly describes 20 data points. Such a model is called just-identified. Although this gives you the highly desirable perfect fit... it is essentially meaningless. ' ... View more

hi, Is the location mentioned in the events or you want to graph by iplocation? It will be nice to see a sample of your events ... View more

not clear what you want...I need the above stream to be broken up into separate events. Sorta like a wireshark view?? Can you give a sample of how you would like the events to be broken up into? ... View more

Hi, Couple of things here. The regression equation that you get , and if it is having R^2 ~ 98% is a good fit. I am assuming all the dependent variables can be got from the index(and actual CPU utilization is missing in the splunk index), now you can use the regression equation generated by your linear regression algorithm to predict the CPU utilization. Regarding the KALMN filter , it is important to realize that this is a time series predictor. There s a 'method' dropdown under the KALMAN filter, you can try the various methods and see which is predicting better a lot of which can be done WITHOUT the machine learning toolkit - http://docs.splunk.com/Documentation/SplunkCloud/6.5.1/SearchReference/Predict My suggestion (and just a suggestion) is to use your regression model , which as you say returns good results rather than the kalman time series predictor, remember the time series predictor just predicts future values based on historical trends of the same value mostly. It is great if you have seasonal variations etc. which is the most important factor in prediction BUT if you want to make a prediction based on dependent variables (like CPU utilization based on no.of log ins etc.) it is better to go for a regression model. Have you tried the decision tree regressor and the random forest regressor in the ML toolkit? They might actually give even better results than the linear regression. ... View more

well, i don't know about the index time but I guess you want an output with one row (that will have the second column values) and as many columns as there are rows in the CSV with first column as column headers. I tried with the below test data as an input CSV test 3 field1 abc field2 4.5 So, your expected out put will be something like this - test field1 field2 3 abc 4.5 Now, assuming your first column (headers) do not change too often , as it should be even if you have the first line in CSV as headers like usual..with a bit of manipulation you can get the output in the statistics tab in the desired way. I guess you want to perform further processing after you are able to get the output in a desired form - | sort test | transpose | rename column as test,"row 1" as "field 1" , "row 2" as "field 2" | head 1 One time manual work is needed to map the fields properly 'test' needs to be replaced by the first value in the CSV. 'row 1', 'row 2'....'row n' has to be renamed to the column values , like i have renamed row 1 and row 2 to field 1 and field 2. You will of course have many rows and I wonder how many rows we are talking about here and what effort this needs to be done manually, but once done it will work. Even if say a couple of new rows are added all you need to do is subsequently rename the newly added rows to the new first column values. I am guessing your column 1 values won't change so much but how many rows do you have first up that needs to be worked upon manually using this approach? If there are too many this approach may be too cumbersome. ... View more

just tried to use your sample in a text input, it worked fine, below is how the event looks like when i ingest it through a text file...see and compare against your event { [-] bandwidth_bytes_per_second: 0 dst_address: X.X.X.X dst_enforcement_epg_name: [ [+] ] dst_hostname: appServerXXXX dst_is_internal: true fwd_ack_count: 7 fwd_bytes: 1022 fwd_cwr_count: 0 fwd_nc_count: 0 fwd_synfin_count: 0 fwd_synrst_count: 0 fwd_tiny_count: 0 fwd_xmas_count: 0 rev_ack_count: 8 rev_allzero_count: 0 rev_cwr_count: 0 rev_finnoack_count: 0 rev_nc_count: 0 rev_pingdeath_count: 0 rev_pkts: 8 rev_psh_count: 4 rev_rst_count: 0 rev_synrst_count: 0 server_app_latency_usec: 0 src_address: Y.Y.Y.Y srtt_available: SRTT_NONE start_timestamp: 1505491512000000 timestamp: 2017-09-15T16:05:00.000Z total_perceived_latency_usec: 0 user_dst_businessUnit: null user_dst_pod: null user_src_businessUnit: null user_src_datacenter: null user_src_department: null user_src_lifecycle: null user_src_location: null user_src_pod: null } And when i ran the query | table timestamp,_time , this is what i get: timestamp _time 2017-09-15T16:05:00.000Z 2017-09-15 21:35:00 so mgarciar _timee is not really the current time , is it? I think if you do the event compare between my text ingestion and your event you will get some clues ... View more

Hi - Sorry for responding late. I had to find a different way, probably more suited to what I need. Well I used the Jellyfisher app and am using the jaro winkler distance to compare similarity between user input and historical data. The initial results look good, but I need to train the model for some months more with more data...will keep this forum updated 🙂 and many thanks for your help ... View more

hmm some issue with the pasting - line 4 in the rex , after the first? mark and before the . write ... View more

| eval host_message=m1+" " +m2 | eval t=strptime(strftime(_time,"%m/%d/%Y %H:%M:%S"),"%m/%d/%Y %H:%M:%S" ) | reverse | rex field=host_alert ^(?.*?)";" | streamstats current=false last(status) as prevst,last(t) as prevt | eval downtime=if((status="UP" AND(prevst="DOWN")) OR (status="DOWN" AND(prevst="DOWN")),round((t-prevt)/60,2),0) | eval uptime=if(downtime=0,round((t-prevt)/60,2),0) | fieldformat _time=strftime(_time,"%m/%d/%Y %H:%M") | table _time,uptime,downtime ... View more

For some reason the rex did not get copied properly, use the below one instead. | eval t=strptime(strftime(_time,"%m/%d/%Y %H:%M:%S"),"%m/%d/%Y %H:%M:%S" ) | reverse | rex field=host_alert ^(?.*?)";" | streamstats current=false last(st) as prevst,last(t) as prevt | eval downtime=if((st="UP" AND(prevst="DOWN")) OR (st="DOWN" AND(prevst="DOWN")),round((t-prevt)/60,2),0) | eval uptime=if(downtime=0,round((t-prevt)/60,2),0) | fieldformat _time=strftime(_time,"%m/%d/%Y %H:%M") | table _time,uptime,downtime ... View more

well, i don't know what exactly you mean by - 'I need to prepare a line chart showing the time duration the server was up and the time duration the server was down' you do realize that downtime is very very small compared uptime and having both on same time axis makes the graph looks very ugly, anyway here is the query : | eval t=strptime(strftime(_time,"%m/%d/%Y %H:%M:%S"),"%m/%d/%Y %H:%M:%S" ) | reverse | rex field=host_alert ^(?.*?)";" | streamstats current=false last(st) as prevst,last(t) as prevt | eval downtime=if((st="UP" AND(prevst="DOWN")) OR (st="DOWN" AND(prevst="DOWN")),round((t-prevt)/60,2),0) | eval uptime=if(downtime=0,round((t-prevt)/60,2),0) | fieldformat _time=strftime(_time,"%m/%d/%Y %H:%M") | table _time,uptime,downtime === I recommend using the multiseries chart mode with Y axis independent. The stats column from the output will give you what you are looking for. uptime and downtime are calculated in minutes in the above query. ... View more

hi please my answer below in details, i posted it as a separate entry ... View more

I tried this with your events in a notepad as follows: Sep 15 2017 07:06:07 app=yahoo dataconsumed=50 Sep 15 2017 08:16:07 app=skype dataconsumed=150 Sep 14 2017 10:26:07 app=facebook dataconsumed=10 Sep 14 2017 12:26:07 app=facebook dataconsumed=5 Sep 13 2017 7:26:07 app=yahoo dataconsumed=10 Sep 13 2017 9:26:07 app=skype dataconsumed=50 Sep 12 2017 3:26:07 app=facebook dataconsumed=80 Sep 12 2017 1:26:07 app=facebook dataconsumed=0 | eval t=strftime(_time,"%Y-%m-%d") | eval t1=strptime(t,"%Y-%m-%d") | eval d=strftime(relative_time(now(),"-2d"),"%Y-%m-%d") | eventstats max(t1) as f by app | eval d1=t1-f | eval c=if(f=t1,"c",if(d1=d,"p","nc")) | chart sum(dataconsumed) by app,c | eval dataconsumed=c-nc |eval gainer_or_looser=if(dataconsumed >0,"gainer",if(dataconsumed=0,"even","looser")) | fields app,gainer_or_looser,dataconsumed | sort - app ... View more

hmm are these columns in a CSV file or what is the format of the data source? The result needs a bit tweaking based on your data source ... View more

well, the query and alert is simple , it has to work. Are you sure that the time you checked / expected the alerts actually HAD any failures to set the trigger alert condition? ... View more

hi, not clear what you want. but append cols will only work if you have values in all the fields that you are basing your append cols on. for example: | appendcols StatDate [search.....] unless the start dates are common across all the sub searches you are using the appendcols will not work. the mongodb JSON you are consuming for id1,2 & 3 are they in the same index? Consider using stats command rather than append or join ... View more

if you need just the customer_name in the drop down, why not use something like this? index="index1" sourcetype="source1" | lookup kunde_test CUSTOMER_ID OUTPUT CUSTOMER_NAME | top 10 CUSTOMER_NAME dedup CUSTOMER_NAME | table CUSTOMER_NAME ... View more

hi - check this older discussion out https://answers.splunk.com/answers/107674/unknown-sid-error-when-doing-multiple-searches.html ... View more

how are you extracting the field? Why can't you just use a , as delimiter since all fields are separated by commas in your sample. You can then rename field9 as whatever you want during extraction. I think you might have tried using regex extraction from the web and selected just '700 auth' field as a sample. This will only return this specific value only. ... View more

Hi, Why don't you try blacklist in inputs.conf if you are on universal forwarder? [your stanza / what you are monitoring] blacklist = 400 will ignore all 400 type errors ... View more