About CarmineCalo

CarmineCalo · ‎02-22-2018

Splunkers! While, trying to export a dashboard into a pdf, i got this error message. I'm running the dashboard on Splunk 7.0 I understood that I've to update something on pdfgen_endpoint.py, what is not clear to me is how... Any support? Tks! Carmine

CarmineCalo · ‎02-22-2018

Mmm, agree with you.... few data and relation not really clear (i was trying to apply ML to come up with some statistical proven weights to apply to we model we created in a standard way). Tks! Carmine

CarmineCalo · ‎02-21-2018

Tks for the feedback, so you confirm my understanding... The problem is that each "fit model" run provide me a different predictor model. How can i "null" these differences (a kind of best fitting)? Is it feasible/ sounding in your opinion? Or, as far as I've very limited datapoints (only 12 occurences) i cannot expect to have a realiable model? Tks for the help! Carmine

CarmineCalo · ‎02-21-2018

Splunkers! I'm a newbie in the ML world, with some academic background, but very limited practice... We're now working on a prediction model trying to forecast ICT Incidents based on some input variables, having very limited datapoints (basically a monthly value for the last 12 months) We approached the problem in very "old fashion" way, trying to apply on "judgmental" basis, weights to each independent variable try to achieve a sounding forecast. I'm now trying to back-test the model with Splunk ML (leveraging on the "Predict Numeric Field" option). I have a big question on how Splunk Machine Learning Toolkit works. Basically, leveraging on the same datasets and same prediction parameters ("Field to predict", "Predictors" and Training/ Test ratio), each time I run the "Fit Model" button I got a different outcome. My initial understanding was that pushing "Fit Model", SML ran multiple iterations on the same dataset (with a different sample on the same dataset) to provide finally a sounding statistical result. But, in reality, it seems to me that SMLT just run a single iteration on a single sample extracted from the input dataset, just returning the result. Is my understanding correct? Is there any chance to run multiple iterations on the same dataset (different samples) to get finally the best fit model? Thanks for the support! Carmine

CarmineCalo · ‎01-31-2018

This option works, great 🙂 Tks! Carmine

CarmineCalo · ‎01-31-2018

Splunkers! I have a new problem I'm not able to solve, I hope you can help me... Basically, I'm counting the number of incidents occurring on weekly basis related to the last 2 years (events beginning in Jan 2016): ... | eval dateweek_year=strftime(_time,"%Y-%U") | chart count as Num_Incidents over dateweek_year Now, I'd like to present the outcome as "moving sum" of the last 52 weeks, starting from Jan 2017. So 01-2017 period has to show the sum of incidents from 02-2016 to 01-2017, 02-2017 from 03-2016 to 02-2017 etc... Any help? I've no clue about how to do it... Eventstats/Streamstats should help? Tks! Carmine

CarmineCalo · ‎01-29-2018

Tks, it works! Now I got how to use strptime. Tks Again, Carmine

CarmineCalo · ‎01-29-2018

Splunkers! I need to compute the duration of a event, as the difference between the two field (END_TIME and OPEN_TIME). Time format of END_TIME and OPEN_TIME is the following: 2017-12-31-20.01.37.000000 Code is the following: | eval END_INC=strptime(CLOSE_TIME, "%b %d %Y %H.%M.%S.%3N%P"), BEG_INC=strptime(OPEN_TIME, "%b %d %Y %H.%M.%S.%3N%P") | eval duration=END_INC - BEG_INC Not clear how to set parameters in strptime. Any help? Tks! Carmine

CarmineCalo · ‎01-24-2018

Splunkers! How should i modify the regula expression | rex field=duration "(?<hour>\d{2}):(?<min>\d{2}):(?<sec>\d{2})" to be able to convert in the same time into hours, minutes and seconds time value like 1;20:15 or 11:20:15? Final goal is to calculate duration in seconds, with this simple codeline | eval duration_h = hour+min/60+sec/3600 Currently code it is working only for the second format (values having hours with only 1 char are not converted). Tks! Carmine

CarmineCalo · ‎01-24-2018

Great, it works! The second requirement was related to the need to avoid multivalue fields in the following case CI ReqWeekAva Cluster_Ava_Code App1 10 CAC1 App1 40 CAC2 App1 30 CAC3 App1 40 CAC2 I mean avoid output like App1 40 CAC1 40 CAC1 But your solution it's fine, so no issue 🙂

CarmineCalo · ‎01-24-2018

Splunkers! I need to solve this problem. Basically, starting from a Service Catalogue (having the same AppID linked to more Business Services having potentially different Operational WIndows), i need to select the max Operational Window required (both in term of number and ID). Starting from an inputlookup (so no events as input) i need to select a full row (all columns) based on the max value in a specific column. Current code | inputlookup DOM_ServiceCatalogue | lookup AMAP_ReqAvailability Cluster_Availability as PrimaryWindows OUTPUTNEW ReqWeeklyAvailability as ReqWeekAva, Cluster_Ava_Code as Cluster_Ava_Code | stats max(ReqWeekAva) as ReqWeekAva, values(Cluster_Ava_Code) by CI But values function creates a MV fields, while i just need the "Cluster_Ava_Code" corresponding to the max value selected by stats function. Additionally, It might happen to have multiple rows in the Service Catalogue having the same max value for different services supported by the same AppID.... How can i do it? Tks! Carmine

CarmineCalo · ‎01-24-2018

Find the Issue! THere where a strange char (different than ") in some fields... Removed the strange char, now i pipe all the data from inputlookup! Tks! Carmine

CarmineCalo · ‎01-23-2018

In lookups folder (...\Splunk\etc\apps\search\lookups) there are only the lookup files (i tried to delete "DOM_ServiceCatalogue" file, but than inputlookup stopped to work). THere is also a subfolder: lookup_file_backups, but nothing within the sub/sub/sub folder related to "DOM_ServiceCatalogue".

CarmineCalo · ‎01-23-2018

Unfortunately it continues to not working...

CarmineCalo · ‎01-22-2018

This is the main content of the search log but it doesn't help me... 01-22-2018 23:01:40.847 INFO SearchParser - PARSING: | inputlookup DOM_ServiceCatalogue\n| rename ApplicationID as CI\n| lookup AMAP_ReqAvailability Cluster_Availability as PrimaryWindows OUTPUTNEW \n ReqWeeklyAvailability as ReqWeekAva, \n Sun as SunAvailability, \n Mon as MonAvailability, \n Tue as TueAvailability, \n Wed as WedAvailability, \n Thu as ThuAvailability, \n Fri as FriAvailability, \n Sat as SatAvailability,\n Cluster_Ava_Code as Cluster_Ava_Code\n| stats max(ReqWeekAva) as ReqWeekAva, \n max(MonAvailability) as ReqMonAva, \n max(TueAvailability) as ReqTueAva, \n max(WedAvailability) as ReqWedAva, \n max(ThuAvailability) as ReqThuAva, \n max(FriAvailability) as ReqFriAva, \n max(SatAvailability) as ReqSatAva by CI, Cluster_Ava_Code\n| fillnull value=0\n \n| search CI="FRM" 01-22-2018 23:01:40.861 INFO ISplunkDispatch - Not running in splunkd. Bundle replication not triggered. 01-22-2018 23:01:40.871 INFO UserManager - Setting user context: admin 01-22-2018 23:01:40.871 INFO UserManager - Free version does not have user services 01-22-2018 23:01:40.871 INFO UserManager - Done setting user context: NULL -> NULL 01-22-2018 23:01:40.910 INFO SortOperator - maxmem = 209715200 01-22-2018 23:01:40.915 INFO UnifiedSearch - Processed search targeting arguments 01-22-2018 23:01:40.915 INFO DispatchThread - BatchMode: allowBatchMode: 1, conf(1): 1, timeline/Status buckets(0):0, realtime(0):0, report pipe empty(0):0, reqTimeOrder(0):0, summarize(0):0, statefulStreaming(0):0 01-22-2018 23:01:40.917 INFO DispatchThread - required fields list to add to remote search = * 01-22-2018 23:01:40.917 INFO DispatchCommandProcessor - summaryHash=NS3d9d854163f8f07a summaryId=C7342F8D-CFAC-43F1-A7E8-3EF975823866_search_admin_NS3d9d854163f8f07a remoteSearch= Thi inputlookup is both part of a big query (that it's not properly working due to the bug that I'm trying to fix), but in I'm debugging it stand-alone to fix the problem. Carmine

CarmineCalo · ‎01-22-2018

Yes, they were. Replaced double quotes (") with quotes ('), but inputlookup continue to doesn't return all values... Any other suggestion? Carmine

CarmineCalo · ‎01-22-2018

Hello splunkers! New problem to be solved... This simple lookup | inputlookup DOM_ServiceCatalogue is not returning all the values (csv file is ~ 4MB, far away from the max size limit of 10MB set in the limit.conf, having ~ 7200 rows, 3 columns). It seems to stop piping data from inputlook around row 2.500-3.000. Lookup table is fine (i checked the content through the lookup editor app add-on). These are the limit.conf settings # maximum size of static lookup file to use a in-memory index for max_memtable_bytes = 10000000 # maximum matches for a lookup max_matches = 1000 # maximum reverse lookup matches (for search expansion) max_reverse_matches = 50 # default setting for if non-memory file lookups (for large files) should batch queries # can be override via a lookup table's stanza in transforms.conf batch_index_query = true # when doing batch request, what's the most matches to retrieve # if more than this limit of matches would otherwise be retrieve, we will fall back to non-batch mode matching batch_response_limit = 5000000 # maximum number of lookup error messages that should be logged max_lookup_messages = 20 Do I have to change something to pipe all the data from inputlookup? Tks! Carmine

CarmineCalo · ‎01-19-2018

See above, the problem was related to <> values in ReqMonthAva. Tks! Carmine

CarmineCalo · ‎01-19-2018

It works! I found the root problem... basically RegMonthAva had some <> value (corresponding with <> values for Unavailability_month and Incident_Duration). With | fillnull i solve the problem. Tks! Carmine

CarmineCalo · ‎01-18-2018

Splunkers! I'm not able to solve a strange issue... Basically, the stats avg() is omitting values in the calculation. Code is the following: <<search>> | fillnull Incident_Duration,Unavailabilty_Month value=0 | stats avg(Incident_Duration) as Incident_Avg, avg(Unavailabilty_Month) as Unavailabilty_Avg by ReqMonthAva, AppID So, null value are transformed into 0 in the table resulting from the previous search, but avg function is not considering them while processing Incident_Avg and Unavailabilty_Avg Seems to me very strange... how can i solve? Tks in advance!

CarmineCalo · ‎01-18-2018

Great! It works now 🙂

CarmineCalo · ‎01-18-2018

Only one additional thing... list(ApplicationID) create a field with "only" 100 value inside (my list of APpID is 4k+!) How can i increase the number of values to listed? Unfortunately "limit" option not works with stats...

CarmineCalo · ‎01-18-2018

Now It works! Tks!

CarmineCalo · ‎01-18-2018

No, unfortunately it's not working... It generate something like (hyp that Month = (Month 1, Month 2) App1 Month 1 App2 Month 2 App3 App4 ... Carmine

CarmineCalo · ‎01-18-2018

Splunkers! I need to join the follow inputlookup + event searche in order to have, for each AppID, the full set of month buckets given from the time range picker Example: Search 1 (Fromm inputlookup): App1 App2 ... Search 2 (from index search) Month 1 Month 2 ... Desired outcome: App1 Month1 App1 Month2 App1 ... App2 Month1 App2 Month2 App2 ... ... ... Here the code for the two searches Search 1 | inputlookup DOM_ApplicationCatalogue | search Status="Production" | stats count by ApplicationID Search 2 | search index=Incidents | dedup id_inc | timechart span=1mon count | eval datemonth_year=strftime(_time,"%Y-%m") | fields count datemonth_year] Any help? Tks! Carmine

Posts	42
Solutions	1
Karma Given	6
Karma Received	4
Member Since	‎01-04-2018

Online Status	Offline
Date Last Visited	‎06-05-2020 02:03 AM

Bailing out of Integrated PDF Generation. Exceptio...

Splunk Machine Learning Toolkit(SMLT) - Different ...

How to calculate the "Moving" sum of the last 52 w...

Duration of a single event

Converting [h]:mm:ss into hour, minutes and second...

How to get full row based on max

Why inputlookup doesn't return all the values?

Include zero in the average

How to "join" two different searches with no commo...

Lookup with variable Output

Bailing out of Integrated PDF Generation. Exceptio...

Re: Splunk Machine Learning Toolkit(SMLT) - Differ...

Re: Splunk Machine Learning Toolkit(SMLT) - Differ...

Splunk Machine Learning Toolkit(SMLT) - Different ...

Re: How to calculate the "Moving" sum of the last ...

How to calculate the "Moving" sum of the last 52 w...

Re: Duration of a single event

Duration of a single event

Converting [h]:mm:ss into hour, minutes and second...

Re: How to get full row based on max

How to get full row based on max

Re: Why inputlookup doesn't return all the values?

Re: Why inputlookup doesn't return all the values?

Re: Why inputlookup doesn't return all the values?

Re: Why inputlookup doesn't return all the values?

Re: Why inputlookup doesn't return all the values?

Why inputlookup doesn't return all the values?

Re: Include zero in the average

Re: Include zero in the average

Include zero in the average

Re: How to "join" two different searches with no c...

Re: How to "join" two different searches with no c...

Re: How to "join" two different searches with no c...

Re: How to "join" two different searches with no c...

How to "join" two different searches with no commo...