Splunk Search

MLTKC how should i check jupyter notebook func in splunk

ryanaa
Explorer

I want to use an autoencoder model in Splunk for anomaly detection. I have already built my own model, and I did not use a scaler during the process. However, I still encountered the following error.

Here is my code:

 

I want to check the fields returned by my func in the search bar. What syntax can I use to verify this?

this is my python code

 

 

 

def apply(model, df, param):
    X = df[param['feature_variables']].copy()
    # 1. 類型轉換
    X = X.replace({True: 1, False: 0})
    
    # 2. 處理特殊字符/缺失值
    X = X.apply(pd.to_numeric, errors='coerce')  # 將無法轉換的值設為NaN
    X = X.fillna(0)
    
    # 3. 類型統一
    X = X.astype('float32').values
    
    """
    应用模型进行异常检测(无标准化)
    """
    # X = df[param['feature_variables']].values
    
    # 重建预测
    X_reconstructed = model.predict(X)
    
    # 计算重建误差
    reconstruction_errors = np.mean(np.square(X - X_reconstructed), axis=1)
    
    # 异常阈值设置
    
    threshold_percentile = param.get('options', {}).get('params', {}).get('threshold_percentile', 95)
    threshold = np.percentile(reconstruction_errors, threshold_percentile)

    # 构建结果
    df_result = df.copy()

    df_result['reconstruction_error'] = reconstruction_errors
    filtered_errors_1 = df_result.loc[df_result['is_work'] == 1, 'reconstruction_error']
    filtered_errors_0 = df_result.loc[df_result['is_work'] == 0, 'reconstruction_error']

    threshold_1 = np.percentile(filtered_errors_1, threshold_percentile) if not filtered_errors_1.empty else np.nan
    threshold_0 = np.percentile(filtered_errors_0, threshold_percentile) if not filtered_errors_0.empty else np.nan
    df_result['threshold'] = np.where(df_result['is_work'] == 1, threshold_1, threshold_0)

    
    df_result['is_anomaly'] = (reconstruction_errors > threshold).astype(int)
    
    # 可选隐藏层特征
    if param.get('options', {}).get('params', {}).get('return_hidden', False):
        intermediate_model = Model(inputs=model.inputs, outputs=model.layers[1].output)
        hidden = intermediate_model.predict(X)
        hidden_df = pd.DataFrame(hidden, columns=[f"hidden_{i}" for i in range(hidden.shape[1])])
        df_result = pd.concat([df_result, hidden_df], axis=1)
    
    return df_result

 

 

 

I used apply to call this model, but I want to see the threshold field returned in df_result.

 

 

 

Tags (2)
0 Karma
Get Updates on the Splunk Community!

AI for AppInspect

We’re excited to announce two new updates to AppInspect designed to save you time and make the app approval ...

App Platform's 2025 Year in Review: A Year of Innovation, Growth, and Community

As we step into 2026, it’s the perfect moment to reflect on what an extraordinary year 2025 was for the Splunk ...

Operationalizing Entity Risk Score with Enterprise Security 8.3+

Overview Enterprise Security 8.3 introduces a powerful new feature called “Entity Risk Scoring” (ERS) for ...