All Apps and Add-ons

Apply command on a large field

KrithikaRamakri
Explorer

Hi everyone, I am trying to apply logistic regression to predict phishing based on a baseline of phishing emails data. But, the issue I am facing is that, the apply command execution inside Splunk is not consistent, it was working fine, but now, the job is stuck at Finalizing.
When I inspected the job, it has these 2 errors.

  1. With the python csv module - Error: field larger than field limit splunk
  2. With the apply command - Error in 'apply' command: Failed to load model

I have tried clearing the cache, recreating the fit and apply model, nothing works. Not sure how to resolve this issue. Can someone please help me on this?

grana_splunk
Splunk Employee
Splunk Employee

Can you check the mlspl.log? Its under $SPLUNK_HOME/var/log/splunk and share the error message here.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

This is most likely a hardware limitation. Have you attempted to predict then do a partial fit on new data coming in? How big is your data set? How far back are you looking? Have you attempted to normalize your data through pre-processing? Whats your hardware look like?

https://docs.splunk.com/Documentation/MLApp/3.3.0/API/Methodcallingconvention

0 Karma

jkat54
SplunkTrust
SplunkTrust

I think this can be resolved by increasing the default kv limit value as described here:

http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Extractfieldsfromfileswithstructureddata#Stru...

Can you try increasing kv in limits.conf, restarting splunk and then trying again?

Thanks,
Jkat

0 Karma

jkat54
SplunkTrust
SplunkTrust

I think this can be resolved by increasing the default kv limit value as described here:

http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Extractfieldsfromfileswithstructureddata#Stru...

Can you try increasing kv in limits.conf, restarting splunk and then trying again?

Thanks,
Jkat

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...