Apply command on a large field

[KrithikaRamakri]

Hi everyone, I am trying to apply logistic regression to predict phishing based on a baseline of phishing emails data. But, the issue I am facing is that, the apply command execution inside Splunk is not consistent, it was working fine, but now, the job is stuck at Finalizing.
When I inspected the job, it has these 2 errors.

1. With the python csv module - Error: field larger than field limit splunk
2. With the apply command - Error in 'apply' command: Failed to load model

I have tried clearing the cache, recreating the fit and apply model, nothing works. Not sure how to resolve this issue. Can someone please help me on this?

[grana_splunk]

Can you check the mlspl.log? Its under $SPLUNK_HOME/var/log/splunk and share the error message here.

[skoelpin]

This is most likely a hardware limitation. Have you attempted to predict then do a partial fit on new data coming in? How big is your data set? How far back are you looking? Have you attempted to normalize your data through pre-processing? Whats your hardware look like?

https://docs.splunk.com/Documentation/MLApp/3.3.0/API/Methodcallingconvention

[jkat54]

I think this can be resolved by increasing the default kv limit value as described here:

http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Extractfieldsfromfileswithstructureddata#Stru...

Can you try increasing kv in limits.conf, restarting splunk and then trying again?

Thanks,
Jkat

[jkat54]

I think this can be resolved by increasing the default kv limit value as described here:

http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Extractfieldsfromfileswithstructureddata#Stru...

Can you try increasing kv in limits.conf, restarting splunk and then trying again?

Thanks,
Jkat