topic Re: Upload large file in Splunk Search

Upload large file

mwdbhyat — Wed, 07 Sep 2016 16:23:12 GMT

Hi,

How does one upload files larger than 500mb? I get an error "File too large. The file selected is 996Mb. Maximum file size is 500Mb" Is this due to using the trial ?

Thanks

Re: Upload large file

somesoni2 — Wed, 07 Sep 2016 16:27:24 GMT

The 500MB limit is for uploading file from Splunk Web, regardless of the license type.

http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Uploaddata

Re: Upload large file

ChrisG — Wed, 07 Sep 2016 16:28:31 GMT

Yes, both the trial and free licenses have an indexing limit of 500MB a day. For file upload on any license, the maximum file size is 500 MB, as the UI indicates.

Re: Upload large file

dbcase — Wed, 07 Sep 2016 16:38:30 GMT

I've had the same problem. You can use the Linux SPLIT command to break the file up into smaller parts

http://askubuntu.com/questions/54579/how-to-split-larger-files-into-smaller-parts

Re: Upload large file

mwdbhyat — Wed, 07 Sep 2016 17:03:37 GMT

Thanks - Is there a way around this without using the Web, not splitting the file ?

Re: Upload large file

dbcase — Wed, 07 Sep 2016 17:06:49 GMT

I don't think so but you could try using the universal forwarder instead of uploading the file

Re: Upload large file

dbcase — Wed, 07 Sep 2016 17:08:24 GMT

Is it possible to zip the file so it is less than 500MB?

https://answers.splunk.com/answers/279/does-splunk-index-gzip-files.html

Re: Upload large file

mwdbhyat — Wed, 07 Sep 2016 17:29:17 GMT

Figured it out with the CLI.. did a oneshot monitor. http://docs.splunk.com/Documentation/Splunk/6.4.2/Data/MonitorfilesanddirectoriesusingtheCLI

Re: Upload large file

dbcase — Wed, 07 Sep 2016 17:31:27 GMT

Ah good to know, haven't used that before

Re: Upload large file

marnall — Wed, 27 Mar 2024 22:04:00 GMT

For people finding this question in the years after 2016, you can set the max_upload_size setting in web.conf

[settings] # set to the MB max max_upload_size = 500 # you can also set a larger splunkdConnectionTimeout value so it wont timeout when uploading splunkdConnectionTimeout=600

ref: https://docs.splunk.com/Documentation/Splunk/9.2.0/Admin/Webconf

Re: Upload large file

Rizqi_Iskandar — Mon, 22 Jul 2024 03:38:00 GMT

btw, where can i find web.conf in windows?

Because i cant find the right one to edit this file

Re: Upload large file

inventsekar — Mon, 22 Jul 2024 11:45:06 GMT

You can find it here..

For list of all config files

https://docs.splunk.com/Documentation/Splunk/9.2.2/Admin/Listofconfigurationfiles

Re: Upload large file

Rizqi_Iskandar — Mon, 22 Jul 2024 14:45:22 GMT

I have edit the max_upload size from 500 to 8000 but still it cant upload enterprise security.

I try this way repeatly and restart splunk everytime i save this configuration, but nothing happen

Do you have other way to install splunk ES on windows?

Re: Upload large file

marnall — Mon, 22 Jul 2024 18:54:29 GMT

What happens if you run btool on the settings stanza and grep for max_upload_size?

e.g.

/opt/splunk/bin/splunk btool web list settings | grep max_upload

If it shows a value other than 8000, then likely your web.conf file is in the wrong place, or being overridden by another.

Re: Upload large file

Rizqi_Iskandar — Mon, 22 Jul 2024 21:57:05 GMT

I think this is happen because its run on Windows

I will try to install it on Linux first and i will let you know if the problem fixed

thanks for the help dude

Re: Upload large file

marnall — Tue, 23 Jul 2024 20:19:38 GMT

Sorry, I am too used to Linux. I believe the equivalent btool command on windows is:

$splunkhome$/bin/splunk.exe btool web list settings | FINDSTR max_upload

Re: Upload large file

AceAxis — Sat, 07 Mar 2026 21:13:04 GMT

The default upload size for files in Splunk is 500Mb to consider the browser memory usage. The good part is that this can be altered, as admins usually upload files 10 to 50 Gb in size using CLI ingestion.

The command below can be used to change the upload size to 2Gb. Note - run the CMD as administrator.

echo [settings] > "C:\Program Files\Splunk\etc\system\local\web.conf"
echo max_upload_size = 2048 >> "C:\Program Files\Splunk\etc\system\local\web.conf"

size can be increased by increasing the limit. Post this restart the Splunk.

cd "C:\Program Files\Splunk\bin"
splunk restart

Re: Upload large file

PickleRick — Sun, 08 Mar 2026 21:53:43 GMT

Don't do that!

Firstly, it's not a good practice to put settings into system/local except for very few settings which are really, really local.

But most importantly, if you run the supplied commands, they will overwrite any settings you might have already had there! (and while it's not the best practice, people often put some stuff there; like cert definitions).