Python Forum

i create a web page in Django for upload big file (any file) with process bar

i used IIS and django version 2.2.2
views.py :

def upload_file(request):
if request.method == 'POST':
    form = UploadFileForm(request.POST, request.FILES)
    if form.is_valid():
        handle_uploaded_file(request.FILES['file'])
        return HttpResponse('/success/url/')
else:
    form = UploadFileForm()
return render(request, 'Demo/home.html', {'form': form})


def handle_uploaded_file(f):
 namefile = f.name
 with open(namefile, 'wb+') as destination:
    print(f.chunks())
    for chunk in f.chunks():
        destination.write(chunk)

forms.py

from django import forms


class UploadFileForm(forms.Form):
 title = forms.CharField(max_length=50)
 file = forms.FileField()

upload.js :

   $(document).ready(function () {

    $('form').on('submit', function (event) {

        event.preventDefault();

        var formData = new FormData($('form')[0]);

        $.ajax({
            xhr: function () {
                var xhr = new window.XMLHttpRequest();

                xhr.upload.addEventListener('progress', function (e) {

                    if (e.lengthComputable) {

                        console.log('Bytes Loaded: ' + e.loaded);
                        console.log('Total Size: ' + e.total);
                        console.log('Percentage Uploaded: ' + (e.loaded / e.total))

                        var percent = Math.round((e.loaded / e.total) * 100);

                        $('#progressBar').attr('aria-valuenow', percent).css('width', percent + '%').text(percent + '%');

                    }

                });

                return xhr;
            },
            type: 'POST',
            url: '/',
            data: formData,
            processData: false,
            contentType: false,
            success: function () {
                alert('File uploaded!');
            }
        });

    });

});

html:

<!DOCTYPE html>
<html lang="en">

<head>
    {% load static %}
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Document</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.4/jquery.min.js"></script>
    <script src="{% static 'Demo/upload.js' %}"></script>

    <!-- Latest compiled and minified CSS -->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css"
        integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" crossorigin="anonymous">

    <!-- Optional theme -->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap-theme.min.css"
        integrity="sha384-fLW2N01lMqjakBkx3l/M9EahuwpSfeNvV63J5ezn3uZzapT0u7EYsXMjQV+0En5r" crossorigin="anonymous">

    <!-- Latest compiled and minified JavaScript -->
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"
        integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" crossorigin="anonymous">
    </script>
</head>

<body>


    <div>
        <form action="" method="post" enctype="multipart/form-data">
            {% csrf_token %}
            {{form.as_p}}
            <input type="submit" value="ok">
        </form>

        <div class="progress">
            <div id="progressBar" class="progress-bar" role="progressbar" aria-valuenow="0" aria-valuemin="0"
                aria-valuemax="100" style="width: 0%;">
                0%
            </div>
        </div>
    </div>
</body>

</html>

i get this error :

Quote:Internal Server Error: / Traceback (most recent call last): File "C:\Python37\lib\site-packages\django\core\handlers\exception.py", line 34, in inner response = get_response(request) File "C:\Python37\lib\site-packages\django\core\handlers\base.py", line 106, in _get_response response = middleware_method(request, callback, callback_args, callback_kwargs) File "C:\Python37\lib\site-packages\django\middleware\csrf.py", line 295, in process_view request_csrf_token = request.POST.get('csrfmiddlewaretoken', '') File "C:\Python37\lib\site-packages\django\core\handlers\wsgi.py", line 110, in _get_post self._load_post_and_files() File "C:\Python37\lib\site-packages\django\http\request.py", line 314, in _load_post_and_files self._post, self._files = self.parse_file_upload(self.META, data) File "C:\Python37\lib\site-packages\django\http\request.py", line 274, in parse_file_upload return parser.parse() File "C:\Python37\lib\site-packages\django\http\multipartparser.py", line 254, in parse chunk = handler.receive_data_chunk(chunk, counters[i]) File "C:\Python37\lib\site-packages\django\core\files\uploadhandler.py", line 174, in receive_data_chunk self.file.write(raw_data) MemoryError

in settings.py i add:

Quote:FILE_UPLOAD_MAX_MEMORY_SIZE = 5368709120

DATA_UPLOAD_MAX_MEMORY_SIZE = 5368709120

what is my mistake .. is better way for upload big file with process bar in django

I'm not familiar with Django.

Maybe the print(f.chunks()) causes the MemoryError.
Normally it should not, because the print function does not consume generators.

In the documentation they write, that the method UploadedFile.chunks() should used always.
It the method returns a generator, which is consumed by the for-loop.
The for-loop itself should not cause this issue. For each iteration you get a new chunk
and the old chunk object is garbage collected.

Maybe you overwrite at some other place your settings. Does it work with small uploads?

Hi tanks for your answer

I don't know what is problems it is work for small file upload 50 mb but for 700 mb i tested my code and give me error "memory error "

As DeaD_EyE sad, "print" function can be the reason of troubles, especially in Windows (you use Windows as I guess seeing IIS).
I've run the same code on Windows 10 Pro core I7 64GB, And Linux virtual machine.

import timeit
print(timeit.timeit("print(1)", number=1000))

Now, look to results:

Windows:
0.2165976

Linux:
0.0051541

First of all rid of print. It gives you terrible overhead in any case.
Second, try your code on the usually recommended environment: Linux + uwsgi + Nginx. I can't except, that IIS is the trouble maker.

ma_norouzifar

DeaD_EyE

ma_norouzifar

fishhook