Python Forum
Upload big file on the server HTTP protocol
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Upload big file on the server HTTP protocol
#1
Now I am trying to learn some basics about cgi scripts (on Python). I want to implement saving a file on the server,when User send it (push the button in html form). I made it like in manual : http://cgi.tutorial.codepoint.net/big-file-upload

However when I send a large csv file ,which includes 100 000 rows, I get response that file is written successfully,but really few number of rows were written (around 30000) and the process seems to stop. What are the possible reasons?And how to deal with these files in future? Scrypt:

# -*- coding: utf-8 -*-
import cgi, os
import cgitb 

cgitb.enable()

try: # Windows needs stdio set for binary mode.
    import msvcrt
    msvcrt.setmode (0, os.O_BINARY) # stdin  = 0
    msvcrt.setmode (1, os.O_BINARY) # stdout = 1
except ImportError:
    pass

form = cgi.FieldStorage()

# Generator to buffer file chunks
def fbuffer(f, chunk_size=1024):
    while True:
        chunk = f.read(chunk_size)
        if not chunk: break
        yield chunk

# A nested FieldStorage instance holds the file
fileitem = form['file']

# Test if the file was uploaded
if fileitem.filename:


    fn = os.path.basename(fileitem.filename)
    f = open('C:/Users/timna/' + fn, 'wb')

    # Read the file in chunks
    for chunk in fbuffer(fileitem.file):
      f.write(chunk)
    f.close()
Server:

# -*- coding: utf-8 -*-
from http.server import HTTPServer, CGIHTTPRequestHandler
server_address = ("", 8000)
httpd = HTTPServer(server_address, CGIHTTPRequestHandler)
httpd.serve_forever()
Thanks for the response!
Reply
#2
(May-15-2020, 03:10 AM)Timych Wrote: Now I am trying to learn some basics about cgi scripts (on Python)
You should not Confused
Python Wrote:cgi is going to be deprecated in Python 3.8 and removed in 3.10
So the modern way to do CGI in Python is the use a smaller web-framework like Flask, Bottle, ect...
Python community has written WSGI(all Python code) to replace CGI(had a lot of problems that could not be fixed).

So eg Flask is layer above WSGI to make all easier connect to web with HTML/CSS than using WGSI directly.
Uploading Files | Flask file upload.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  upload big file in Django with process bar and i get error : MemoryError ma_norouzifar 3 4,341 Aug-06-2019, 06:57 AM
Last Post: fishhook
  upload folder in server ftp falloff 3 8,145 Nov-02-2018, 09:46 PM
Last Post: Gribouillis
  http.server start delay pastacolsugo 1 2,947 Apr-18-2018, 08:58 PM
Last Post: micseydel
  python largesize(6GB) file upload fails nithya_g 2 4,098 Jan-29-2018, 09:11 PM
Last Post: micseydel
  file upload from windows10 machine using send_key in selenium fails nithya_g 3 4,591 Jan-12-2018, 09:32 PM
Last Post: metulburr
  Rewriting HTTP Server from Arduino codingKnight 1 3,027 Sep-25-2017, 10:47 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020