Python Forum
How to scan huge files and make it in chunks
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to scan huge files and make it in chunks
#1
Question :

Scan a 10M log file, and create chunks of files of 100 chars lines in each chunks, from the 10M log file. 100th character can be end of a word or a space. If it’s a broken word, add it to next file
Reply
#2
A 10M file is quite small. Why on earth would you want to break it into 100 byte files?
Reply
#3
This sounds like a fairly simple homework problem.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Sad How to split a String from Text Input into 40 char chunks? lastyle 7 1,122 Aug-01-2023, 09:36 AM
Last Post: Pedroski55
  is there an itertor of chunks? Skaperen 8 1,928 Jul-22-2022, 10:17 PM
Last Post: Skaperen
  Scan for Bluetooth device korenron 0 2,611 Jan-10-2022, 01:06 PM
Last Post: korenron
  Exporting a huge dataFrame stylingpat 5 15,544 Mar-23-2021, 12:13 AM
Last Post: stylingpat
  Convert looping home security to a one time scan of switches and sensors duckredbeard 0 1,748 Dec-08-2020, 04:31 AM
Last Post: duckredbeard
  scan drives in windows from Cygwin RRR 1 1,622 Nov-29-2020, 04:34 PM
Last Post: Larz60+
  Pip prints huge error when installing p5 hayden2s 1 2,243 Aug-08-2020, 02:30 PM
Last Post: snippsat
  Huge CSV Reading and Sorting for comparison akshaynimkar 3 2,468 Aug-04-2020, 11:20 AM
Last Post: Larz60+
  how to divide one big equation entered as a string to small chunks gungurbuz 1 1,614 May-28-2020, 03:46 PM
Last Post: Larz60+
  Iterate through all files in a folder and make simple calculation danielappleton 2 2,311 Apr-01-2020, 09:57 AM
Last Post: danielappleton

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020