Avoid output buffering when redirecting large data (40KB) to another process

[Ramphic]

I have following python script, which is multi-threaded and a C# script from which I call python code. The data outputted from python script is not getting printed in C# code when data size is large (like 40KB). It's working fine for less than 100 bytes size. I am already using "-u" option while running python, to disable output buffering. But it's still buffering for large data (Some kind of block or line buffereing is happening, how to avoid this?)

How to get this work, please ., or any other alternate ways?


My python code:

import _thread
import time

# Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print ("%s: %s" % ( threadName, time.ctime(time.time()) ))

# Create two threads as follows
try:
   _thread.start_new_thread( print_time, ("T"*40000, 1, ) )
   _thread.start_new_thread( print_time, ("Thread-2", 0.5, ) )
except:
   print ("Error: unable to start thread")

x = input()
print (x)

My c# code:

using System;
using System.Collections;
using System.Diagnostics;
using UnityEngine;

public class MultiThreadCall : MonoBehaviour
{
    private ProcessStartInfo startInfo;
    private Process process;

    private void Start()
    {
        initialiseProcessStartInfo();
    }

    private void OnGUI()
    {
        if (GUI.Button(new Rect(5, 5, 50, 50), "START"))
        {
            if(process == null)
               run_cmd();
        }

        if (GUI.Button(new Rect(500, 5, 100, 50), "KILL"))
        {
            if (process != null)
               process.Kill();
        }

    }


    private void initialiseProcessStartInfo()
    {
        startInfo = new ProcessStartInfo();
        startInfo.FileName = "C:/Windows/py.exe";
        startInfo.Arguments = @"-u C:\Users\name\Desktop\multithreaded.py";
        startInfo.WorkingDirectory = @"C:\Users\name\Desktop\";

        startInfo.CreateNoWindow = true;
        startInfo.UseShellExecute = false;
        startInfo.RedirectStandardOutput = true;
        startInfo.RedirectStandardError = true;
        startInfo.WindowStyle = ProcessWindowStyle.Hidden;
    }

    private void run_cmd()
    {
        process = new Process();
        process.StartInfo = startInfo;
        process.EnableRaisingEvents = true;
        process.OutputDataReceived += new DataReceivedEventHandler(OnDataReceived);
        process.ErrorDataReceived += new DataReceivedEventHandler(OnErrorReceived);

        process.Start();
        process.BeginOutputReadLine();
        process.BeginErrorReadLine();


    }

    private void OnDestroy()
    {
        if (process != null)
            process.Kill();
    }

    private void OnDataReceived(object sender, DataReceivedEventArgs e)
    {
        if (e.Data != null && e.Data.Length > 0)
        {
            UnityEngine.Debug.Log(e.Data + Environment.NewLine);
        }
    }

    private void OnErrorReceived(object sender, DataReceivedEventArgs e)
    {
        if (e.Data != null && e.Data.Length > 0)
        {
            UnityEngine.Debug.Log(e.Data + Environment.NewLine);
        }
    }

}

[Gribouillis]

You could perhaps add calls to sys.stdout.flush() in the code.

It may not be directly related to the issue, but I've always been told to use module threading instead of the low level _thread module. It would probably be a good idea to use this in your code.

[Ramphic]

Thankyou for your reply. I have even tried what you said, but the issue still remains. It's working fine for data less than a KB. If data is something like 40 KB, it's getting buffered somewhere else !!

[Larz60+]

you are creating an item of 40,000 'T's here,

"T"*40000

Is that your intention?