Program read routines are slow

Mac OSX specific forum
User avatar
deseven
Enthusiast
Enthusiast
Posts: 362
Joined: Wed Jan 12, 2011 3:48 pm
Location: Serbia
Contact:

Program read routines are slow

Post by deseven »

I've stumbled upon this when I needed to read an output of ffprobe several thousand times. As it turned out, you can't do it efficiently in PB, it takes too much resources and it gets much worse when the amount of data gets bigger. Adding delays don't help either.

Look at this example:

Code: Select all

EnableExplicit

Define i
Define program
Define output.s
Define startTime.i
Define arg.s = "long test string for stdout"
For i = 1 To 5 ; make this number bigger to see even bigger difference
  arg + arg
Next

Define args = CocoaMessage(0,0,"NSArray arrayWithObject:$",@arg)
Define readPipe,readHandle,outputData,outputNative

Debug "testing RunProgram()..."
startTime = ElapsedMilliseconds()
For i = 1 To 1000
  output = ""
  program = RunProgram("/bin/echo",arg,"",#PB_Program_Open|#PB_Program_Read)
  If program
    While ProgramRunning(program)
      If AvailableProgramOutput(program) ; comment this to see some improvement
        output + ReadProgramString(program) ; ReadProgramData() also isn't any better here
      EndIf
    Wend
    CloseProgram(program)
  EndIf
  If output <> arg
    Debug "failed"
  EndIf
Next
Debug "RunProgram() took " + Str(ElapsedMilliseconds() - startTime)

Debug "testing NSTask..."
startTime = ElapsedMilliseconds()
For i = 1 To 1000
  output = ""
  program = CocoaMessage(0,CocoaMessage(0,0,"NSTask alloc"),"init")
  If program
    CocoaMessage(0,program,"setLaunchPath:$",@"/bin/echo")
    CocoaMessage(0,program,"setArguments:",args)
    readPipe = CocoaMessage(0,0,"NSPipe pipe")
    readHandle = CocoaMessage(0,readPipe,"fileHandleForReading")
    CocoaMessage(0,program,"setStandardOutput:",readPipe)
    CocoaMessage(0,program,"launch")
    outputData = CocoaMessage(0,readHandle,"readDataToEndOfFile")
    CocoaMessage(0,readHandle,"closeFile")
    While CocoaMessage(0,program,"isRunning") ; this isn't even needed, because the program already closed its output and we can safely detach 
    Wend ; but since there is no such functionality in PB I added it to have more or less fair comparison
    If outputData
      outputNative = CocoaMessage(0,CocoaMessage(0,0,"NSString alloc"),"initWithData:",outputData,"encoding:",#NSUTF8StringEncoding)
      output = PeekS(CocoaMessage(0,outputNative,"UTF8String"),-1,#PB_UTF8)
    EndIf
    CocoaMessage(0,program,"release")
  EndIf
  If output <> arg + ~"\n"
    Debug "failed"
  EndIf
Next
Debug "NSTask took " + Str(ElapsedMilliseconds() - startTime)