Memory leaks using ReadProgramString, RunProgram, WriteProgramStringN etc?

Everything else that doesn't fall into one of the other PB categories.
User avatar
pdwyer
Addict
Addict
Posts: 2813
Joined: Tue May 08, 2007 1:27 pm
Location: Chiba, Japan

Memory leaks using ReadProgramString, RunProgram, WriteProgramStringN etc?

Post by pdwyer »

I don't think this is a PB issue as such, more likely best practices when using IO communication between processes using

RunProgram
ReadProgramString
WriteProgramStringN
AvailableProgramOutput

etc

I'm not sure if I have a specific question in this case, I'm just wondering if anyone has used these for communications between processes and run into memory leaks and has any lessons learned they can share.

I've been surprised a few times when getting communications out of sync, perhaps the spawned app is trying to send information back at a different rate to the parent collecting it and I look away from the screen for a minute and 64gb has been eaten up and my system is slowing to a crawl.

That's an impressively fast way to eat memory for two apps that just pass back a few small strings.

Generally, I'd prefer to use network comms but in this case, one of the programs is out of my control and needs this stdio comms method. I seem to have it working okay now but I'm not really sure what causes the leaks or how to avoid it other than some trial and error on timing. (try to process input the at the same rate as the sender outputs in terms of function calls per second)

There is also a secondary leak of some kind that doesn't go away when the process dies. It's much slower and after thousands of executions of these apps eats memory and the system doesn't seem to show anywhere that is using it. A reboot is required in these cases.

Anyway, if anyone has any hints or experience on these things, I'd be interested to learn more

Thanks
Paul Dwyer

“In nature, it’s not the strongest nor the most intelligent who survives. It’s the most adaptable to change” - Charles Darwin
“If you can't explain it to a six-year old you really don't understand it yourself.” - Albert Einstein