5.73 - Invalid memory access from AllocateMemory() during program termination

Just starting out? Need help? Post your questions and find answers here.
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

5.73 - Invalid memory access from AllocateMemory() during program termination

Post by Mistrel »

I'm experiencing intermittent [ERROR] Invalid memory access errors (read error at a random address) at an AllocateMemory() after a program has reached the end (no End keyboard; the program simply reaches the end of the main source file).

The access error is inconsistent on each run but appears to be a race condition with an allocation which occurs in another thread. I have been unable to reproduce this with a small test but it always appears at the same line in my program.

I'm reporting this as a bug instead of a coding question as I would like for AllocateMemory() or the process termination code to be looked at to see if there is anything obvious for this particular use case that might have been overlooked.

It would be possible to try and signal the thread to end before termination but I don't think this kind of housekeeping should be necessary. At a minimum I would expect AllocateMemory() to return 0 instead of trying to access invalid memory.

I'm using PureBasic 5.73 x86 on Windows 10 x64 with the thread safety compiler option turned on and the debugger enabled. I don't know what would happen with the debugger off and the error is not consistent enough to test.
User avatar
STARGÅTE
Addict
Addict
Posts: 2067
Joined: Thu Jan 10, 2008 1:30 pm
Location: Germany, Glienicke
Contact:

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by STARGÅTE »

Mistrel wrote: Mon Nov 08, 2021 6:56 am I'm reporting this as a bug instead of a coding question as I would like for AllocateMemory() or the process termination code to be looked at to see if there is anything obvious for this particular use case that might have been overlooked.

It would be possible to try and signal the thread to end before termination but I don't think this kind of housekeeping should be necessary. At a minimum I would expect AllocateMemory() to return 0 instead of trying to access invalid memory.
Have you enabled the purifier? The purifier can search out invalid memory accesses which probably destroy the memory heap and later generate a IMA in another AllocateMemory(). I think you have a memory overwrite somewhere which destroys the memory heap, but without code I can just speculate.
However, your post is either a code question or a feature request, but it is no bug, if you do not post a example code which illustrate a valid code which generate an IMA.
PB 6.01 ― Win 10, 21H2 ― Ryzen 9 3900X, 32 GB ― NVIDIA GeForce RTX 3080 ― Vivaldi 6.0 ― www.unionbytes.de
Lizard - Script language for symbolic calculations and moreTypeface - Sprite-based font include/module
User avatar
jacdelad
Addict
Addict
Posts: 1437
Joined: Wed Feb 03, 2021 12:46 pm
Location: Planet Riesa
Contact:

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by jacdelad »

I wouldn't consideryself am expert, but from what I experienced is, if I can't reproduce it with a small(er) code, it's usually a fault caused by me.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
freak
PureBasic Team
PureBasic Team
Posts: 5929
Joined: Fri Apr 25, 2003 5:21 pm
Location: Germany

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by freak »

I'd say this is the expected behavior. You need to shutdown all threads before ending the program. Otherwise such conditions are bound to happen because the thread is still accessing resources that the main program is in the process of cleaning up.

In your case the crash is probably happening because the PB memory heap has already been destroyed when the AllocateMemory() call happens.
quidquid Latine dictum sit altum videtur
BarryG
Addict
Addict
Posts: 3294
Joined: Thu Apr 18, 2019 8:17 am

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by BarryG »

freak wrote: Mon Nov 08, 2021 8:16 pmYou need to shutdown all threads before ending the program.
Wait, what? I never knew this. I thought "End" was supposed to clean up everything, included ending threads? So I guess I should amend all my threads to watch for an "appquitting=1" global var and exit the thread before "End" is executed?
User avatar
STARGÅTE
Addict
Addict
Posts: 2067
Joined: Thu Jan 10, 2008 1:30 pm
Location: Germany, Glienicke
Contact:

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by STARGÅTE »

BarryG wrote: Tue Nov 09, 2021 12:05 am
freak wrote: Mon Nov 08, 2021 8:16 pmYou need to shutdown all threads before ending the program.
Wait, what? I never knew this. I thought "End" was supposed to clean up everything, included ending threads? So I guess I should amend all my threads to watch for an "appquitting=1" global var and exit the thread before "End" is executed?
How should "End" ending a thread? E.g, if this thread has an infinite loop, "End" can not stop the thread in a safe way, just by "KillThread".
The result is: "End" frees all resources step by step which are probably still used by the thread --> IMA.
However, I don't know in which order the objects of the libraries are freed. Perhaps, threads are killed at first, when "End" is called.
PB 6.01 ― Win 10, 21H2 ― Ryzen 9 3900X, 32 GB ― NVIDIA GeForce RTX 3080 ― Vivaldi 6.0 ― www.unionbytes.de
Lizard - Script language for symbolic calculations and moreTypeface - Sprite-based font include/module
BarryG
Addict
Addict
Posts: 3294
Joined: Thu Apr 18, 2019 8:17 am

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by BarryG »

STARGÅTE wrote: Tue Nov 09, 2021 12:39 amHow should "End" ending a thread?
I don't know; I never really thought about it before. All I know is I've seen plenty of posts about how "End" will clean up everything, just like exiting a procedure will clean up everything in it (local arrays in it are freed, etc). I just (wrongly) assumed threads would be ended, too.

I will change my code to account for this new knowledge.
User avatar
STARGÅTE
Addict
Addict
Posts: 2067
Joined: Thu Jan 10, 2008 1:30 pm
Location: Germany, Glienicke
Contact:

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by STARGÅTE »

Of cause, the thread will be ended anyway, but the question is in which order.

Here is an example, how it can fails, when a thread draws continuously on an image, while the main program calls "End" without stopping the thread:

Code: Select all

CompilerIf Not #PB_Compiler_Thread
	CompilerError "Enable thread safe"
CompilerEndIf

; Infinite loop with some drawing on an image.
Procedure WriteLoop(Image)
	Repeat
		If StartDrawing(ImageOutput(Image))
			Box(0, 0, OutputWidth(), OutputHeight(), Random($FFFFFF))  ; In some cases this line generates an IMA
			StopDrawing()
		EndIf
	ForEver
EndProcedure

Define I.i

; Create some images and the corresponding threads.
For I = 1 To 16
	CreateImage(I, 160, 160)
	CreateThread(@WriteLoop(), I)
Next
; Some more images without thread.
For I = 17 To 32
	CreateImage(I, 160, 160)
Next

; Short delay
Delay(1000)

; Call "End" without stopping the threads
End
; Durring "End" the images are freed, but the threads still try to draw on them.
Execution log:
Successful
Successful
IMA
Successful
Successful
Successful
Successful
IMA
IMA
Successful
...
Edit: Code edit
PB 6.01 ― Win 10, 21H2 ― Ryzen 9 3900X, 32 GB ― NVIDIA GeForce RTX 3080 ― Vivaldi 6.0 ― www.unionbytes.de
Lizard - Script language for symbolic calculations and moreTypeface - Sprite-based font include/module
Mistrel
Addict
Addict
Posts: 3415
Joined: Sat Jun 30, 2007 8:04 pm

Re: 5.73 - Invalid memory access from AllocateMemory() during program termination

Post by Mistrel »

freak wrote: Mon Nov 08, 2021 8:16 pm ... such conditions are bound to happen because the thread is still accessing resources that the main program is in the process of cleaning up.

In your case the crash is probably happening because the PB memory heap has already been destroyed when the AllocateMemory() call happens.
If this is what PureBasic is doing then I have to disagree with this implementation of process termination. These kinds of memory access errors can occur due to the inherent unpredictability of trying to intentionally perform this kind of manual cleanup during process termination. The recommendation by Raymond Chen is to simply halt any work that is being done and end the process.

KillThread() in this regard is also risky because it's unknown what other threads may exist that rely upon another. The recommended procedure is to simply ExitProcess() and, if you really want to lock down any existing threads, enumerate through anything that is not the main thread and SuspendThread() instead. But it is completely unnecessary to free memory allocations and this kind of housekeeping simply delays the process termination.

My initial report was based on the assumption that some kind of behavior is occurring during process termination that is causing this access error and this meticulous memory freeing by PureBasic seems to be the cause. I could keep track of my threads and perform signaling and suspension myself but this shouldn't be necessary. If you are insistent on freeing memory then halting all active threads should be part of this cleanup process to mitigate these kinds of access errors.

I tried to call EndProcess_(0) directly but this results in a debugger error "The debugged executable quit unexpectedly." The same occurs with TerminateProcess_(GetCurrentProcess_(),0).

If I had anything I needed to cleanup in my threads I would do so but I just want the process to end without having to keep track of them all and signal them to end. This is not always practical without tightly coupling anything and everything that might use a thread now and in the future to the main thread of execution. This is very undesirable and should not be needed.
Post Reply