Background processing

Everything else that doesn't fall into one of the other PB categories.
lesserpanda
User
User
Posts: 65
Joined: Tue Feb 11, 2020 7:50 am

Background processing

Post by lesserpanda »

Hi guys, is there a way to throw a procedure into the background and then trigger an event when it comes back with the results?

Much like Observables in JS. I'm writing an IMAP search for email and then do something, but the IMAP inbox is fairly large like 10GB, it takes time. I just want it to run, but come back with the results.

Also, I want to run several parallel procedures of the same thing. So one of them would be search for A, the other would be search for B.

Thanks
User avatar
Demivec
Addict
Addict
Posts: 4086
Joined: Mon Jul 25, 2005 3:51 pm
Location: Utah, USA

Re: Background processing

Post by Demivec »

You would use threads (and probably mutexes and semaphores) and PostEvent().
lesserpanda
User
User
Posts: 65
Joined: Tue Feb 11, 2020 7:50 am

Re: Background processing

Post by lesserpanda »

Thanks for pointing me in the right direction. Much appreciated.

So this is what I'm doing

Create a thread with CreateThread() and push it back to the background. I use WaitThread() to wait for the thread to finish?

Question.

1. How do I know how many threads I can do before things go haywire? Like a limit?
2. So if I setup a button to run a procedure, thread it, is there an event which is triggered to let me know that it's done? Or is the WaitThread() the way to go for?

Thanks
BarryG
Addict
Addict
Posts: 3292
Joined: Thu Apr 18, 2019 8:17 am

Re: Background processing

Post by BarryG »

I don't think there's a thread limit, other than the amount of RAM on your PC.

Important: When using threads, make sure to ENABLE "Create threadsafe executable" in the Compiler Settings.

Here's a thread example:

Code: Select all

Procedure Background(null)
  timeout.q=ElapsedMilliseconds()+2000
  Repeat
    SetWindowTitle(0,"Timeout: "+Str(Abs(ElapsedMilliseconds()-timeout)))
    Delay(25)
  Until ElapsedMilliseconds()>timeout
  SetWindowTitle(0,"PureBasic")
  DisableGadget(0, #False)
EndProcedure

If OpenWindow(0, 100, 200, 200, 50, "PureBasic", #PB_Window_SystemMenu)
 
  ButtonGadget(0, 10, 10, 180, 25, "Click to start and wait for thread")
 
  Repeat
    Event = WaitWindowEvent()
    If Event = #PB_Event_Gadget
      DisableGadget(0, #True)
      CreateThread(@Background(),0)
    ElseIf Event = #PB_Event_CloseWindow
      Quit = 1
    EndIf
   
  Until Quit = 1
 
EndIf
User avatar
mk-soft
Always Here
Always Here
Posts: 5335
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Background processing

Post by mk-soft »

Changing the GUI does not work from threads. On Windows it works partially, and macOS and Linux will crash.
To change the GUI from threads, see signature ThreadToGUI (PostEvents)

Examples of thread management, see example Mini Thread Control :wink:
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
TI-994A
Addict
Addict
Posts: 2512
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore
Contact:

Re: Background processing

Post by TI-994A »

This example creates and executes two threads which simulate processing times of ten and twenty seconds respectively. Upon completion, each thread notifies the main thread via the PostEvent() function, and passes along some numerical and string data as well. It also demonstrates the method to kill the threads gracefully, without using the KillThread() function.

As BarryG has mentioned, be sure to select the Create threadsafe executable option under the Compiler -> Compiler Options menu. And as mk-soft has said, GUI elements should not be manipulated from within threads. Always post a corresponding event to the main thread and let it handle any and all GUI changes.

Code: Select all

; ensure that [Compiler > Compiler Options > Create threadsafe executable] is selected

#thread1CompleteEvent = #PB_Event_FirstCustomValue
#thread2CompleteEvent = #PB_Event_FirstCustomValue + 1

Global thread1Quit, thread2Quit

Procedure thread1(startTime)
  
  ; numerical value to be passed
  eData = 1234567890
  
  ; simulate processing time (10 seconds)
  While ElapsedMilliseconds() - startTime < 10000
    
    ; gracefully terminate thread if quit flag is set
    If thread1Quit : Break : EndIf  
    
  Wend
  
  ; post completion event if thread not terminated
  If Not thread1Quit
    PostEvent(#thread1CompleteEvent, #PB_Ignore, #PB_Ignore, #PB_Ignore, eData)
  EndIf
  
EndProcedure

Procedure thread2(startTime)
  
  ; string value to be passed - write into memory
  *strPointer = AllocateMemory(100)
  PokeS(*strPointer, "Hello, thread!")
  
  ; simulate processing time (20 secoonds)
  While ElapsedMilliseconds() - startTime < 20000
    
    ; gracefully terminate thread if quit flag is set
    If thread2Quit : Break : EndIf  
    
  Wend
  
  ; post completion event if thread not terminated
  If Not thread2Quit
    PostEvent(#thread2CompleteEvent, #PB_Ignore, #PB_Ignore, #PB_Ignore, *strPointer)
  EndIf
  
EndProcedure

wFlags = #PB_Window_SystemMenu | #PB_Window_ScreenCentered
If OpenWindow(0, #PB_Ignore, #PB_Ignore, 300, 180, "Thread Example", wFlags)
  ResizeWindow(0, #PB_Ignore, WindowY(0) - 120, #PB_Ignore, #PB_Ignore)
  
  TextGadget(0, 0, 20, 300, 30, "", #PB_Text_Center)
  TextGadget(1, 0, 50, 300, 30, "", #PB_Text_Center)  
  
  ButtonGadget(2, 10, 90, 280, 30, "Kill Thread 1")
  ButtonGadget(3, 10, 130, 280, 30, "Kill Thread 2")
  
  startTime1 = ElapsedMilliseconds()
  startTime2 = ElapsedMilliseconds()
  
  testThread1 = CreateThread(@thread1(), startTime1)
  testThread2 = CreateThread(@thread2(), startTime2)
  
  ; for displaying the countdown timers
  AddWindowTimer(0, 0, 1000)
  
  Repeat
    
    Select WaitWindowEvent()
        
      Case #PB_Event_CloseWindow
        appQuit = 1
        
      Case #PB_Event_Timer
        counter1 = 10 - ((ElapsedMilliseconds() - startTime1) / 1000)
        counter2 = 20 - ((ElapsedMilliseconds() - startTime2) / 1000)
        
        If counter1 > -1 And IsThread(testThread1)
          SetGadgetText(0, "Thread 1 completing in " + Str(counter1) + " seconds...")
        EndIf
        
        If counter2 > -1 And IsThread(testThread2)
          SetGadgetText(1, "Thread 2 completing in " + Str(counter2) + " seconds...")
        EndIf
        
      Case #thread1CompleteEvent
        SetGadgetText(0, "Thread 1 completed!")
        MessageRequester("Thread 1 Complete:",
                         "Numerical data = " + Str(EventData()),
                         #PB_MessageRequester_Ok)    
        
      Case #thread2CompleteEvent
        SetGadgetText(1, "Thread 2 completed!")
        MessageRequester("Thread 2 Complete:",
                         "String data = " + PeekS(EventData()),
                         #PB_MessageRequester_Ok)   
        
      Case #PB_Event_Gadget
        
        Select EventGadget()
            
          Case 2
            If IsThread(testThread1)
              thread1Quit = #True
              SetGadgetText(0, "Thread 1 terminated!")
            EndIf
            
            
          Case 3
            If IsThread(testThread2)
              thread2Quit = #True
              SetGadgetText(1, "Thread 2 terminated!")
            EndIf
            
        EndSelect
        
    EndSelect
    
  Until appQuit
  
EndIf
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too! Please visit my YouTube Channel :D
lesserpanda
User
User
Posts: 65
Joined: Tue Feb 11, 2020 7:50 am

Re: Background processing

Post by lesserpanda »

Thank you for the examples. Understand the concepts.

Got a question ... for this set of code, AllocateMemory and PokeS

Code: Select all

  *strPointer = AllocateMemory(100)
  PokeS(*strPointer, "Hello, thread!")
What's would be the practical usage of this direct access?
User avatar
skywalk
Addict
Addict
Posts: 3972
Joined: Wed Dec 23, 2009 10:14 pm
Location: Boston, MA

Re: Background processing

Post by skywalk »

The size and location of str pointer is controlled by you. Not the internal PB string library, which dynamically adjusts its memory and location. That is trickier to manage in dll and threads.
The nice thing about standards is there are so many to choose from. ~ Andrew Tanenbaum
User avatar
TI-994A
Addict
Addict
Posts: 2512
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore
Contact:

Re: Background processing

Post by TI-994A »

lesserpanda wrote:...for this set of code, AllocateMemory and PokeS

Code: Select all

  *strPointer = AllocateMemory(100)
  PokeS(*strPointer, "Hello, thread!")
What's would be the practical usage of this direct access?
There is no direct way of passing data between a thread and the main program. However, the PostEvent() function is able to send some numerical data through its event data parameter. This example shows how to use this EventData() function to pass such values.

The first thread simply passes a numerical value to the main thread (123345678890), while the second thread writes some string (Hello, thread!) into memory, then passes the memory address (*strPointer) to the main thread. The AllocateMemory(), PokeS() and PeekS() functions, as used in the example, perform this task.

Alternatively, global variables could also be used to achieve this very same thing.
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too! Please visit my YouTube Channel :D
User avatar
mk-soft
Always Here
Always Here
Posts: 5335
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Background processing

Post by mk-soft »

@TI-994A

very nice memory leak. I don't see FreeMemory.

I like this with AllocateString and FreeString...

Code: Select all

; ensure that [Compiler > Compiler Options > Create threadsafe executable] is selected

CompilerIf #PB_Compiler_Thread = 0
  CompilerError "Use Compiler Option ThreadSafe!"
CompilerEndIf

Enumeration CustomEvent #PB_Event_FirstCustomValue
  #thread1CompleteEvent
  #thread2CompleteEvent
EndEnumeration

Procedure AllocateString(String.s)
  Protected *Memory.String = AllocateStructure(String)
  If *Memory
    *Memory\s = String
  EndIf
  ProcedureReturn *Memory
EndProcedure

Procedure.s FreeString(*Memory.String)
  Protected r1.s
  If *Memory
    r1 = *Memory\s
    FreeStructure(*Memory)
  EndIf
  ProcedureReturn r1
EndProcedure

Global thread1Quit, thread2Quit

Procedure thread1(startTime)
  
  ; numerical value to be passed
  eData = 1234567890
  
  ; simulate processing time (10 seconds)
  While ElapsedMilliseconds() - startTime < 10000
    
    ; gracefully terminate thread if quit flag is set
    If thread1Quit : Break : EndIf  
    
  Wend
  
  ; post completion event if thread not terminated
  If Not thread1Quit
    PostEvent(#thread1CompleteEvent, #PB_Ignore, #PB_Ignore, #PB_Ignore, eData)
  EndIf
  
EndProcedure

Procedure thread2(startTime)
  
  ; string value to be passed - write into memory
  strData.s = "Hello, thread!"
  
  ; simulate processing time (20 secoonds)
  While ElapsedMilliseconds() - startTime < 20000
    
    ; gracefully terminate thread if quit flag is set
    If thread2Quit : Break : EndIf  
    
  Wend
  
  ; post completion event if thread not terminated
  If Not thread2Quit
    PostEvent(#thread2CompleteEvent, #PB_Ignore, #PB_Ignore, #PB_Ignore, AllocateString(strData))
  EndIf
  
EndProcedure

wFlags = #PB_Window_SystemMenu | #PB_Window_ScreenCentered
If OpenWindow(0, #PB_Ignore, #PB_Ignore, 300, 180, "Thread Example", wFlags)
  ResizeWindow(0, #PB_Ignore, WindowY(0) - 120, #PB_Ignore, #PB_Ignore)
  
  TextGadget(0, 0, 20, 300, 30, "", #PB_Text_Center)
  TextGadget(1, 0, 50, 300, 30, "", #PB_Text_Center)  
  
  ButtonGadget(2, 10, 90, 280, 30, "Kill Thread 1")
  ButtonGadget(3, 10, 130, 280, 30, "Kill Thread 2")
  
  startTime1 = ElapsedMilliseconds()
  startTime2 = ElapsedMilliseconds()
  
  testThread1 = CreateThread(@thread1(), startTime1)
  testThread2 = CreateThread(@thread2(), startTime2)
  
  ; for displaying the countdown timers
  AddWindowTimer(0, 0, 1000)
  
  Repeat
    
    Select WaitWindowEvent()
        
      Case #PB_Event_CloseWindow
        appQuit = 1
        
      Case #PB_Event_Timer
        counter1 = 10 - ((ElapsedMilliseconds() - startTime1) / 1000)
        counter2 = 20 - ((ElapsedMilliseconds() - startTime2) / 1000)
        
        If counter1 > -1 And IsThread(testThread1)
          SetGadgetText(0, "Thread 1 completing in " + Str(counter1) + " seconds...")
        EndIf
        
        If counter2 > -1 And IsThread(testThread2)
          SetGadgetText(1, "Thread 2 completing in " + Str(counter2) + " seconds...")
        EndIf
        
      Case #thread1CompleteEvent
        SetGadgetText(0, "Thread 1 completed!")
        MessageRequester("Thread 1 Complete:",
                         "Numerical data = " + Str(EventData()),
                         #PB_MessageRequester_Ok)    
        
      Case #thread2CompleteEvent
        SetGadgetText(1, "Thread 2 completed!")
        MessageRequester("Thread 2 Complete:",
                         "String data = " + FreeString(EventData()),
                         #PB_MessageRequester_Ok)   
        
      Case #PB_Event_Gadget
        
        Select EventGadget()
            
          Case 2
            If IsThread(testThread1)
              thread1Quit = #True
              SetGadgetText(0, "Thread 1 terminated!")
            EndIf
            
            
          Case 3
            If IsThread(testThread2)
              thread2Quit = #True
              SetGadgetText(1, "Thread 2 terminated!")
            EndIf
            
        EndSelect
        
    EndSelect
    
  Until appQuit
  
EndIf
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
User avatar
TI-994A
Addict
Addict
Posts: 2512
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore
Contact:

Re: Background processing

Post by TI-994A »

mk-soft wrote:very nice memory leak. I don't see FreeMemory...
Because it's not required in the context of this example. All allocated memory blocks are automatically freed when the program ends.

So, no memory leak. :wink:
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too! Please visit my YouTube Channel :D
User avatar
mk-soft
Always Here
Always Here
Posts: 5335
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: Background processing

Post by mk-soft »

TI-994A wrote:
mk-soft wrote:very nice memory leak. I don't see FreeMemory...
Because it's not required in the context of this example. All allocated memory blocks are automatically freed when the program ends.

So, no memory leak. :wink:
In this context, maybe okay.
But you should already post memory safe examples :mrgreen:
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
Post Reply