StringToken() as an alternative to StringField()

Share your advanced PureBasic knowledge/code with the community.
Axolotl
Enthusiast
Enthusiast
Posts: 449
Joined: Wed Dec 31, 2008 3:36 pm

StringToken() as an alternative to StringField()

Post by Axolotl »

Function: with each call StringToken() returns the next token. different separators can be used. Because empty strings are not returned, the procedure also functions as trim().
maybe someone can use it...
i deliberately decided to use the string-pointer, the resulting limitations or weaknesses can be seen in the example calls.

Code: Select all

EnableExplicit

Procedure.s StringToken(*String.STRING, Separator$)  ; STRING -- 
	Static *s_string.CHARACTER, *s_currCh.CHARACTER 
	Protected result$, *sepCh.CHARACTER, *start.CHARACTER 

  If *String And *String <> *s_string    ; new string to tokenize .. 
		*s_string = *String      ; .. memorize the string pointer 
		*s_currCh = *s_string    ; .. point to current character 
	EndIf 

  If *s_currCh = 0  ; end of string reached by last call or not valid 
    ProcedureReturn "" 
  EndIf 

  *start = *s_currCh    ; memorize the pointer to the current character 
  While *s_currCh\c  
		*sepCh = @Separator$    ; point to separator string (start new) 
		While *sepCh\c 
			If *s_currCh\c = *sepCh\c    ; found a separator character 
				If *start <> *s_currCh     ; not at the start position 
					result$ = PeekS(*start, (*s_currCh - *start) >> 1) 
					*s_currCh + SizeOf(CHARACTER) 
					ProcedureReturn result$ 
				EndIf 
				*start = *s_currCh + SizeOf(CHARACTER)    ; new string start 
				Break  
			EndIf 
			*sepCh + SizeOf(CHARACTER) 
    Wend 
		*s_currCh + SizeOf(CHARACTER) 
  Wend  
	result$ = PeekS(*start, (*s_currCh - *start) >> 1) 
	*s_currCh = 0    ; reset current character 

	ProcedureReturn result$ 
EndProcedure ;() 


CompilerIf #PB_Compiler_IsMainFile 
  Global DemoText$, nn, ret, ret$  

  DemoText$ = "Command = DIR " + #DOUBLEQUOTE$ + "*.*" + #DOUBLEQUOTE$ + ", /s" + " " 

	ret$ = StringToken(@DemoText$, "=, ")   :Debug " => '" + ret$ + "'" 
	ret$ = StringToken(@DemoText$, "=, ")   :Debug " => '" + ret$ + "'" 
	ret$ = StringToken(@DemoText$, "=, ")   :Debug " => '" + ret$ + "'" 
	ret$ = StringToken(@DemoText$, "=, ")   :Debug " => '" + ret$ + "'" 
	ret$ = StringToken(@DemoText$, "=, ")   :Debug " => '" + ret$ + "'" 

  ; New String 
  Debug #LF$ + "====================" 
  ret$ = StringToken(@"TestA TestB", " ")  :Debug " => '" + ret$ + "'" 
  ret$ = StringToken(@"TestA TestB", " ")  :Debug " => '" + ret$ + "'"
  ret$ = StringToken(@"TestA TestB", " ")  :Debug " => '" + ret$ + "'"

  
  ; empty string works 
  Debug #LF$ + "====================" 
  ret$ = StringToken(@"", "")              :Debug " => '" + ret$ + "'  (empty string and separator)"
  ret$ = StringToken(@"", " ")             :Debug " => '" + ret$ + "'  (empty string)"
  ret$ = StringToken(@"Test with ", "")    :Debug " => '" + ret$ + "'  (empty separator)"

  ; new string 
  Debug #LF$ + "====================" 
  DemoText$ = "Key1,Test1;Key2,Test2;  , ;		,		;Key3	,Test3	;" 
  For nn = 1 To 10 
  	ret$ = StringToken(@DemoText$, ",;	 ")   :Debug " => '" + ret$ + "'" 
  Next nn 

  ; same string .. needs a reset 
  StringToken(@"", "") 

  ; same variable has the same pointer !!! 
  DemoText$ = "This is	a test	   	 	 	 	string	 	to	 	 	 	see	   	 	 	 	 	if	split is	working."
  Debug #LF$ + "====================" 
  For nn = 1 To 15
  	ret$ = StringToken(@DemoText$, #CRLF$ + #TAB$ + #FF$ + #VT$ + " ")    :Debug "["+nn+"] = '" + ret$ + "'" 
  Next nn 
CompilerEndIf 
 
Mostly running PureBasic <latest stable version and current alpha/beta> (x64) on Windows 11 Home
User avatar
Thorsten1867
Addict
Addict
Posts: 1366
Joined: Wed Aug 24, 2005 4:02 pm
Location: Germany

Re: StringToken() as an alternative to StringField()

Post by Thorsten1867 »

You could also do all the tokens at once.

Code: Select all

#Example = 1

Procedure StringToken(String.s, Separator.s, List Result.s())
  Define.i c, sLen, Pos, EndPos, StartPos = 1
  
  ClearList(Result())

  sLen = Len(String)
  
  While StartPos < Len(String)
    
    EndPos = sLen
    
    For c=1 To Len(Separator)
      
      If Mid(String, StartPos, 1) = Mid(Separator, c, 1)
        StartPos + 1
        Endpos = StartPos
        Break
      Else  
        Pos = FindString(String, Mid(Separator, c, 1), StartPos)
        If Pos And Pos < EndPos : EndPos = Pos : EndIf 
      EndIf  

    Next

    If EndPos <> StartPos
      If AddElement(Result()) : Result() = Mid(String, StartPos, EndPos - Startpos) : EndIf 
    EndIf
    
    StartPos = EndPos
  Wend
  
EndProcedure  

NewList Result.s()

Select #Example
  Case 1
    Text$ = "Command = DIR " + #DOUBLEQUOTE$ + "*.*" + #DOUBLEQUOTE$ + ", /s" + " " 
    StringToken(Text$, "=, ", Result())
  Case 2
    Text$ = "Key1,Test1;Key2,Test2;  , ;   ,    ;Key3	,Test3 ;" 
    StringToken(Text$, ",;  ", Result())
  Case 3
    Text$ = "This is	a test	   	 	 	 	string	 	to	 	 	 	see	   	 	 	 	 	if	split is	working."
    StringToken(Text$, #CRLF$ + #TAB$ + #FF$ + #VT$ + " ", Result())
EndSelect    

ForEach Result()
  Debug "'" + Result() + "'"
Next  
Translated with http://www.DeepL.com/Translator

Download of PureBasic - Modules
Download of PureBasic - Programs

[Windows 11 x64] [PB V5.7x]
AZJIO
Addict
Addict
Posts: 1364
Joined: Sun May 14, 2017 1:48 am

Re: StringToken() as an alternative to StringField()

Post by AZJIO »

Thorsten1867 wrote: Sat Jun 11, 2022 6:17 pm You could also do all the tokens at once.
https://www.purebasic.fr/english/viewto ... 23#p585423
I need to check the speed in the evening
User avatar
Demivec
Addict
Addict
Posts: 4091
Joined: Mon Jul 25, 2005 3:51 pm
Location: Utah, USA

Re: StringToken() as an alternative to StringField()

Post by Demivec »

Thanks for sharing this code.
Axolotl wrote: Sat Jun 11, 2022 3:22 pmi deliberately decided to use the string-pointer, the resulting limitations or weaknesses can be seen in the example calls.
Procedure.s StringToken(, Separator$) ;
Just a note on the string-pointer parameter. You defined the parameter as the structured pointer *String.STRING but you never dereference that pointer and in fact the pointer's value is assigned incorrectly for it to be properly dereferenced .

You can easily see what the misassignment does by dereference the pointer with 'Debug *String\s'. It will display garbage at best.

The structure STRING is defined as:

Code: Select all

Structure STRING
  S.s ; this is a pointer to the actual string's content
EndStructure
When you assign an address to a *String.STRING pointer it should be the address of a pointer holding the address of a string's contents, not simply the address of string's contents.

Since you never dereference the pointer you can simply leave off the structure type and make it *String.
User avatar
NicTheQuick
Addict
Addict
Posts: 1227
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: StringToken() as an alternative to StringField()

Post by NicTheQuick »

If you replace

Code: Select all

Static *s_string.CHARACTER, *s_currCh.CHARACTER 
with

Code: Select all

Threaded *StringToken_s_string.CHARACTER, *StringToken_s_currCh.CHARACTER
outside the procedure and change the variable names accordingly inside the procedure, it should be threadsafe automatically. Unfortunately there is not "Static Threaded" for procedure available, so this is the only way.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Axolotl
Enthusiast
Enthusiast
Posts: 449
Joined: Wed Dec 31, 2008 3:36 pm

Re: StringToken() as an alternative to StringField()

Post by Axolotl »

Thank you all for your feedback.

Hi Demivec and NicTheQuick,

thank you very much for the qualified answers.
I was/am still not really sure about pointers under PB. That's why I was hoping for this kind of feedback.

I had thought that this procedure creates minimal memory requirements, so for large "strings" it doesn't take the same amount of memory for lists or arrays. (There are endless very good examples of this).
I wonder if you ever get into this situation of not having enough memory. (I probably/certainly don't).

Due to the now global variables, i have added two new procedures, called:

Code: Select all

StringToken_First()
StringToken_Next()

Here is a modified version (I didn't change the code above so the comments still make sense):

Code: Select all

EnableExplicit
DebugLevel 9    ; activates all my Debug messages (inside includes) 


; // make procedure threadsafe (if program is multithreaded) 
CompilerIf #PB_Compiler_Thread = 1                             ; // Set to 1 if the executable is compiled in thread safe mode, set to 0 else.
  Debug ">> Thread Safe Mode = ON " + #LF$, 9  
  Threaded *StringToken_string.CHARACTER, *StringToken_currCh.CHARACTER  ; // A Threaded object can't be declared in a procedure, its scope is always global. 

CompilerElse ; #PB_Compiler_Thread 
  Debug ">> Thread Safe Mode = OFF " + #LF$, 9 
  Define *StringToken_string.CHARACTER, *StringToken_currCh.CHARACTER    ; // I call it semi global, because of Shared inside a procedure ;) 

CompilerEndIf ; #PB_Compiler_Thread 


; --- declare procedures 

Declare.s StringToken_Next(Separator$)  ; STRING -- 


; Procedure.s StringToken(*String.STRING, Separator$)  ; STRING -- // parameter as the structured pointer *String.STRING is wrong .. 
Procedure.s StringToken(*String, Separator$)  ; STRING --          // .. better use this. 
	;Static *s_string.CHARACTER, *s_currCh.CHARACTER  ; not threadsafe 
  CompilerIf #PB_Compiler_Thread = 0    ; threadsafe not needed .. 
    Shared *StringToken_string, *StringToken_currCh  ; .. use semiglobal variables 
  CompilerEndIf 
	Protected result$, *currCh.CHARACTER, *sepCh.CHARACTER, *start ;.CHARACTER 

  If *String And *String <> *StringToken_string    ; new string to tokenize .. 
		*StringToken_string = *String                ; .. memorize the string pointer 
		*StringToken_currCh = *StringToken_string    ; .. point to current character 
	EndIf 

  If *StringToken_currCh = 0  ; end of string reached by last call or not valid 
    ProcedureReturn "" 
  EndIf 

	*currCh = *StringToken_currCh    ; .. work with a copy of threaded or global variable (easier to read!) 
  *start  = *currCh    ; memorize the pointer to the current character 

  While *currCh\c  
		*sepCh = @Separator$    ; point to separator string (start new) 
		While *sepCh\c 
			If *currCh\c = *sepCh\c    ; found a separator character 
				If *start <> *currCh     ; not at the start position 
					result$ = PeekS(*start, (*currCh - *start) >> 1) 
					*currCh + SizeOf(CHARACTER) 
					*StringToken_currCh = *currCh    ; memorize in threaded or global variable 
					ProcedureReturn result$ 
				EndIf 
				*start = *currCh + SizeOf(CHARACTER)    ; new string start 
				Break  
			EndIf 
			*sepCh + SizeOf(CHARACTER) 
    Wend 
		*currCh + SizeOf(CHARACTER) 
  Wend  
	result$ = PeekS(*start, (*currCh - *start) >> 1) 
	*StringToken_currCh = 0    ; reset current character (directly the threaded or global variable) 
	ProcedureReturn result$ 
EndProcedure ;() 

Procedure.s StringToken_First(*String, Separator$)  ; STRING -- 
  CompilerIf #PB_Compiler_Thread = 0    ; threadsafe not needed .. 
    Shared *StringToken_string, *StringToken_currCh  ; .. use semiglobal variables 
  CompilerEndIf 

	*StringToken_string = *String                ; .. memorize the string pointer 
	*StringToken_currCh = *StringToken_string    ; .. point to current character 
	ProcedureReturn StringToken_Next(Separator$) 
EndProcedure ;() 

Procedure.s StringToken_Next(Separator$)  ; STRING -- 
  CompilerIf #PB_Compiler_Thread = 0    ; threadsafe not needed .. 
    Shared *StringToken_string, *StringToken_currCh  ; .. use semiglobal variables 
  CompilerEndIf 
	Protected result$, *currCh.CHARACTER, *sepCh.CHARACTER, *start 

  If *StringToken_currCh = 0  ; end of string reached by last call or not valid 
    ProcedureReturn "" 
  EndIf 

	*currCh = *StringToken_currCh    ; .. work with a copy of threaded or global variable (easier to read!) 
  *start  = *currCh    ; memorize the pointer to the current character 

  While *currCh\c  
		*sepCh = @Separator$    ; point to separator string (start new) 
		While *sepCh\c 
			If *currCh\c = *sepCh\c    ; found a separator character 
				If *start <> *currCh     ; not at the start position 
					result$ = PeekS(*start, (*currCh - *start) >> 1) 
					*currCh + SizeOf(CHARACTER) 
					*StringToken_currCh = *currCh    ; memorize in threaded or global variable 
					ProcedureReturn result$ 
				EndIf 
				*start = *currCh + SizeOf(CHARACTER)    ; new string start 
				Break  
			EndIf 
			*sepCh + SizeOf(CHARACTER) 
    Wend 
		*currCh + SizeOf(CHARACTER) 
  Wend  
	result$ = PeekS(*start, (*currCh - *start) >> 1) 
	*StringToken_currCh = 0    ; reset current character (directly the threaded or global variable) 

	ProcedureReturn result$ 
EndProcedure ;() 

CompilerIf #PB_Compiler_IsMainFile 
;/---------------------------------------------
;| Test Call Collection 
;\---------------------------------------------
  Global DemoText$, nn, ret, ret$  

  DemoText$ = "Command = DIR " + #DOUBLEQUOTE$ + "*.*" + #DOUBLEQUOTE$ + ", /s" + ", /autorun " 

	ret$ = StringToken_First(@DemoText$, "=, ") 
	While ret$ 
	  Debug " => '" + ret$ + "'" 
		ret$ = StringToken_Next("=, ") 
	Wend   

  ; New String 
  Debug #LF$ + "====================" 
  ret$ = StringToken(@"TestA TestB", " ")  :Debug " => '" + ret$ + "'" 
  ret$ = StringToken(@"TestA TestB", " ")  :Debug " => '" + ret$ + "'"
  ret$ = StringToken(@"TestA TestB", " ")  :Debug " => '" + ret$ + "'"

  
  ; empty string works 
  Debug #LF$ + "====================" 
  ret$ = StringToken(@"", "")              :Debug " => '" + ret$ + "'  (empty string and separator)"
  ret$ = StringToken(@"", " ")             :Debug " => '" + ret$ + "'  (empty string)"
  ret$ = StringToken(@"Test with ", "")    :Debug " => '" + ret$ + "'  (empty separator)"

  ; new string 
  Debug #LF$ + "=== 1. =================" 
  DemoText$ = "Key1,Test1;Key2,Test2;  , ;		,		;Key3	,Test3	;" 
  For nn = 1 To 10 
  	ret$ = StringToken(@DemoText$, ",;	 ")   :Debug " => '" + ret$ + "'" 
  Next nn 

  Debug #LF$ + "=== 2. =================" 
	ret$ = StringToken_First(@DemoText$, ",;	 ") 
	While ret$ 
	  Debug " => '" + ret$ + "'" 
		ret$ = StringToken_Next(",;	 ") 
	Wend   

  ; same string .. needs a reset 
  StringToken(@"", "") 
  ;*StringToken_string = 0   ; variable is global now :( 

  ; same variable has the same pointer !!! 
  DemoText$ = "This is	a test	   	 	 	 	string	 	to	 	 	 	see	   	 	 	 	 	if	split is	working."
  Debug #LF$ + "====================" 
  For nn = 1 To 15
  	ret$ = StringToken(@DemoText$, #CRLF$ + #TAB$ + #FF$ + #VT$ + " ")    :Debug "["+nn+"] = '" + ret$ + "'" 
  Next nn 
CompilerEndIf 
 
Mostly running PureBasic <latest stable version and current alpha/beta> (x64) on Windows 11 Home
User avatar
NicTheQuick
Addict
Addict
Posts: 1227
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: StringToken() as an alternative to StringField()

Post by NicTheQuick »

"Threaded" is automatically "Global". Also please don't use #PB_Compiler_Thread to distinguish between "Threaded" or "Define/Global". Just use "Threaded" everytime. Because you do not have to use the ThreadSafe option to be able to use threads in generell.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
User avatar
NicTheQuick
Addict
Addict
Posts: 1227
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: StringToken() as an alternative to StringField()

Post by NicTheQuick »

Btw here is a nice trick to get rid of the @ when calling your procedure:

Code: Select all

Prototype StringTokenizer(string.p-Unicode, separator.s)

Procedure _StringToken(*string.Character, separator.s)
	Debug "Inside _StringToken()"
	Debug "    String pointer: " + *string
	Debug "    Separator: " + separator
EndProcedure


Global StringToken.StringTokenizer = @_StringToken()


Define myString.s = "hallo"

Debug "Main"
Debug "    String pointer: " + @myString

; No @ needed but the procedure will get the correct address without copying the string
StringToken(myString, "a")
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
Axolotl
Enthusiast
Enthusiast
Posts: 449
Joined: Wed Dec 31, 2008 3:36 pm

Re: StringToken() as an alternative to StringField()

Post by Axolotl »

Thanks again.
I already know that programs work with threads even without thread safe mode... but then Threaded alone should not be helpful, right?
According to help a program slows down by using Threaded ?

Nice trick anyway. The help writes at prototypes 'For advanced programmers.' So probably nothing for me.

And again I have learned something.
Mostly running PureBasic <latest stable version and current alpha/beta> (x64) on Windows 11 Home
User avatar
Thorsten1867
Addict
Addict
Posts: 1366
Joined: Wed Aug 24, 2005 4:02 pm
Location: Germany

Re: StringToken() as an alternative to StringField()

Post by Thorsten1867 »

All in one. ;-)

Code: Select all

;/ ==========================
;/ =    StringModule.pbi    =
;/ ==========================
;/
;/ [ PB V5.7x - V6.0 / 64Bit / all OS ]
;/
;/ © 2022 Thorsten1867 (06/2022)
;/

DeclareModule String
  
  Declare   Split(String.s, Array Result.s(1), Separator.s=" ")
  Declare.s Join(Array Result.s(1), Separator.s=" ")
  Declare   MulitSplit(String.s, Array Result.s(1), Separator.s=" ")
  Declare   Token(String.s, Array Result.s(1), Array Separators.s(1))
  Declare   Extract(String.s, Array Result.s(1), StartToken.s, EndToken.s)
  
EndDeclareModule

Module String
  
  Procedure   Split(String.s, Array Result.s(1), Separator.s=" ")
    ; by wilbert
    Define S.String, *S.Integer = @S
    Define.i aSize, i, Pos, sLen
    
    aSize = CountString(String, Separator)
    sLen  = Len(Separator)
    
    ReDim Result(aSize)
    
    *S\i = @String
    
    While i < aSize
      Pos = FindString(S\s, Separator)
      Result(i) = PeekS(*S\i, Pos - 1)
      *S\i + (Pos + sLen - 1) << #PB_Compiler_Unicode
      i + 1
    Wend
    
    Result(i) = S\s
    *S\i = 0
    
  EndProcedure

  Procedure.s Join(Array Result.s(1), Separator.s=" ")
    ; by wilbert
    Define.i aSize, i, sLen, tLen, *Buffer
    
    aSize = ArraySize(Result())
    sLen = Len(Separator)
    
    For i = 0 To aSize
      tLen + Len(Result(i)) + sLen
    Next
    
    tLen - sLen
    
    Dim Buffer.c(tLen)
    
    *Buffer = @Buffer()
    CopyMemoryString(Result(0), @*Buffer)
    For i = 1 To aSize
      CopyMemoryString(Separator)
      CopyMemoryString(Result(i))
    Next
    
    ProcedureReturn PeekS(@Buffer())
  EndProcedure
  
  Procedure   MulitSplit(String.s, Array Result.s(1), Separators.s=" ")
    Define.i c, sLen, aSize, Pos, EndPos, StartPos = 1
    
    NewList Token$()
    
    sLen = Len(String)
    
    While StartPos < sLen
      
      EndPos = sLen
      
      For c=1 To Len(Separators)
        
        If Mid(String, StartPos, 1) = Mid(Separators, c, 1)
          StartPos + 1
          Endpos = StartPos
          Break
        Else  
          Pos = FindString(String, Mid(Separators, c, 1), StartPos)
          If Pos And Pos < EndPos : EndPos = Pos : EndIf 
        EndIf  
  
      Next
  
      If EndPos <> StartPos
        If AddElement(Token$()) : Token$() = Mid(String, StartPos, EndPos - Startpos) : EndIf 
      EndIf
      
      StartPos = EndPos
    Wend
    
    aSize = ListSize(Token$()) - 1
    
    If aSize >= 0
    
      ReDim Result(aSize)
      
      c = 0
      
      ForEach Token$()
        Result(c) = Token$()
        c + 1
      Next 
      
    EndIf
    
  EndProcedure  
  
  Procedure   Token(String.s, Array Result.s(1), Array Separator.s(1))
    Define.i sLen, spSize, spLen, aSize
    Define.i i, Pos, EndPos, StartPos = 1

    NewList Token$()
    
    sLen   = Len(String)
    spSize = ArraySize(Separator())
    
    While StartPos < sLen
      
      spLen  = 0
      EndPos = sLen
      
      For i=0 To spSize
        
        If Mid(String, StartPos, Len(Separator(i))) = Separator(i)
          StartPos + Len(Separator(i))
          Endpos = StartPos
          Break
        Else  
          Pos = FindString(String, Separator(i), StartPos)
          If Pos And Pos < EndPos
            EndPos = Pos
            spLen  = Len(Separator(i))
          EndIf   
        EndIf
        
      Next

      If EndPos <> StartPos
        If AddElement(Token$()) : Token$() = Mid(String, StartPos, EndPos - Startpos) : EndIf 
      EndIf

      StartPos = EndPos + SpLen
    Wend
    
    aSize = ListSize(Token$()) - 1
    If aSize >= 0
    
      ReDim Result(aSize)
      
      c = 0
      
      ForEach Token$()
        Result(c) = Token$()
        c + 1
      Next 
      
    EndIf
    
  EndProcedure  
  
  Procedure   Extract(String.s, Array Result.s(1), StartToken.s, EndToken.s)
    Define.i sLen, aSize, Pos, EndPos, StartPos, Idx
    
    sLen  = Len(String)
    aSize = CountString(String, StartToken) - 1
    
    ReDim Result(aSize)
    
    Pos = 1
    StartPos = Pos
    
    While Pos
      
      Pos = FindString(String, StartToken, StartPos)
      If Pos 
        
        StartPos = Pos + Len(StartToken)
        
        EndPos = FindString(String, EndToken, StartPos)
        If StartPos < EndPos
          Result(Idx) = Mid(String, StartPos, EndPos - StartPos)
          Idx + 1
        EndIf  
        
        If EndPos : StartPos = Endpos + Len(EndToken) : EndIf
        
      EndIf  

    Wend  
    
  EndProcedure  
  
EndModule 

;- ========  Module - Example ========

CompilerIf #PB_Compiler_IsMainFile

#Example = 4

; 1: Split strings
; 2: Split with multiple separators
; 3: Token
; 4: Extrakt from token

  Select #Example
    Case 1 ;{ Split & join strings
      
      Dim Result.s(0)
      
      Test$ = "This is a test string to see if split and join are working."
      
      String::Split(Test$, Result())
      
      For i = 0 To ArraySize(Result())
        Debug Str(i) + ": " + Result(i)
      Next
      
      Debug "---------------------"
      
      Result$ = String::Join(Result(), "_")
      
      Debug "=> " + Result$
      ;}
    Case 2 ;{ Split with multiple separators
      
      Dim Result.s(0)
      
      Test$ = "Command = DIR " + #DOUBLEQUOTE$ + "*.*" + #DOUBLEQUOTE$ + ", /s" + " " 
      
      String::MulitSplit(Test$, Result(), "=, ")
      
      For i = 0 To ArraySize(Result())
        Debug Str(i) + ": " + Result(i)
      Next
      ;}
    Case 3 ;{ Token
      
      Dim Result.s(0)
      Dim Separator.s(1)
      
      Separator(0) = "<b>"
      Separator(1) = "</b>"
      
      Test$ = "This is a <b>text</b> with <b>bold</b> words."
      
      String::Token(Test$, Result(), Separator())
      
      For i = 0 To ArraySize(Result())
        Debug Str(i) + ": " + Result(i)
      Next
      ;}
    Case 4 ;{ Extrakt from token
      
      Dim Result.s(0)
      
      Test$ = "This is a <b>text</b> with <b>bold</b> words."
      
      String::Extract(Test$, Result(), "<b>", "</b>")
      
      For i = 0 To ArraySize(Result())
        Debug Str(i) + ": " + Result(i)
      Next
      ;}
  EndSelect    

CompilerEndIf
Last edited by Thorsten1867 on Tue Jun 14, 2022 10:10 am, edited 1 time in total.
Translated with http://www.DeepL.com/Translator

Download of PureBasic - Modules
Download of PureBasic - Programs

[Windows 11 x64] [PB V5.7x]
User avatar
NicTheQuick
Addict
Addict
Posts: 1227
Joined: Sun Jun 22, 2003 7:43 pm
Location: Germany, Saarbrücken
Contact:

Re: StringToken() as an alternative to StringField()

Post by NicTheQuick »

Axolotl wrote: Mon Jun 13, 2022 2:40 pm I already know that programs work with threads even without thread safe mode... but then Threaded alone should not be helpful, right?
According to help a program slows down by using Threaded ?
I don't see any hint in the help file which says that "Threaded" slows down the program. It fact it makes no sense to me. It just declares a dedicated variable for each thread. Keep in mind that each process has at least one thread, the main thread. So if you do not use CreateThread() it just acts like the "Global" keyword.
ThreadSafe has nothing to do with "Threaded". ThreadSafe is only needed if you use strings in multiple threads. So... yes, in your case you are working with strings and it would make sense. But no, "Threaded" can also be important in programs which do not do anything with strings at all.
The english grammar is freeware, you can use it freely - But it's not Open Source, i.e. you can not change it or publish it in altered way.
AZJIO
Addict
Addict
Posts: 1364
Joined: Sun May 14, 2017 1:48 am

Re: StringToken() as an alternative to StringField()

Post by AZJIO »

Thorsten1867

Code: Select all

While StartPos < Len(String)
It is not economical to calculate the length of a string multiple times in a loop.
User avatar
Thorsten1867
Addict
Addict
Posts: 1366
Joined: Wed Aug 24, 2005 4:02 pm
Location: Germany

Re: StringToken() as an alternative to StringField()

Post by Thorsten1867 »

@AZJIO

Not really. Changed it. ;-)
Translated with http://www.DeepL.com/Translator

Download of PureBasic - Modules
Download of PureBasic - Programs

[Windows 11 x64] [PB V5.7x]
Axolotl
Enthusiast
Enthusiast
Posts: 449
Joined: Wed Dec 31, 2008 3:36 pm

Re: StringToken() as an alternative to StringField()

Post by Axolotl »

Hi NicTheQuick,
I don't see any hint in the help file which says that "Threaded" slows down the program.

yes, you're right. i remembered this wrong here. It was the thread safe mode, not the keyword Threaded.
"Without this mode, certain functions (and also the string access) are faster "

BTW: With the fact that "... its scope is always global. " I can live with.

Thank you for the interesting conversation.
Mostly running PureBasic <latest stable version and current alpha/beta> (x64) on Windows 11 Home
User avatar
mk-soft
Always Here
Always Here
Posts: 5406
Joined: Fri May 12, 2006 6:51 pm
Location: Germany

Re: StringToken() as an alternative to StringField()

Post by mk-soft »

If you use strings as variables in the thread, you must activate ThreadSafe.
If you use a pointer to a string as a character array, then you do not need ThreadSafe.
My Projects ThreadToGUI / OOP-BaseClass / EventDesigner V3
PB v3.30 / v5.75 - OS Mac Mini OSX 10.xx - VM Window Pro / Linux Ubuntu
Downloads on my Webspace / OneDrive
Post Reply