Fast string

Got an idea for enhancing PureBasic? New command(s) you'd like to see?
User_Russian
Addict
Addict
Posts: 1183
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Fast string

Post by User_Russian »

STARGÅTE wrote:Fast string is implimented es CopyMemoryString()
This function does not allocate memory for the string.
Then it is not much different from PokeS().
In addition, its use does not accelerate the string functions, such as for example FindString(). In which takes a long time to measure the length of the string.
User_Russian
Addict
Addict
Posts: 1183
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Fast string

Post by User_Russian »

Perhaps not everyone understands that it is by measuring the length of the string, reduces the speed of execution.

Suppose that it is work with a string without measuring its length.

Code: Select all

DisableDebugger

Time = ElapsedMilliseconds()

For i=1 To 100000
Next i

MessageRequester("", StrF((ElapsedMilliseconds()-Time)/1000, 3))
This length measurement (assume that every time a string is added to 10 characters).

Code: Select all

DisableDebugger

Time = ElapsedMilliseconds()

Len = 10

For i=1 To 100000
  
  For y=1 To Len : Next y ; Analog of function Len().
  
  Len + 10
Next i

MessageRequester("", StrF((ElapsedMilliseconds()-Time)/1000, 3))
Compare speed of execution.
Thorium
Addict
Addict
Posts: 1268
Joined: Sat Aug 15, 2009 6:59 pm

Re: Fast string

Post by Thorium »

I think it would be very nice to have fast strings.
Just as User_Russian wrote, add a length descriptor to every string. Makes strings use more memory, but today memory is no issue, speed is more importent in most cases.
User_Russian
Addict
Addict
Posts: 1183
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Fast string

Post by User_Russian »

Thorium wrote:Makes strings use more memory,
String is longer by 4 bytes in x86, and longer by 8 bytes in x64.
User avatar
langinagel
Enthusiast
Enthusiast
Posts: 129
Joined: Fri Jan 28, 2005 11:53 pm
Location: Germany
Contact:

Re: Fast string

Post by langinagel »

Gentlemen,

the code below yield the following result on my PC (asus laptop with lubuntu):
10 0.000
100 0.000
1000 0.014
10000 1.125
100000 125.365

This means that for strings below 1000 characters the speed of processing sufficient high from my point of view. I think the majority of processed strings are inputs from forms that have less than 1000 characters.

My suggestion:
If we need faster strings a second class of "longstrings" should be defined with the length attached. This should be the real feature request.
But the usual string should be kept. Why?
Because in cases of memory faults, maybe in the string length bytes, the processing will fail and/or we might have a security problem.

Comments?

Code: Select all

OpenConsole()
Str.s
#Text = "1234567890"
mstart =1
y =  Int(Pow(10,mstart))
Time = ElapsedMilliseconds()

For i=1 To 100010
  Str + #Text
  If i = y
    PrintN(Str(i)+"           "+StrF((ElapsedMilliseconds()-Time)/1000, 3))
    mstart = mstart + 1
    y = Int(Pow(10,mstart))
    Time = ElapsedMilliseconds()
  EndIf
  
Next i
Print( "ende")
Input()

Greetings
LN
https://www.doerpsoft.org

Boost. Work. Efficiency.
User_Russian
Addict
Addict
Posts: 1183
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Fast string

Post by User_Russian »

This problem refers to the string functions.
Compare the speed of these two codes.

Code: Select all

OpenConsole()
Str.s = Space(100010)
mstart =1
y =  Int(Pow(10,mstart))
Time = ElapsedMilliseconds()
Pos=0
For i=1 To 100010
  
  s.s=Mid(Str, i, 1)
  If i = y
    PrintN(Str(i)+"           "+StrF((ElapsedMilliseconds()-Time)/1000, 3))
    mstart = mstart + 1
    y = Int(Pow(10,mstart))
    Time = ElapsedMilliseconds()
  EndIf
  
Next i
Print( "ende")
Input()

Code: Select all

OpenConsole()
Str.s = Space(100010)
mstart =1
y =  Int(Pow(10,mstart))
Time = ElapsedMilliseconds()
Pos=0
For i=1 To 100010
  
  s.s=PeekS(@Str+i, 1)
  If i = y
    PrintN(Str(i)+"           "+StrF((ElapsedMilliseconds()-Time)/1000, 3))
    mstart = mstart + 1
    y = Int(Pow(10,mstart))
    Time = ElapsedMilliseconds()
  EndIf
  
Next i
Print( "ende")
Input()
User avatar
langinagel
Enthusiast
Enthusiast
Posts: 129
Joined: Fri Jan 28, 2005 11:53 pm
Location: Germany
Contact:

Re: Fast string

Post by langinagel »

UR,

the comparison yields on my PC (core3 with Ubuntu12.04 -x68) to:
#repeats code above code below
10 0.000 0.000
100 0.000 0.000
1000 0.000 0.000
10000 0.044 0.000
100000 3.431 0.006

Again: for strings of the length of 50KB and below I see no need for change.

For a dedicated long-string, I agree to the new feature.
But: the basic routine should contain a check: if the string-end marking could not be found at the attached byte offset then the string should have a dedicated error marking(e.g. negative string length).

Comments?

Greetings
LN
https://www.doerpsoft.org

Boost. Work. Efficiency.
User_Russian
Addict
Addict
Posts: 1183
Joined: Wed Nov 12, 2008 5:01 pm
Location: Russia

Re: Fast string

Post by User_Russian »

Can add new functions

Code: Select all

StringVerificationLength(String.s)
which will measure the length of the string and write it the correct value.
Opcode
Enthusiast
Enthusiast
Posts: 124
Joined: Thu Jul 18, 2013 4:58 am

Re: Fast string

Post by Opcode »

I agree, I am working on a project right now and the slowness of PB's string handling is effecting it's responsiveness. Something that should be improved for the future.
Image
User avatar
Andre
PureBasic Team
PureBasic Team
Posts: 1877
Joined: Fri Apr 25, 2003 6:14 pm
Location: Germany (Saxony, Deutscheinsiedel)
Contact:

Re: Fast string

Post by Andre »

As I'm also working a lot with .csv files and following string operations I would be very happy too, if the native implementation could be optimized! :mrgreen:
Bye,
...André
(PureBasicTeam::Docs & Support - PureArea.net | Order:: PureBasic | PureVisionXP)
User avatar
gurj
Enthusiast
Enthusiast
Posts: 640
Joined: Thu Jan 22, 2009 3:48 am
Location: china
Contact:

Re: Fast string

Post by gurj »

'strings in PB, very slow'
'Just as User_Russian wrote, add a length descriptor to every string'

PB must let .s=.BSTR inside pb internal ,for each string:

.s add size.l:

Structure BSTR
size.l ;"ABC" = 3 (Asc) or 6 (unicode)
s.s ;Never directly WRITE to this; always use bFunctions() to ensure \Size is updated.
EndStructure

then optimize more :
len(string$)=string$\size
...

see :
BSTR* fast dynamic string datatype
http://www.purebasic.fr/english/viewtop ... 12&t=68121
my pb for chinese:
http://ataorj.ys168.com
User avatar
marcoagpinto
Enthusiast
Enthusiast
Posts: 707
Joined: Sun Mar 10, 2013 3:01 pm
Location: Portugal
Contact:

Re: Fast string

Post by marcoagpinto »

+1

In my Hunspell tool, "Proofing Tool GUI", I have tons of string operations and I notice the slowness.
User avatar
Thunder93
Addict
Addict
Posts: 1773
Joined: Tue Mar 21, 2006 12:31 am
Location: Canada

Re: Fast string

Post by Thunder93 »

Keya really have done up an impressive demonstration. Really makes apparent the need for faster string.

I hope User_Russian gets his wish fulfilled.

+1 for me
ʽʽSuccess is almost totally dependent upon drive and persistence. The extra energy required to make another effort or try another approach is the secret of winning.ʾʾ --Dennis Waitley
ShadowStorm
User
User
Posts: 99
Joined: Tue Feb 14, 2017 12:07 pm

Re: Fast string

Post by ShadowStorm »

I do not know if this is what you want:

Code: Select all

Structure MyStringBuilder
 
  MyStringBuilderAllocateMemory.B
  *MyStringBuilderMemoryID
  MyStringBuilderSize.I
  MyStringBuilderRemaining.I
  MyStringBuilderPosition.I
 
EndStructure

Global MyStringBuilder.MyStringBuilder

Procedure.B IsStringBuilder()
 
  If MyStringBuilder\MyStringBuilderAllocateMemory = #True And MyStringBuilder\MyStringBuilderMemoryID <> 0
    ProcedureReturn  1
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.I InitialiseStringBuilder(Taille.I = 1000)
 
  If Taille.I <= 0
    Taille.I = 1
  EndIf
 
  If Not IsStringBuilder()
   
    MyStringBuilder\MyStringBuilderMemoryID = AllocateMemory(Taille.I)
   
    If MyStringBuilder\MyStringBuilderMemoryID <> 0
     
      MyStringBuilder\MyStringBuilderAllocateMemory = #True
      MyStringBuilder\MyStringBuilderRemaining = MemorySize(MyStringBuilder\MyStringBuilderMemoryID)
      MyStringBuilder\MyStringBuilderSize = MemorySize(MyStringBuilder\MyStringBuilderMemoryID)
      MyStringBuilder\MyStringBuilderPosition = 0
     
      ProcedureReturn MyStringBuilder\MyStringBuilderRemaining
     
    Else
      ProcedureReturn 0
     
    EndIf
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.B StringBuilderClearString(Taille.I = 1000)
 
  If Taille.I <= 0
    Taille.I = 1
  EndIf
 
  If IsStringBuilder()
   
    FreeMemory(MyStringBuilder\MyStringBuilderMemoryID)
   
    MyStringBuilder\MyStringBuilderAllocateMemory = #False
    MyStringBuilder\MyStringBuilderMemoryID = 0
    MyStringBuilder\MyStringBuilderSize = 0
    MyStringBuilder\MyStringBuilderRemaining = 0
    MyStringBuilder\MyStringBuilderPosition = 0
   
    If InitialiseStringBuilder(Taille.I)
      ProcedureReturn 1
     
    Else
      ProcedureReturn 0
     
    EndIf
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.I StringBuilderAddString(String.s)
 
  If IsStringBuilder()
   
    If Len(String.s) > MyStringBuilder\MyStringBuilderRemaining.I
     
      MyStringBuilder\MyStringBuilderMemoryID = ReAllocateMemory(MyStringBuilder\MyStringBuilderMemoryID, MemorySize(MyStringBuilder\MyStringBuilderMemoryID) + (Len(String.s) * 10))
     
      If MyStringBuilder\MyStringBuilderMemoryID <> 0
       
        MyStringBuilder\MyStringBuilderRemaining = MemorySize(MyStringBuilder\MyStringBuilderMemoryID) - MyStringBuilder\MyStringBuilderPosition
        MyStringBuilder\MyStringBuilderSize = MemorySize(MyStringBuilder\MyStringBuilderMemoryID)
       
        OctetsWrite = PokeS(MyStringBuilder\MyStringBuilderMemoryID + MyStringBuilder\MyStringBuilderPosition, String.s, -1, #PB_Ascii | #PB_String_NoZero)
       
        MyStringBuilder\MyStringBuilderRemaining - OctetsWrite
        MyStringBuilder\MyStringBuilderPosition + OctetsWrite
       
        ProcedureReturn OctetsWrite
       
      Else
        ProcedureReturn 0
       
      EndIf
     
    Else
     
      OctetsWrite = PokeS(MyStringBuilder\MyStringBuilderMemoryID + MyStringBuilder\MyStringBuilderPosition, String.s, -1, #PB_Ascii | #PB_String_NoZero)
     
      MyStringBuilder\MyStringBuilderRemaining - OctetsWrite
      MyStringBuilder\MyStringBuilderPosition + OctetsWrite
     
      ProcedureReturn OctetsWrite
     
    EndIf
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.S StringBuilderGetString()
 
  If IsStringBuilder()
    ProcedureReturn PeekS(MyStringBuilder\MyStringBuilderMemoryID)
   
  Else
    ProcedureReturn ""
   
  EndIf
 
EndProcedure

Procedure.I StringBuilderGetMemoryID()
 
  If IsStringBuilder()
    ProcedureReturn MyStringBuilder\MyStringBuilderMemoryID
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.I StringBuilderGetMemorySize()
 
  If IsStringBuilder()
    ProcedureReturn MyStringBuilder\MyStringBuilderSize
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.I StringBuilderGetMemoryRemaining()
 
  If IsStringBuilder()
    ProcedureReturn MyStringBuilder\MyStringBuilderRemaining
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Procedure.I StringBuilderGetMemoryPosition()
 
  If IsStringBuilder()
    ProcedureReturn MyStringBuilder\MyStringBuilderPosition
   
  Else
    ProcedureReturn 0
   
  EndIf
 
EndProcedure

Abc$ = "Abcdefghijklmnopqrstuvwxyz"

If InitialiseStringBuilder() And IsStringBuilder()
 
  MessageRequester("StringBuilder Initialisation...", "StringBuilder MemoryID = " + Str(StringBuilderGetMemoryID()) + Chr(13) + Chr(10) +
                                                      "StringBuilder Memory Size = " + Str(StringBuilderGetMemorySize()) + Chr(13) + Chr(10) +
                                                      "StringBuilder Memory Remaining = " + Str(StringBuilderGetMemoryRemaining()) + Chr(13) + Chr(10) +
                                                      "StringBuilder Memory Position = " + Str(StringBuilderGetMemoryPosition()))
 
  Time1 = ElapsedMilliseconds()
 
  For I = 1 To 100000
   
    MemoryID = StringBuilderGetMemoryID()
    MemorySize = StringBuilderGetMemorySize()
    MemoryRemaining = StringBuilderGetMemoryRemaining()
    MemoryPosition = StringBuilderGetMemoryPosition()
    OctetsWrite = StringBuilderAddString(Abc$)
   
  Next
 
  Time2 = ElapsedMilliseconds()
 
  MessageRequester("StringBuilder Résultat...", "StringBuilder MemoryID = " + Str(StringBuilderGetMemoryID()) + Chr(13) + Chr(10) +
                                                "StringBuilder Memory Size = " + Str(StringBuilderGetMemorySize()) + Chr(13) + Chr(10) +
                                                "StringBuilder Memory Remaining = " + Str(StringBuilderGetMemoryRemaining()) + Chr(13) + Chr(10) +
                                                "StringBuilder Memory Position = " + Str(StringBuilderGetMemoryPosition()) + Chr(13) + Chr(10) +
                                                "StringBuilder Last Octets Write = " + Str(OctetsWrite) + Chr(13) + Chr(10) +
                                                "Time Elapsed = " + Str(Time2 - Time1) + " Ms.")
 
Else
  MessageRequester("StringBuilder Résultat...", "Erreur, Le StringBuilder n'a pas put être initialisé !", 16)
 
EndIf
I am French, I do not speak English.
My apologies for the mistakes.
User avatar
marcoagpinto
Enthusiast
Enthusiast
Posts: 707
Joined: Sun Mar 10, 2013 3:01 pm
Location: Portugal
Contact:

Re: Fast string

Post by marcoagpinto »

Fred!!!!

Please implement this:
The easy solution would be to add an option to the PureBasic compiler "x Use fast strings (more RAM needed)"
Post Reply