Fast string

Got an idea for enhancing PureBasic? New command(s) you'd like to see?
pdwyer
Addict
Addict
Posts: 2761
Joined: Tue May 08, 2007 1:27 pm
Location: Chiba, Japan

Re: Fast string

Post by pdwyer »

In the past I often worked around this with loops inside loops with a mod(loop,100) or something to build smaller strings first which was reasonably effective but recently, after a little thought I tried something else.

In many cases, these sorts of long string concatenation loops are for preparing data for disk (not always of course). Many times I've hit this is a loop to prepare lines of text or csv for a data dump, export or log dump. In fact I had this come up just the other day and was seeing significant slowdown as the string length went over 100's of kb and was just about to add the mod() trick and thought, "why not just stream straight to disk?" and then had that auto-thought (because disk is slower than memory!).

So I tried it, Openfile(), loop the strings being built to hold a single line of CSV, WriteString(), end loop, CloseFile().

All delays went away and it scales a lot larger.

I would guess that if your target was a network destination and streaming data the situation would be similar.

Perhaps if you are building huge strings you should ask what your ultimate goal of the built string is as there may be a way to stream it. If strings of data get really huge then streaming will be a better plan in most cases anyway.

I'm sure there will always be cases where fast string cat of large strings is needed and I'm sure people here are hitting them, but perhaps there are less than I first imagined.
Paul Dwyer

“In nature, it’s not the strongest nor the most intelligent who survives. It’s the most adaptable to change” - Charles Darwin
“If you can't explain it to a six-year old you really don't understand it yourself.” - Albert Einstein
User avatar
HeX0R
Enthusiast
Enthusiast
Posts: 655
Joined: Mon Sep 20, 2004 7:12 am
Location: Hell

Re: Fast string

Post by HeX0R »

No one thought about CopyMemoryString()?

Code: Select all

DisableDebugger


Str.s   = ""
Text.s  = "1234567890"
Time    = ElapsedMilliseconds()
*Buffer  = AllocateMemory(2000002)
*Pointer = *Buffer
CopyMemoryString(Text, @*Pointer)
For i=1 To 99999
  CopyMemoryString(Text)
Next i

MessageRequester("", StrF((ElapsedMilliseconds()-Time)/1000, 3))
EnableDebugger
Debug PeekS(*Buffer)
Link dead?
Change h3x0r.ath.cx into hex0rs.coderbu.de and all will be fine.
BarryG
Addict
Addict
Posts: 1522
Joined: Thu Apr 18, 2019 8:17 am

Re: Fast string

Post by BarryG »

HeX0R wrote:No one thought about CopyMemoryString()?
Nice, but all the tricks like this shouldn't be needed.
pdwyer wrote:if you are building huge strings you should ask what your ultimate goal of the built string is as there may be a way to stream it
Nah, PureBasic should just build them faster, as HeX0R just showed.
Post Reply