It's only one 1 byte longer than the size of the file. I was wondering though, if something else happens to be occupying that memory location (1 byte past the size of our file), then its 'byte' could be included in the process. It's a minor point but I just wanted to make sure I was following it correctlyinfratec wrote: ↑Mon Aug 15, 2022 10:38 pm You are right:If you have enough RAM ... no problem.Code: Select all
*FileEnd = *File + MemorySize(*File) - 1
I already read 1GB files in RAM without problems.
Or you have to do a chunk management which is not trivial if a rest of bytes needs to be copied to the begin.
If I remember, I already posted such an example.
ReadString() performance best practice
Re: ReadString() performance best practice
Re: ReadString() performance best practice
I assume this key isn't always in the same position?! For large files I would chunk it into 1MB pieces (or some other useful size) and analyze them. ReadData() would be your friend.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Re: ReadString() performance best practice
That's right, there are no positions in the file, only delimited sequences of variable-length data. I might see better performance using ReadData() into fixed blocks as you say, the only problem would be that if I'm searching for a string of bytes that happens to straggle two consecutive blocks, it's difficult to find it, but of course there are ways around that, such as saving the last 'n' bytes from the block before.
But at the moment, using ReadByte() to process small delimited sections at a time is giving me fairly good performance. It takes 15 seconds to find something in a 45Mb file. I don't know if that sounds reasonable.
Re: ReadString() performance best practice
Yeah, just add the subtract the length of the search string (be aware of ASCII and unicode differences) from the actual read position and read the next block.
PureBasic 6.04/XProfan X4a/Embarcadero RAD Studio 11/Perl 5.2/Python 3.10
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB
Windows 11/Ryzen 5800X/32GB RAM/Radeon 7770 OC/3TB SSD/11TB HDD
Synology DS1821+/36GB RAM/130TB
Synology DS920+/20GB RAM/54TB
Synology DS916+ii/8GB RAM/12TB