FileBuffersSize(#PB_Default, 0) does not work.

Just starting out? Need help? Post your questions and find answers here.
User avatar
Rescator
Addict
Addict
Posts: 1769
Joined: Sat Feb 19, 2005 5:05 pm
Location: Norway

FileBuffersSize(#PB_Default, 0) does not work.

Post by Rescator »

I found while messing around with some code that FileBuffersSize(#PB_Default, 0) does not work.

For a long time I could not understand why the filereading code was so slow (I used a custom buffering for a test) and I thought that instead of turning off the buffer for each file/handle again and again, why not turn it off by default.

But FileBuffersSize(#PB_Default, 0) does not work, it seems to revert to the default (4KB) which was very slow in this case (a file of several GB with random seeks and sequential reads).


Expected behavior is that FileBuffersSize(#PB_Default, 0) works the same as FileBuffersSize(#File1, 0) would or FileBuffersSize(filehandle, 0) would.
User avatar
Shield
Addict
Addict
Posts: 1021
Joined: Fri Jan 21, 2011 8:25 am
Location: 'stralia!
Contact:

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by Shield »

Rescator wrote:Expected behavior is that FileBuffersSize(#PB_Default, 0) works the same as FileBuffersSize(#File1, 0) would or FileBuffersSize(filehandle, 0) would.
It doesn't. As the help file states, #PB_Default only changes the buffer size for files that are opened AFTER the call to FileBufferSize(#PB_Default, 0). :)
Image
Blog: Why Does It Suck? (http://whydoesitsuck.com/)
"You can disagree with me as much as you want, but during this talk, by definition, anybody who disagrees is stupid and ugly."
- Linus Torvalds
User avatar
Rescator
Addict
Addict
Posts: 1769
Joined: Sat Feb 19, 2005 5:05 pm
Location: Norway

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by Rescator »

If #PB_Default is used as this parameter then the new buffer size will apply to all newly opened files with OpenFile(), CreateFile() or ReadFile().
Then that text must be fixed to read:
#1
If #PB_Default is used as this parameter then the new buffer size will only apply to existing files opened/created with OpenFile(), CreateFile() or ReadFile().
The way I read the current texts meaning it should be this:
#2
If #PB_Default is used as this parameter then the new buffer size will apply to all future files opened/created with OpenFile(), CreateFile() or ReadFile().
One of those two are correct, the other not.
You say #1 is the interpretation, and I say #2 is.

I guess Fred or Freak will clarify what the original intent is.

Personally I see no point in FileBuffersSize(#PB_Default, something) being used after you have opened all files, what about files you are about to open, do you need to use FileBuffersSize(#PB_Default, something) again?

If #PB_Default do not set an actual default then what is the point of a "default"?

With it like how you described it working this is what must be done, which is no different from FileBuffersSize(file, something)
OpenFile()
FileBuffersSize(#PB_Default, something)
CloseFile()
OpenFile()
FileBuffersSize(#PB_Default, something)
CloseFile()
OpenFile()
FileBuffersSize(#PB_Default, something)
CloseFile()
OpenFile()
FileBuffersSize(#PB_Default, something)
CloseFile()

While with a default like I describe it this can be done.
FileBuffersSize(#PB_Default, something)
OpenFile()
CloseFile()
OpenFile()
CloseFile()
OpenFile()
CloseFile()
OpenFile()
CloseFile()


Now if you say that ¤PB_Default sets it on the filehandle as a default then that is different, but that makes it useless if you open files with #PB_Any for example.

Right now I got code that looks like this:

Code: Select all

handle=ReadFile(#PB_Any, "blah")
FileBuffersSize(handle, 0)
...code...
CloseFile(handle)
handle=ReadFile(#PB_Any, "blah")
FileBuffersSize(handle, 0)
...code...
CloseFile(handle)
handle=ReadFile(#PB_Any, "blah")
FileBuffersSize(handle, 0)
...code...
CloseFile(handle)
handle=ReadFile(#PB_Any, "blah")
FileBuffersSize(handle, 0)
...code...
CloseFile(handle)
A lot of extra code is needed and allocation of memory for filebuffers that need to be freed again (I assume that using 0 causes them to be freed 0 do they still take up memory?)

With #PB_Default working as intended (as I interpret it):
Right now I got code that looks like this:

Code: Select all

FileBuffersSize(#PB_Default, 0)
handle=ReadFile(#PB_Any, "blah")
...code...
CloseFile(handle)
handle=ReadFile(#PB_Any, "blah")
...code...
CloseFile(handle)
handle=ReadFile(#PB_Any, "blah")
...code...
CloseFile(handle)
handle=ReadFile(#PB_Any, "blah")
...code...
CloseFile(handle)
A lot less code is needed and no need to allocate memory for filebuffers that need to be freed again.

Your interpretation is more like a #PB_All to set the buffersize for all existing filehandles, now that is a valid feature as well, but #PG_Default should do that as well to existing files as well as future files opened.
User avatar
TI-994A
Addict
Addict
Posts: 2490
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by TI-994A »

Rescator wrote:Expected behavior is that FileBuffersSize(#PB_Default, 0) works the same as FileBuffersSize(#File1, 0) would...
Rescator wrote:#1: If #PB_Default is used as this parameter then the new buffer size will only apply to existing files opened/created with OpenFile(), CreateFile() or ReadFile().
Shield wrote:...#PB_Default only changes the buffer size for files that are opened AFTER the call to FileBufferSize(#PB_Default, 0).
Rescator wrote:#2: If #PB_Default is used as this parameter then the new buffer size will apply to all future files opened/created with OpenFile(), CreateFile() or ReadFile().
Rescator wrote:You say #1 is the interpretation, and I say #2 is.
Actually, Shield's interpretation is more congruent with #2. Your's is #1. :wink:
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too!
User avatar
TI-994A
Addict
Addict
Posts: 2490
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by TI-994A »

My interpretation:

Code: Select all

OpenFile(1, "file1.txt")     ;this file will utilise PureBasic's default 4K file buffer

FileBuffersSize(#PB_Default, 10240)

WriteString(1, text$)        ;this file will still utilise only the 4K file buffer
OpenFile(2, "file2.txt")     ;this file will utilise the new default 10K file buffer

FileBuffersSize(#PB_Default, 0)

WriteString(1, text$)        ;this file will still utilise the 4K file buffer
CreateFile(3, "file3.txt")   ;this file will not utilise any file buffer
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too!
User avatar
Rescator
Addict
Addict
Posts: 1769
Joined: Sat Feb 19, 2005 5:05 pm
Location: Norway

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by Rescator »

TI-994A wrote:FileBuffersSize(#PB_Default, 0)

WriteString(1, text$) ;this file will still utilise the 4K file buffer
CreateFile(3, "file3.txt") ;this file will not utilise any file buffer[/code]
Wrong, that is not my code at all. Please re-read my first post.
My code is/what I said was...
FileBuffersSize(#PB_Default, 0)

ReadFile(#PB_Any, "somefile.txt")[/code]
And this does (did) not work.
Last edited by Rescator on Sat Dec 20, 2014 7:36 am, edited 1 time in total.
User avatar
Rescator
Addict
Addict
Posts: 1769
Joined: Sat Feb 19, 2005 5:05 pm
Location: Norway

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by Rescator »

TI-994A wrote:Actually, Shield's interpretation is more congruent with #2. Your's is #1. :wink:
*sigh*
Re-read my first post again.

Here is a full sourcecode http://www.purebasic.fr/english/viewtop ... 27&t=37518

If you look at line 69 and line 89 you will see that "FileBuffersSize(fh, 0)".

I previously tried to not use it, and instead use FileBuffersSize(#PB_Default, 0) on line 57 (after selecting the files but before the main loop.

But the this whole thread (and bug report) may be pointless due to this http://www.purebasic.fr/english/viewtop ... 97#p458197
Windows file caching causes unpredictable caching (first read slow and consequtive reads fast) and this may be a false bug report (I repeated tests on the same USB stick and commenting and uncommenting and FileBuffersSize(#PB_Default, 0) prior to the loop now gets the same performance as when using FileBuffersSize(fh, 0) right after each ReadFile().
I'd still like a confirmation from Fred or Freak that FileBuffersSize(#PB_Default, 0) also works/affect #PB_Any opened files (I'll assume they do, and apologies to Fred/Freak for wasting their time).



But back to what i said in this thread (TI-994A and Shield).
When I ran the tests earlier FileBuffersSize(#PB_Default, 0) gave me worse performance of #PB_Any opened files than if I did individual FileBuffersSize(fh, 0) after opening each file.
That is what my first post was about. Maybe my English wasn't good enough? I'm not a native English speaker so I may screw up the grammar now and again. If that is the case then I apologize for that, but if that is not the case then please re-read the first post again.

Never did I say anywhere that I used FileBuffersSize(#PB_Default, 0) after I opened the files. I assumed people understood that I meant that FileBuffersSize(#PB_Default, 0) was used BEFORE any files where opened. (as that is what I what I interpret the manual to say).
In my first post it is interpretation #2 that I state.

Why you two seems to think I'm talking about interpretation #1 I have no idea, it was Shield that alleges that I said that, not me, I never did say that. (do note that in a later comment I do show an example of what Shield alleged I said, are you getting confused by that? Do note that may not be the only confusion here, as I said it may be a false bug report as I was surprised windows caches almost 2GB of files without me knowing/realizing it. I guess the MB/s should have been a clue but earlier versions of the code did not show that stat)
Fred
Administrator
Administrator
Posts: 14423
Joined: Fri May 17, 2002 4:39 pm
Location: France
Contact:

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by Fred »

I just checked, and when using (#PB_Default, 0), you set the PB buffer size to zero for all following Read/Open/CreateFile(). But all ReadString(), ReadChar() like function will be really slow, as it will be read byte per byte...
User avatar
TI-994A
Addict
Addict
Posts: 2490
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by TI-994A »

Rescator wrote:Why you two seems to think I'm talking about interpretation #1 I have no idea...
Hi Rescator. Perhaps it's just a miscommunication; my sincerest apologies for the confusion. :)

Important thing is that Fred has confirmed that the buffer parameters of the FileBuffersSize(#PB_Default, x) function would apply only to "future" files and not already opened files.
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too!
User avatar
Rescator
Addict
Addict
Posts: 1769
Joined: Sat Feb 19, 2005 5:05 pm
Location: Norway

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by Rescator »

For those interested, this might be worth looking at http://www.purebasic.fr/english/viewtop ... 27&t=61273
The source there behaves as expected and shows how to do unbuffered reads of individual files (on Windows only, sorry, no idea on how to do that on Linux) so one won't end up like me and thinking #PB_Default didn't work :/
User avatar
TI-994A
Addict
Addict
Posts: 2490
Joined: Sat Feb 19, 2011 3:47 am
Location: Singapore

Re: FileBuffersSize(#PB_Default, 0) does not work.

Post by TI-994A »

For the benefit of those who might search and stumble upon this thread by title, the FileBuffersSize() function works as intended. This snippet illustrates its scope:

Code: Select all

OpenFile(1, "file1.txt")              ;this file will utilise PureBasic's default 4K file buffer

FileBuffersSize(#PB_Default, 10240)   ;sets the default file buffer to 10K

WriteString(1, text$)                 ;this file will still utilise only the 4K file buffer
OpenFile(2, "file2.txt")              ;this file will utilise the new default 10K file buffer

FileBuffersSize(#PB_Default, 0)       ;disables the default file buffer by setting it to zero

WriteString(1, text$)                 ;this file will still utilise the 4K file buffer
WriteString(2, text$)                 ;this file will still utilise the 10K file buffer
CreateFile(3, "file3.txt")            ;this file will not utilise any file buffer
Please correct me if I'm wrong. Thank you. :)
Texas Instruments TI-99/4A Home Computer: the first home computer with a 16bit processor, crammed into an 8bit architecture. Great hardware - Poor design - Wonderful BASIC engine. And it could talk too!
Post Reply