Curious which chunksize is fastest?,
and indirectly what FilebufferSize() might make the most sense?
Or want to compare Hard Drive speeds?
Or which OS or Operating Filesystem has best speeds?
Or test USB memory stick speeds or SSD speeds?
Then this tool might be of interest! (Hopefully)
Make sure you compile the tool without the debugger. Or just create an executable and run that (advised).
v2.6 was mostly rewritten and the whole program is now native (PureBasic) code no conditional Window/Linux stuff like with v2.4 etc.
One requirement though is you must use one of the more recent PureBasic released with the improved accuracy on ElapsedMilliseconds().
When you run the test tool it open a file requester will appear, simply select 1 or more files to test. I suggest selecting a assortment of files both very small and large, do not that the larger the file (and the more there is of them) the slower the test will go, especially the random part of it. (Both random seek and a sequential read is performed). When done a messagebox will show the result of the test, after you click the OK button it will also try to launch your browser with a special url for Google Charts API, if all works as intended you should get a nice looking chart representing the info in the message box. The slowest chunksize is used as the base line and will show as 100% in the Google graph, in the message requester it's shown as 1.0x instead.
Please feel free to post your results in this thread.
This way I (and anyone else interested) can simply compare results.
The source is Public Domain, although I request that if you intend to post results in this thread that the way the speed measurement is done etc. is not changed, this is just to avoid the pain of mixed ways to measure things (change the name etc. if you do that).
Please note that the test starts slow (due to the small chunksizes) and will speed up as it starts increasing the chunksizes.
On my system 40 files with a total if 1.92 GB took about 0.317 minutes (19 seconds) as can be seen as the first example below.
On large files it will look like the test has gotten stuck, it has not it just takes a long time to update the progress/percentage as this is updated once per filechange, I did ponder updating the percentage during the filereading itself but that could have affected the timing a bit (updating the console title could have caused a delay/lag in the stats).
It is highly advised you select a larger amount of files of varying sizes, your download directory should do nicely as it'll have different sized files and both old and new files and most likely quite some fragmentation, making for a nice "real" test.
Do note that a lot can influence the results and you may find that testing with the same files again may cause one test to cause a particular chunksize to be very slow or fast, if you see such anomalies you may want to re-run your test until you get consistent results.
Anomalies are unfortunate and do happen but are impossible to optimize against. If you want to know a good size to use then pick the fastest one from multiple tests on multiple drives and different file collections.
Myself I've found my system seems to favor 128KB and 256K with 128KB being the more consistent one.
Also note that you will probably need to use PureBasic's URLEncoder() to make the Google chart url useable on the forum here as the forum has issues with certain urls that are not urlencoded.
Readspeed Test v2.6 on a HDD (SATA2)
Code: Select all
Readspeed Test v2.6
---------------------------
The baseline is the slowest chunksize, and shown as 1.0x
Data based on 40 files.
Amount of total data processed is ~35493.81 MB.
The test took 0.317 minutes to complete.
16384 chunksize speed = 1.00x
32768 chunksize speed = 1.14x
65536 chunksize speed = 1.29x
131072 chunksize speed = 1.38x
262144 chunksize speed = 1.39x
524288 chunksize speed = 1.31x
1048576 chunksize speed = 1.34x
2097152 chunksize speed = 1.31x
4194304 chunksize speed = 1.07x
Code: Select all
Readspeed Test v2.6
---------------------------
The baseline is the slowest chunksize, and shown as 1.0x
Data based on 2 files.
Amount of total data processed is ~22555.12 MB.
The test took 2.200 minutes to complete.
16384 chunksize speed = 1.00x
32768 chunksize speed = 82.24x
65536 chunksize speed = 90.61x
131072 chunksize speed = 98.10x
262144 chunksize speed = 97.62x
524288 chunksize speed = 94.57x
1048576 chunksize speed = 95.17x
2097152 chunksize speed = 92.97x
4194304 chunksize speed = 76.23x
Code: Select all
Readspeed Test v2.6
---------------------------
The baseline is the slowest chunksize, and shown as 1.0x
Data based on 6 files.
Amount of total data processed is ~35209.25 MB.
The test took 15.517 minutes to complete.
16384 chunksize speed = 1.00x
32768 chunksize speed = 397.83x
65536 chunksize speed = 437.27x
131072 chunksize speed = 461.82x
262144 chunksize speed = 459.72x
524288 chunksize speed = 446.68x
1048576 chunksize speed = 451.31x
2097152 chunksize speed = 442.14x
Code: Select all
Readspeed Test v2.6
---------------------------
The baseline is the slowest chunksize, and shown as 1.0x
Data based on 19 files.
Amount of total data processed is ~7831.46 MB.
The test took 2.117 minutes to complete.
16384 chunksize speed = 1.00x
32768 chunksize speed = 215.75x
65536 chunksize speed = 255.92x
131072 chunksize speed = 283.83x
262144 chunksize speed = 272.45x
524288 chunksize speed = 271.85x
1048576 chunksize speed = 276.14x
2097152 chunksize speed = 267.10x
4194304 chunksize speed = 219.23x
Code: Select all
;Test Readspeed v2.10, Public Domain.
EnableExplicit
DisableDebugger
#Program_Title = "Readspeed Test v2.10"
Enumeration 1
#Window_Main
EndEnumeration
Enumeration 1
#Gadget_Progress
#Gadget_Button1
#Gadget_Button2
EndEnumeration
Global result$, url$
Procedure.i ReadspeedTest()
Protected *mem
Protected text$
Protected size.q, totalsize.q, flen.q, filesizesum.q
Protected fh.i, index.i
Protected base.d, sum.d, maxvalue.d, filecount.d, filecountmax.d
Protected start.l, stop.l, datestart.l, datestop.l, chunksize.l
Protected NewList speedtest.q()
Protected Newlist file.s()
chunksize = 4194304
*mem=AllocateMemory(4194304)
If *mem=#Null
ProcedureReturn #False
EndIf
For index = 1 To 9
AddElement(speedtest())
speedtest() = 0
Next
text$ = OpenFileRequester("Choose file(s) for readspeed test:", "", "All (*.*)|*.*", 0, #PB_Requester_MultiSelection)
While text$
size = FileSize(text$)
If size > 0
AddElement(file())
file() = text$
filesizesum + size
filecount + 1
EndIf
text$ = NextSelectedFileName()
Wend
If ListSize(file()) < 1
If *mem
FreeMemory(*mem)
EndIf
ProcedureReturn #False
EndIf
filecountmax = filecount * 2 * ListSize(speedtest())
filecount = 0.0
chunksize = 16384
datestart = Date()
ForEach speedtest()
ConsoleTitle(#Program_Title + " " + StrD((filecount / filecountmax) * 100.0, 2) + "% Random " + Str(chunksize) + "B chunksize.")
ForEach file()
fh = ReadFile(#PB_Any, file())
If fh
size = Lof(fh)
PrintN(GetFilePart(file()) + " " + Str(size) + " bytes.")
FileBuffersSize(fh, 0)
flen = size
totalsize + size
start = ElapsedMilliseconds()
Repeat
If flen > chunksize
FileSeek(fh, Random(flen - chunksize))
EndIf
size - ReadData(fh, *mem, chunksize)
Until size <= 0
stop = ElapsedMilliseconds()
speedtest() + (stop - start)
CloseFile(fh)
EndIf
filecount + 1
Next
ConsoleTitle(#Program_Title + " " + StrD((filecount / filecountmax) * 100.0, 2) + "% Sequential " + Str(chunksize) + "B chunksize.")
ForEach file()
fh = ReadFile(#PB_Any, file())
If fh
FileBuffersSize(fh, 0)
size = Lof(fh)
PrintN(GetFilePart(file()) + " " + Str(size) + " bytes.")
totalsize + size
start = ElapsedMilliseconds()
Repeat
size - ReadData(fh, *mem, chunksize)
Until size <= 0
stop = ElapsedMilliseconds()
speedtest() + (stop - start)
CloseFile(fh)
EndIf
filecount + 1
Next
chunksize << 1
Next
datestop = Date()
ConsoleTitle(#Program_Title + "100%")
If *mem
FreeMemory(*mem)
EndIf
base = 0
ForEach speedtest()
If base < speedtest()
base = speedtest()
EndIf
Next
text$ = ""
text$ + "The baseline is the slowest chunksize, and shown as 100.00 %" + #LF$
size = ListSize(file())
If size > 1
text$ + "Based on " + Str(size) + " files and " + StrD(filesizesum / 1048576.0,2) + " MB of data (processed " + StrD(totalsize / 1048576.0,2) + ")." + #LF$
Else
text$ + "Based on 1 file" + " and " + StrD(filesizesum / 1048576.0,2) + " MB of data (rocessed " + StrD(totalsize / 1048576.0,2) + ")." + #LF$
EndIf
text$ + "Test took " + StrD((datestop - datestart) / 60.0, 3) + " minutes, speed was " + StrD(totalsize / 1048576.0 / (datestop - datestart), 2) + " MB/s (" + StrD(totalsize / 131072.0 / (datestop - datestart), 2) + " mbit/s)." + #LF$
url$ = "http://chart.apis.google.com/chart?chtt=" + #Program_Title
url$ + "&chs=388x773&chm=N*f*%,000000,0,-1,11&cht=bvg&chbh=a"
url$ + "&chco=00007F|7F7F7F|7F0000|007F00|7F7F00|007F7F|7F007F|FF7F7F|7F7FFF"
url$ + "|7F7F00|007F7F|7F007F|FF7F7F|7F7FFF&chdl="
chunksize = 16384
ForEach speedtest()
index + 1
If chunksize < 1024
url$ + Str(chunksize) + "B"
ElseIf chunksize < 1048576
url$ + Str(chunksize / 1024) + "KB"
Else
url$ + Str(chunksize / 1048576) + "MB"
EndIf
If ListIndex(speedtest()) <> (ListSize(speedtest()) - 1)
url$ + "|"
EndIf
chunksize << 1
Next
url$ + "&chd=t:"
chunksize = 16384
ForEach speedtest()
sum = speedtest()
If sum = 0
sum = 1.0
EndIf
sum = base / sum
If sum > maxvalue
maxvalue = sum
EndIf
text$ + #LF$ + Str(chunksize) + " chunksize speed = " + StrD(sum *100.0, 2) + " %"
chunksize << 1
url$ + Str((sum * 100))
If ListIndex(speedtest()) <> (ListSize(speedtest()) - 1)
url$ + ","
EndIf
Next
result$ = text$
url$ + "&chds=99," + Str((maxvalue * 100) + 1)
ProcedureReturn #True
EndProcedure
Define result.i
If OpenConsole() = #False
;Failed to open window.
End
EndIf
ConsoleTitle(#Program_Title)
result = ReadspeedTest()
CloseConsole()
If result
MessageRequester(#Program_Title, result$)
RunProgram(URLEncoder(url$))
EndIf
End