Populating a ListView quickly?
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by PB.
I have 600 lines of data to add to a ListView gadget, totalling 25k.
Adding these 600 lines to the ListView is very slow, making it totally
inaccessible during the populating. If I stick a single WindowEvent()
in there, it makes the gadget visible during the populating, but the
vertical scroll bar doesn't appear until the end of the list, and my
app is "paused" until the end, too. Any ideas on how to do it faster?
API calls? Searching these forums for "listview slow" came up empty.
I have 600 lines of data to add to a ListView gadget, totalling 25k.
Adding these 600 lines to the ListView is very slow, making it totally
inaccessible during the populating. If I stick a single WindowEvent()
in there, it makes the gadget visible during the populating, but the
vertical scroll bar doesn't appear until the end of the list, and my
app is "paused" until the end, too. Any ideas on how to do it faster?
API calls? Searching these forums for "listview slow" came up empty.
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by Pupil.
I use to fill listview gadget inside a thread, works fine for me, the only thing is that you see when the gadget gets filled(might be that it fills a bit slower because the OS updates the gadget for every item you add), but that's not a big problem, not to me anyway..
I use to fill listview gadget inside a thread, works fine for me, the only thing is that you see when the gadget gets filled(might be that it fills a bit slower because the OS updates the gadget for every item you add), but that's not a big problem, not to me anyway..
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by horst.
It takes 170 ms (@500 Mhz). Is that too slow, or do you have some
other problem?
Horst
I just tested this with 25Kb, 600 lines and (buffered) line reading.Originally posted by PB
I have 600 lines of data to add to a ListView gadget, totalling 25k.
Adding these 600 lines to the ListView is very slow, ...
It takes 170 ms (@500 Mhz). Is that too slow, or do you have some
other problem?
Horst
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by PB.
> I just tested this with 25Kb, 600 lines and (buffered) line reading.
> It takes 170 ms (@500 Mhz). Is that too slow, or do you have some
> other problem?
In comparison, my 600 lines take 5288 ms to complete (@667 Mhz), so
your measly 170 ms is definitely NOT too slow.
Kale/Pupil: Doing it in a thread makes no difference, either.
I neglected to mention what I'm doing in my original post: I am actually
calculating strings prior to adding to the ListView -- I'm not filling
it with pre-made strings or with lines from a file. So perhaps all the
string operations are slowing it down. I'll try creating an array
first, then filling the ListView with the array... back in a sec...
Nope, still takes 4777 ms to fill (on average). Guess I'll just have
to live with it.
> I just tested this with 25Kb, 600 lines and (buffered) line reading.
> It takes 170 ms (@500 Mhz). Is that too slow, or do you have some
> other problem?
In comparison, my 600 lines take 5288 ms to complete (@667 Mhz), so
your measly 170 ms is definitely NOT too slow.
Kale/Pupil: Doing it in a thread makes no difference, either.
I neglected to mention what I'm doing in my original post: I am actually
calculating strings prior to adding to the ListView -- I'm not filling
it with pre-made strings or with lines from a file. So perhaps all the
string operations are slowing it down. I'll try creating an array
first, then filling the ListView with the array... back in a sec...
Nope, still takes 4777 ms to fill (on average). Guess I'll just have
to live with it.
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by Danilo.
Here is a little Test, so you see its little bit
faster if you lock the gagdet redrawing while
adding the new items.
Procedure Test3() which locks the update
while adding strings is 5 times faster
than Test2() here, so maybe _that_ helps you.
cya,
...Danilo
(registered PureBasic user)
Here is a little Test, so you see its little bit
faster if you lock the gagdet redrawing while
adding the new items.
Code: Select all
DisableDebugger
Procedure Test1()
ClearGadgetItemList(1)
While WindowEvent():Wend
StartTime = TimeGetTime_()
For a = 1 To 10000
AddGadgetItem(1,-1,"Entry "+Str(a))
While WindowEvent():Wend
Next a
EndTime = TimeGetTime_() - StartTime
AddGadgetItem(2,-1,"Result Test 1: "+StrU(EndTime,2)+" ms")
EndProcedure
Procedure Test2()
ClearGadgetItemList(1)
While WindowEvent():Wend
StartTime = TimeGetTime_()
For a = 1 To 10000
AddGadgetItem(1,-1,"Entry "+Str(a))
Next a
EndTime = TimeGetTime_() - StartTime
AddGadgetItem(2,-1,"Result Test 2: "+StrU(EndTime,2)+" ms")
EndProcedure
Procedure Test3()
ClearGadgetItemList(1)
While WindowEvent():Wend
StartTime = TimeGetTime_()
SendMessage_(GadgetID(1),#WM_SETREDRAW, #FALSE, 0)
For a = 1 To 10000
AddGadgetItem(1,-1,"Entry "+Str(a))
Next a
SendMessage_(GadgetID(1),#WM_SETREDRAW, #TRUE, 0)
EndTime = TimeGetTime_() - StartTime
AddGadgetItem(2,-1,"Result Test 3: "+StrU(EndTime,2)+" ms")
EndProcedure
OpenWindow(1,0,0,300,300,#PB_Window_ScreenCentered|#PB_Window_SystemMenu,"")
CreateGadgetList(WindowID())
ListViewGadget(1,10,10,280,200)
ListViewGadget(2,10,225,280,65)
While WindowEvent():Wend
Test1()
While WindowEvent():Wend
Test2()
While WindowEvent():Wend
Test3()
AddGadgetItem(2,-1,"DONE!")
Repeat
Until WaitWindowEvent() = #PB_Event_CloseWindow
while adding strings is 5 times faster
than Test2() here, so maybe _that_ helps you.
cya,
...Danilo
(registered PureBasic user)
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by PB.
> SendMessage_(GadgetID(1),#WM_SETREDRAW, #FALSE / #TRUE, 0)
Nice little tip there, Danilo -- definitely one to remember!
However, I have since discovered that it is indeed my string handling
that is causing the delay, because if I pre-render the strings before
enter the AddGadgetItem loop, then all is fine -- the population of
the ListView is virtually instant, with all 600 lines. (When I last
tried this, I was rendering inside the AddGadgetItem loop, which was
silly as it still slowed the loop down).
So I need to optimise my string calculations first, which may be a
bit harder to do. Basically, I'm doing a hex display of data in a
ListView gadget. So I have to get each line, render it into a bunch
of hex digits, followed by the ASCII look, and AddGadgetItem it.
You know how any normal hex editor looks? That's what I'm doing,
but it's proving very slow.
> SendMessage_(GadgetID(1),#WM_SETREDRAW, #FALSE / #TRUE, 0)
Nice little tip there, Danilo -- definitely one to remember!
However, I have since discovered that it is indeed my string handling
that is causing the delay, because if I pre-render the strings before
enter the AddGadgetItem loop, then all is fine -- the population of
the ListView is virtually instant, with all 600 lines. (When I last
tried this, I was rendering inside the AddGadgetItem loop, which was
silly as it still slowed the loop down).
So I need to optimise my string calculations first, which may be a
bit harder to do. Basically, I'm doing a hex display of data in a
ListView gadget. So I have to get each line, render it into a bunch
of hex digits, followed by the ASCII look, and AddGadgetItem it.
You know how any normal hex editor looks? That's what I'm doing,
but it's proving very slow.
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by geoff.
PB, this sounds exactly like the display of my file edit program written in a different language that I was about to convert to Purebasic.
In this program, I look at the position of the window scroll bar, then dive into the file at this position and draw the screen with the hex digits at that point in the file. Of course, this works OK for files of any size, even 100MB say.
Obviously you couldn't load the whole file into a list view gadget. That's why I was enquiring about the possibility of a scoll bar gadget that could be added to any window, but so far Fred has been rather quite on this one.
PB, this sounds exactly like the display of my file edit program written in a different language that I was about to convert to Purebasic.
In this program, I look at the position of the window scroll bar, then dive into the file at this position and draw the screen with the hex digits at that point in the file. Of course, this works OK for files of any size, even 100MB say.
Obviously you couldn't load the whole file into a list view gadget. That's why I was enquiring about the possibility of a scoll bar gadget that could be added to any window, but so far Fred has been rather quite on this one.
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by horst.
Horst
Do it without string operations, like this:Originally posted by PB
[br.. I'm doing a hex display of data ..
Code: Select all
hex.s = "0123456789ABCDEF"
source.s = "Testing ABC"
product.s = Space(Len(source)*3)
For i = 0 To Len(source)-1
byte = PeekB(@source+i)
digit = byte >> 4 : high = PeekB(@hex+digit)
digit = byte & $F : low = PeekB(@hex+digit)
PokeB(@product+p,high) : p+1
PokeB(@product+p,low) : p+2
Next
MessageRequester("Test",source+Chr(13)+product,0)
End
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm
Restored from previous forum. Originally posted by Danilo.
PB, here is a small but fast HEX viewer for you...
Speed should be OK because you said you use
max. 64k data size.
cya,
...Danilo
(registered PureBasic user)
PB, here is a small but fast HEX viewer for you...
Code: Select all
;>--------------------------------------------------------
;>--------------------------------------------------------
;
; Pure HexView, by Danilo, 22.01.2003
;
; Thanks go to 'PB' and 'horst' for
; ideas and hints!
;
DisableDebugger ; !!
;
;- Structures
;
Structure Byte
b.b
EndStructure
;
;- Global variables - arrays - linked lists
;
Dim AsciiChars.b(255)
For a = 0 To 255 : AsciiChars(a) = '.' : Next a
For a = 32 To 127 : AsciiChars(a) = a : Next a
Dim HexChars.b(16)
PokeS(HexChars(),"0123456789ABCDEF")
;
;- Procedures
;
Procedure AskForEnd()
If MessageRequester("Exit","Really exit 'Pure HexView' ?",#MB_YESNO|#MB_ICONQUESTION) = #IDYES
End
EndIf
EndProcedure
Procedure MOD(a,b)
ProcedureReturn a-a/b*b
EndProcedure
Procedure FillListViewHex(*mem.Byte,size)
;
; Fill Listview with hex values
; argument 1: memory-pointer for data
; argument 2: size of this memory area
;
;Delay(1)
StartTime = TimeGetTime_()
SendMessage_(GadgetID(1),#WM_SETREDRAW, #FALSE, 0)
A$ = Space(16*4)
PokeB(@A$+16*3,'|')
For a = 1 To size/16
*Text.Byte = @A$
*RealText.Byte = @A$+16*3+2
For b = 1 To 16
*Text\b = HexChars((*mem\b & $FF) >> 4);*offset\b ;PeekB(@HEX$ + offset)
*Text+1
*Text\b = HexChars((*mem\b & $F));*offset\b ;PeekB(@HEX$ + offset)
*Text+2
*RealText\b = AsciiChars(*mem\b & $FF)
*RealText+1
*mem + 1
Next b
AddGadgetItem(1,-1,A$)
Next a
b = MOD(size,16)
If b
A$ = Space(16*4)
PokeB(@A$+16*3,'|')
*Text.Byte = @A$
*RealText.Byte = @A$+16*3+2
For a = 1 To b
*Text\b = HexChars((*mem\b & $FF) >> 4)
*Text+1
*Text\b = HexChars((*mem\b & $F))
*Text+2
*RealText\b = AsciiChars(*mem\b & $FF)
*RealText+1
*mem + 1
Next a
AddGadgetItem(1,-1,A$)
EndIf
SendMessage_(GadgetID(1),#WM_SETREDRAW, #TRUE, 0)
EndTime = TimeGetTime_() - StartTime
;MessageRequester("INFO","Filled ListViewGadget with "+StrU(size,2)+" HEX values in "+StrU(EndTime,2)+" ms",0)
SetGadgetText(3,StrU(size,2)+" bytes - "+StrU(Size/16+b,2)+" lines generated in "+StrU(EndTime,2)+" ms")
EndProcedure
Procedure LoadFileInListView()
File$ = OpenFileRequester("Load File","","All Files | *.*",0)
If File$
If ReadFile(1,File$)
size = Lof()
If size > 200*1024
If MessageRequester("Warning","Really load BIG file "+GetFilePart(File$)+" ("+StrU(size,2)+" bytes) ??"+Chr(13)+"This can take much time!",#MB_ICONWARNING|#MB_YESNO) = #IDNO
CloseFile(1)
ProcedureReturn
EndIf
EndIf
mem = AllocateMemory(1,size)
If mem = 0
MessageRequester("ERROR","Cant allocate "+StrU(size,2)+" bytes of memory!",#MB_ICONERROR)
CloseFile(1)
ProcedureReturn
EndIf
ReadData(mem,size)
CloseFile(1)
ClearGadgetItemList(1)
SetGadgetText(3,"File "+GetFilePart(File$)+" loaded ... generating HexView")
While WindowEvent():Wend
FillListViewHex(mem,size)
FreeMemory(1)
Else
MessageRequester("ERROR","Cant read file!",#MB_ICONERROR)
EndIf
EndIf
EndProcedure
;
;- App START
;
OpenWindow(1,0,0,760,520,#PB_Window_ScreenCentered|#PB_Window_SystemMenu,"Pure HexView")
hFont = LoadFont(1,"Lucida Console",14)
CreateGadgetList(WindowID())
ListViewGadget(1,0,0,760,500): SendMessage_(GadgetID(1),#WM_SETFONT,hFont,1)
ButtonGadget(2,640,501,120,20,"Load File in ListView")
SetGadgetFont(hFont)
TextGadget(3,0,501,640,20,"",#PB_Text_Center)
While WindowEvent():Wend
; produce 32k random data memory for speed testing :)
mem_data_size = 32*1024
*mem_data.Byte = AllocateMemory(1,mem_data_size)
*mem.Byte = *mem_data
For a = 1 To mem_data_size
*mem\b = Random($FF)
*mem + 1
Next a
PokeS(*mem_data,"HexViewer for PB")
For a = 0 To 255 : PokeB(*mem_data+16+a,a) : Next a
PokeS(*mem_data+mem_data_size-12,"__The End__")
; Fill ListViewGadget with random data
FillListViewHex(*mem_data,mem_data_size)
; Free random data memory
FreeMemory(1)
;
;- App message loop
;
Repeat
Select WaitWindowEvent()
Case #PB_Event_CloseWindow : AskForEnd()
Case #PB_Event_Gadget
Select EventGadgetID()
Case 2 : LoadFileInListView()
EndSelect
EndSelect
ForEver
;
;- App END
;
;>---------------------------------------------------------
;>---------------------------------------------------------
max. 64k data size.
cya,
...Danilo
(registered PureBasic user)
-
- PureBasic Guru
- Posts: 16777133
- Joined: Tue Apr 22, 2003 7:42 pm