Practical Limit for VirtualList with DB

Hi David,

Are you using OnCacheHint event?

I don't have tables as large as you but using the cache hint stuff gives me fine performance on a tables with over 15,000 rows.

If you are not already using it and if you need more detail let me know.
Werner

David Pratt wrote:

···

Hi Josiah. Thank you for you follow up. I am still relatively new to wxPython so I will take a closer look at what you are suggesting. SQLite has cursors and is compliant with python api and sql. It also has advantage of being very fast because it is very light weight.

I guess as far as mechanics, what should I do from within the control to trigger getting more records, is this by using wx.CallAfter? I will be re-reading your response and looking at what these methods provide today. Many thanks.

Regards,
David

Josiah Carlson wrote:

David Pratt <fairwinds@eastlink.ca> wrote:

Hi Josiah

It sounds like your "virtual list" implementation has some issues. How
is your "virtual list" implemented? Are you pulling those 10k records
from a DB before you are displaying them?

Yes, I am working with SQLite (pretty fast) db to pull the data from a database first.

How long does it take to pull the data?

Practically speaking, it doesn't seem like virtual list controls should
have the issues you describe...the virtual list in the wxPython demo has
1 million virtual elements, and it's very fast to load.

This is the reason I asked because I found the virtual list demo quite quick for all those values. I am using a custom vitual list to get sorting of the contents so it has been subclassed. It subclasses wx.ListCtrl, listmix.ListCtrlAutoWidthMixin, and listmix.ColumnSorterMixin. Then have some custom methods to have this work generically for a column data dict and a dictionary of data derived from database so it will work with the control.

Have you any advice on how I could manage this more effectively?

Pull data only as necessary. Push sorting to the DB. etc.

If you are going to pull all of the data, rather than using dictionaries,
use lists of tuples for a smaller memory footprint (and if you can get
your rows as tuples instead of dictionaries initially, it'll also be
faster to create).

When my combobox is selected I do not destroy the virtual list control but delete its contents and refill it with data.

How are you refilling your data? Whenever I've had a virtual list
control, and I wanted to replace the data already there, I usually just
used lc.SetItemCount(0);wx.CallAfter(lc.SetItemCount, count) (also
remembering to sort the previous data columns like they were before).

The lists are pretty snappy from large to small lists but slow going from small list to a pretty large list (where I am hoping to make improvements)

Could be the data fetch, could be the refilling...could be many things.

Also, you may want to consider using a cursor (does sqlite support
cursors?) to get blocks of data and doing a live-update of the control
as it comes in. Say, for example, you know that getting 1k records
takes like .002 seconds. Get 1k records NOW, update the control, then
use wx.CallAfter() or wx.FutureCall(<delay in ms>, <fcn>, *args) with a
delay of like 20-50 ms. When you get more, update the control, etc.

- Josiah

---------------------------------------------------------------------
To unsubscribe, e-mail: wxPython-users-unsubscribe@lists.wxwidgets.org
For additional commands, e-mail: wxPython-users-help@lists.wxwidgets.org