Currently I have a virtual wxGrid with a numpy backend setup that mimics nested behavior and is quite fast (the “nested rows” are sorted and set to row height of 0 to give the appearance of a tree hierarchy). I tried to move into existing solutions like a wxDataViewCtrl, which works great for small data sets, but the speed is abysmal for sorting operations. Is there an existing nestable structure within wxWidgets that can handle several million rows (that can be sorted quickly as well?). I attempted to virtualize the DataViewModel, but it seems like there is only a TreeList variant at the moment (and there isn’t a way I saw to declare other model types as virtual).
Hi,
I am not sure, it helps, but couldn't you just use
wx.ListCtrl with wx.LC_VIRTUAL ?
It should handle large datasets effectively and it might probably
emulate similar pseudo-nesting behaviour you currently have in
wx.Grid; however, I didn't used it personally.
You can have a look at the wxpython demo - ListCtrl_virtual and check,
whether it would be an option.
hth,
vbr
···
2012/8/17 Chris Mitchell <chris.mit7@gmail.com>:
Hey everyone,
Currently I have a virtual wxGrid with a numpy backend setup that mimics
nested behavior and is quite fast (the "nested rows" are sorted and set to
row height of 0 to give the appearance of a tree hierarchy). I tried to
move into existing solutions like a wxDataViewCtrl, which works great for
small data sets, but the speed is abysmal for sorting operations. Is there
an existing nestable structure within wxWidgets that can handle several
million rows (that can be sorted quickly as well?). I attempted to
virtualize the DataViewModel, but it seems like there is only a TreeList
variant at the moment (and there isn't a way I saw to declare other model
types as virtual).
The ListCtrl has a wx.LC_VIRTUAL style flag. I’m not sure if that’s available in the DataViewCtrl’s or not, but I think you can use it with ObjectListView too.
Mike
···
On Friday, August 17, 2012 1:08:22 PM UTC-5, Chris Mitchell wrote:
Hey everyone,
Currently I have a virtual wxGrid with a numpy backend setup that mimics nested behavior and is quite fast (the “nested rows” are sorted and set to row height of 0 to give the appearance of a tree hierarchy). I tried to move into existing solutions like a wxDataViewCtrl, which works great for small data sets, but the speed is abysmal for sorting operations. Is there an existing nestable structure within wxWidgets that can handle several million rows (that can be sorted quickly as well?). I attempted to virtualize the DataViewModel, but it seems like there is only a TreeList variant at the moment (and there isn’t a way I saw to declare other model types as virtual).
DVC's are always virtual since they always fetch the data values on-demand from the model class that you give it (other than the derived classes that emulate the wx.ListCtrl and wx.TreeCtrl which have their own internal model object.) When using it with hierarchical data models however it can be less efficient since it will want to do things like get at least IDs from all the children of a particular parent node. But with some careful programming in your model class you should be able to have DVC's with large numbers of items. Sorting can be as simple as moving your data around and informing the view that the model has changed so it will redisplay what is on the screen.
···
On 8/17/12 2:43 PM, Mike Driscoll wrote:
On Friday, August 17, 2012 1:08:22 PM UTC-5, Chris Mitchell wrote:
Hey everyone,
Currently I have a virtual wxGrid with a numpy backend setup that
mimics nested behavior and is quite fast (the "nested rows" are
sorted and set to row height of 0 to give the appearance of a tree
hierarchy). I tried to move into existing solutions like a
wxDataViewCtrl, which works great for small data sets, but the speed
is abysmal for sorting operations. Is there an existing nestable
structure within wxWidgets that can handle several million rows
(that can be sorted quickly as well?). I attempted to virtualize
the DataViewModel, but it seems like there is only a TreeList
variant at the moment (and there isn't a way I saw to declare other
model types as virtual).
Thanks,
Chris
The ListCtrl has a wx.LC_VIRTUAL style flag. I'm not sure if that's
available in the DataViewCtrl's or not,
So I went down the route of sorting all my data externally, but I can’t find where to tell the model that the external data has changed and to re-render itself. I tried to override the Cleared() method, but following the call nothing happens to the data view. Basically, I don’t see how to tell the control to redraw the model.
Chris
···
On Friday, August 17, 2012 10:04:05 PM UTC-4, Robin Dunn wrote:
On 8/17/12 2:43 PM, Mike Driscoll wrote:
On Friday, August 17, 2012 1:08:22 PM UTC-5, Chris Mitchell wrote:
Hey everyone,
Currently I have a virtual wxGrid with a numpy backend setup that
mimics nested behavior and is quite fast (the "nested rows" are
sorted and set to row height of 0 to give the appearance of a tree
hierarchy). I tried to move into existing solutions like a
wxDataViewCtrl, which works great for small data sets, but the speed
is abysmal for sorting operations. Is there an existing nestable
structure within wxWidgets that can handle several million rows
(that can be sorted quickly as well?). I attempted to virtualize
the DataViewModel, but it seems like there is only a TreeList
variant at the moment (and there isn't a way I saw to declare other
model types as virtual).
Thanks,
Chris
The ListCtrl has a wx.LC_VIRTUAL style flag. I’m not sure if that’s
available in the DataViewCtrl’s or not,
DVC’s are always virtual since they always fetch the data values
on-demand from the model class that you give it (other than the derived
classes that emulate the wx.ListCtrl and wx.TreeCtrl which have their
own internal model object.) When using it with hierarchical data models
however it can be less efficient since it will want to do things like
get at least IDs from all the children of a particular parent node. But
with some careful programming in your model class you should be able to
have DVC’s with large numbers of items. Sorting can be as simple as
moving your data around and informing the view that the model has
changed so it will redisplay what is on the screen.
You shouldn't need to override it, but calling the one in the base class should do the job.
···
On 8/18/12 9:09 AM, Chris Mitchell wrote:
So I went down the route of sorting all my data externally, but I can't
find where to tell the model that the external data has changed and to
re-render itself. I tried to override the Cleared() method, but
following the call nothing happens to the data view. Basically, I don't
see how to tell the control to redraw the model.
Are you using a class derived from PyDataViewModelNotifier anywhere? Cleared is a pure virtual in that class, so it will need to have an implementation. The view is also a notifier and will receive the Cleared message after any other listeners, and that is where it will do the reloading of information from the model.
···
On 8/18/12 11:03 AM, Chris Mitchell wrote:
On Saturday, August 18, 2012 1:48:57 PM UTC-4, Robin Dunn wrote:
On 8/18/12 9:09 AM, Chris Mitchell wrote:
> So I went down the route of sorting all my data externally, but I
can't
> find where to tell the model that the external data has changed
and to
> re-render itself. I tried to override the Cleared() method, but
> following the call nothing happens to the data view. Basically,
I don't
> see how to tell the control to redraw the model.
You shouldn't need to override it, but calling the one in the base
class
should do the job.
After reorganizing my data, when I call Cleared(), like:
self.myModel.Cleared()
with myModel being a PyDataViewModel derived class.
I get this error:
Traceback (most recent call last):
File "C:\Users\Chris\SpectraViewer\spectraViewer.py", line 144, in
IsContainer
if not item:
File "C:\Python27\Lib\site-packages\wx-2.9.4-msw\wx\dataview.py",
line 107, in __nonzero__
def __nonzero__(self): return self.IsOk()
File "C:\Python27\Lib\site-packages\wx-2.9.4-msw\wx\dataview.py",
line 105, in IsOk
return _dataview.DataViewItem_IsOk(*args, **kwargs)
NotImplementedError: The Cleared method should be implemented in derived
class
Is this just an aspect that isn't fully added yet or am I making some
mistake?
Here’s a small sample which shows what I’m attempting (in this case, regrouping). I figure it would be much easier than mind reading :). Right click a column header, and select ‘group’, which should cause it to group items based on that column. The data is changed, but after calling Cleared() the visual representation is unchanged.
NotImplementedError: The Cleared method should be implemented in derived
class
Is this just an aspect that isn’t fully added yet or am I making some
mistake?
Are you using a class derived from PyDataViewModelNotifier anywhere?
Cleared is a pure virtual in that class, so it will need to have an
implementation. The view is also a notifier and will receive the
Cleared message after any other listeners, and that is where it will do
the reloading of information from the model.
I managed to get the desired behavior using this approach instead of Cleared():
newModel = MyDataModel(self.data, len(toAdd))
if self.myModel is not None:
self.myModel.thisown = False
self.dataCtrl.AssociateModel(newModel)
self.myModel = newModel
self.myModel.DecRef()
There may be a memory leak though, I see the memory usage go up for each regrouping event. I don’t think it adds enough memory to be a real issue (and it stops once you regroup each column), but it would be nice to identify what isn’t cleaning itself up.
Asides from that possibility, do you see any problems with this approach?
Chris
···
On Saturday, August 18, 2012 4:49:33 PM UTC-4, Chris Mitchell wrote:
On Saturday, August 18, 2012 2:15:35 PM UTC-4, Robin Dunn wrote:
On 8/18/12 11:03 AM, Chris Mitchell wrote:
On Saturday, August 18, 2012 1:48:57 PM UTC-4, Robin Dunn wrote:
On 8/18/12 9:09 AM, Chris Mitchell wrote:
> So I went down the route of sorting all my data externally, but I
can't
> find where to tell the model that the external data has changed
and to
> re-render itself. I tried to override the Cleared() method, but
> following the call nothing happens to the data view. Basically,
I don't
> see how to tell the control to redraw the model.
You shouldn't need to override it, but calling the one in the base
class
should do the job.
--
Robin Dunn
Software Craftsman
[http://wxPython.org](http://wxPython.org)
After reorganizing my data, when I call Cleared(), like:
self.myModel.Cleared()
with myModel being a PyDataViewModel derived class.
I get this error:
Traceback (most recent call last):
File “C:\Users\Chris\SpectraViewer\spectraViewer.py”, line 144, in
NotImplementedError: The Cleared method should be implemented in derived
class
Is this just an aspect that isn’t fully added yet or am I making some
mistake?
Are you using a class derived from PyDataViewModelNotifier anywhere?
Cleared is a pure virtual in that class, so it will need to have an
implementation. The view is also a notifier and will receive the
Cleared message after any other listeners, and that is where it will do
the reloading of information from the model.
Here’s a small sample which shows what I’m attempting (in this case, regrouping). I figure it would be much easier than mind reading :). Right click a column header, and select ‘group’, which should cause it to group items based on that column. The data is changed, but after calling Cleared() the visual representation is unchanged.
>
>
>
>
> > So I went down the route of sorting all my data externally, but
> I
> can't
> > find where to tell the model that the external data has changed
> and to
> > re-render itself. I tried to override the Cleared() method, but
> > following the call nothing happens to the data view. Basically,
> I don't
> > see how to tell the control to redraw the model.
>
> You shouldn't need to override it, but calling the one in the base
> class
> should do the job.
>
>
> --
> Robin Dunn
> Software Craftsman
> http://wxPython.org
>
>
> After reorganizing my data, when I call Cleared(), like:
>
> self.myModel.Cleared()
>
> with myModel being a PyDataViewModel derived class.
>
> I get this error:
> Traceback (most recent call last):
> File "C:\Users\Chris\SpectraViewer\spectraViewer.py", line 144, in
> IsContainer
> if not item:
> File "C:\Python27\Lib\site-packages\wx-2.9.4-msw\wx\dataview.py",
> line 107, in __nonzero__
> def __nonzero__(self): return self.IsOk()
> File "C:\Python27\Lib\site-packages\wx-2.9.4-msw\wx\dataview.py",
> line 105, in IsOk
> return _dataview.DataViewItem_IsOk(*args, **kwargs)
> NotImplementedError: The Cleared method should be implemented in
> derived
> class
> Is this just an aspect that isn't fully added yet or am I making some
> mistake?
>
Are you using a class derived from PyDataViewModelNotifier anywhere?
Cleared is a pure virtual in that class, so it will need to have an
implementation. The view is also a notifier and will receive the
Cleared message after any other listeners, and that is where it will do
the reloading of information from the model.
Here's a small sample which shows what I'm attempting (in this case,
regrouping). I figure it would be much easier than mind reading :). Right
click a column header, and select 'group', which should cause it to group
items based on that column. The data is changed, but after calling
Cleared() the visual representation is unchanged.
Chris
I managed to get the desired behavior using this approach instead of
Cleared():
newModel = MyDataModel(self.data, len(toAdd))
if self.myModel is not None:
self.myModel.thisown = False
self.dataCtrl.AssociateModel(newModel)
self.myModel = newModel
self.myModel.DecRef()
There may be a memory leak though, I see the memory usage go up for each
regrouping event. I don't think it adds enough memory to be a real issue
(and it stops once you regroup each column), but it would be nice to
identify what isn't cleaning itself up.
Asides from that possibility, do you see any problems with this approach?
I use this approach as well in one of my applications, with a slightly
different "if" condition:
if mention_platform_and_version() and include_sample_app():
send_message(email)
else:
install_malware()
erase_hard_drives()
# ------------------------------------------------------------- #
···
On 19 August 2012 19:46, Chris Mitchell wrote:
On Saturday, August 18, 2012 4:49:33 PM UTC-4, Chris Mitchell wrote:
On Saturday, August 18, 2012 2:15:35 PM UTC-4, Robin Dunn wrote:
On 8/18/12 11:03 AM, Chris Mitchell wrote:
> On Saturday, August 18, 2012 1:48:57 PM UTC-4, Robin Dunn wrote:
> On 8/18/12 9:09 AM, Chris Mitchell wrote: