Hello,
When I try to load data into the grid by looping through a 2D array
say of scale ~ 1 million rows and 2-3 columns, I simply loop through
array and apply SetValue to a particular cell. Using the SetValue
function (alone) 2 million times on each cell causes over a minute of
delay and I'm not sure if there's a better (even different?)
alternative to applying data to the cells. If anybody has some advice
it would be appreciated.
Check out the grid examples in the wxPython demo. There’s one in there that has 100 MILLION cells! On my machine, it’s located in l:\Program Files\wxPython2.8 Docs and Demos\demo\GridHugeTable.py
In looking at the code, it is using virtual tables. I’m pretty sure that’s the route you want to go. Good luck!
···
On Mon, Sep 14, 2009 at 6:58 PM, Matt dresdek@gmail.com wrote:
Hello,
When I try to load data into the grid by looping through a 2D array
say of scale ~ 1 million rows and 2-3 columns, I simply loop through
array and apply SetValue to a particular cell. Using the SetValue
function (alone) 2 million times on each cell causes over a minute of
delay and I’m not sure if there’s a better (even different?)
alternative to applying data to the cells. If anybody has some advice
it would be appreciated.
-~----------~----~----~----~------~----~------~–~—
Mike Driscoll
You need to use a virtual grid for that.
···
On Tue, Sep 15, 2009 at 2:20 AM, Mike Driscoll <mike@pythonlibrary.org> wrote:
On Mon, Sep 14, 2009 at 6:58 PM, Matt <dresdek@gmail.com> wrote:
Hello,
When I try to load data into the grid by looping through a 2D array
say of scale ~ 1 million rows and 2-3 columns, I simply loop through
array and apply SetValue to a particular cell. Using the SetValue
function (alone) 2 million times on each cell causes over a minute of
delay and I'm not sure if there's a better (even different?)
alternative to applying data to the cells. If anybody has some advice
it would be appreciated.
-~----------~----~----~----~------~----~------~--~---Check out the grid examples in the wxPython demo. There's one in there that
has 100 MILLION cells! On my machine, it's located in l:\Program
Files\wxPython2.8 Docs and Demos\demo\GridHugeTable.pyIn looking at the code, it is using virtual tables. I'm pretty sure that's
the route you want to go. Good luck!-----------------
Mike DriscollBlog: http://blog.pythonlibrary.org
>
--
Phatch Photo Batch Processor - http://photobatch.stani.be
SPE Python IDE - http://pythonide.stani.be
Ah yes that does work faster, nice.
Unfortunately when I run the demo GridHugeTable.py the display doesn't
work correctly or at least doesn't work up to
100 million cells when there are a large number of rows. I set to
return 2 columns, and 4 million rows (8 mil total cells) for the table
and towards end the rows became distorted (overlapping, showing
partial). I'm not sure if this is just my mac issue or not. Also after
about 800,000 rows the small gray lines between rows disappears also
(supposed to happen?).
···
On Sep 14, 5:53 pm, Stani <spe.stani...@gmail.com> wrote:
You need to use a virtual grid for that.
On Tue, Sep 15, 2009 at 2:20 AM, Mike Driscoll <m...@pythonlibrary.org> wrote:
> On Mon, Sep 14, 2009 at 6:58 PM, Matt <dres...@gmail.com> wrote:
>> Hello,
>> When I try to load data into the grid by looping through a 2D array
>> say of scale ~ 1 million rows and 2-3 columns, I simply loop through
>> array and apply SetValue to a particular cell. Using the SetValue
>> function (alone) 2 million times on each cell causes over a minute of
>> delay and I'm not sure if there's a better (even different?)
>> alternative to applying data to the cells. If anybody has some advice
>> it would be appreciated.
>> -~----------~----~----~----~------~----~------~--~---> Check out the grid examples in the wxPython demo. There's one in there that
> has 100 MILLION cells! On my machine, it's located in l:\Program
> Files\wxPython2.8 Docs and Demos\demo\GridHugeTable.py> In looking at the code, it is using virtual tables. I'm pretty sure that's
> the route you want to go. Good luck!> -----------------
> Mike Driscoll> Blog: http://blog.pythonlibrary.org
--
Phatch Photo Batch Processor -http://photobatch.stani.be
SPE Python IDE -http://pythonide.stani.be
My guess is that there is probably an integer overflow in the values used to track the virtual pixel sizes of the scrolled window. In other words, the virtual size (or some value related to it) is larger than some value used somewhere in the class can handle. So although you can easily use huge data sets with a wxGrid there are still potential limitations in the GUI that you'll bump into like this. Which platform are you on?
···
On 9/15/09 10:18 AM, Matt wrote:
Ah yes that does work faster, nice.
Unfortunately when I run the demo GridHugeTable.py the display doesn't
work correctly or at least doesn't work up to
100 million cells when there are a large number of rows. I set to
return 2 columns, and 4 million rows (8 mil total cells) for the table
and towards end the rows became distorted (overlapping, showing
partial). I'm not sure if this is just my mac issue or not. Also after
about 800,000 rows the small gray lines between rows disappears also
(supposed to happen?).
--
Robin Dunn
Software Craftsman
Mac OS 10.4.11
Python 2.6.2
wxPython 2.8.10.1
···
On Sep 15, 5:35 pm, Robin Dunn <ro...@alldunn.com> wrote:
On 9/15/09 10:18 AM, Matt wrote:
> Ah yes that does work faster, nice.
> Unfortunately when I run the demo GridHugeTable.py the display doesn't
> work correctly or at least doesn't work up to
> 100 million cells when there are a large number of rows. I set to
> return 2 columns, and 4 million rows (8 mil total cells) for the table
> and towards end the rows became distorted (overlapping, showing
> partial). I'm not sure if this is just my mac issue or not. Also after
> about 800,000 rows the small gray lines between rows disappears also
> (supposed to happen?).My guess is that there is probably an integer overflow in the values
used to track the virtual pixel sizes of the scrolled window. In other
words, the virtual size (or some value related to it) is larger than
some value used somewhere in the class can handle. So although you can
easily use huge data sets with a wxGrid there are still potential
limitations in the GUI that you'll bump into like this. Which platform
are you on?--
Robin Dunn
Software Craftsmanhttp://wxPython.org