wx.grid/wx.grid.GridTableBase can not show data more than 7M rows

wxPython Dev
#1

Hi,

Python: 3.9.5
wxPython: 4.1.2a1.dev5165+64e5d863 msw (phoenix)
wxWidgets 3.1.5

The virtual wx.gridtablebase model is known to be able showing infinite rows, but it failed showing more than 7M rows as the following minimal example.
Are there any way to work around?
Thanks.

import wx
import wx.grid

print(wx.VERSION_STRING)

class GTable(wx.grid.GridTableBase):
def init(self):
wx.grid.GridStringTable.init(self)

def GetNumberRows(self):
    return 10000000

def GetNumberCols(self):
    return 2

def GetColLabelValue(self, col):
    return "A"+str(col)
    
def GetRowLabelValue(self, row):
    return str(row+1)
    
def IsEmptyCell(self, row, col):
    return False
    
def GetValue(self, row, col):
    return str(row)+'-'+str(col)
    
def GetTypeName(self, row, col):
    return wx.grid.GRID_VALUE_STRING

def SetValue(self, row, col, value):
    pass

class MyGrid(wx.grid.Grid):
def init(self, parent):
wx.grid.Grid.init(self, parent, -1)
gtable = GTable()
self.SetTable(gtable,True)

class TestFrame(wx.Frame):
def init(self, parent):
wx.Frame.init(self, parent, -1, “A Grid”, size=(400, 400))
grid = MyGrid(self)

if name == ‘main’:
app = wx.App()
frame=TestFrame(None)
app.SetTopWindow(frame)
frame.Show()
app.MainLoop()

CrashedGrid Image:

CrashedGrid
CrashedGrid1920×1080 188 KB

#2

As is often the case with such issues, there’s nothing wxPython can do about it. You are probably hitting the GDI limits on wxMSW. You may be fine under wxGTK, but I haven’t tested it.

Curiously, a virtual multi-column ListCtrl seems to be up to the task instead, from what I can see by editing the demo example. This may be your fix.

That said, if I may add some unsolicited advice…

The virtual wx.gridtablebase model is known to be able showing infinite rows

Ehm. No it’s not. No physical system can hold infinite information. There’s not even such a thing as infinite information and, above all, no one wants infinite information. Especially if you have to actually digest it on a computer screen.
Specifically, 7M rows are not for the human eye to be scrolled on a GUI list. It’s the most frustrating user experience: if you keep clicking on the scroll bar, it will take forever; if you drag the thumb you’ll miss by thousands at the smallest throb of the mouse. It’s just that GUIs are not meant for this (I’d say, humans are not meant for this…). You may want to provide some filtering/searching capability instead (or at least pagination): your users will thank you.

#3

Hi, ricpol

Thanks for your reply and advice.

there’s nothing wxPython can do about it. You are probably [hitting the GDI limits]

I’m a beginner of wxPython, but i think wxPython can avoid ‘hitting the GDI limits’ by release (or free) the resource (GDI?) when the viewing window moves from one position to another.

You may be fine under wxGTK

I don’t know wxGTK, i’ll try it. can it be used in Windows OS?

multi-column ListCtrl

I wonder if we can add GridLabelRenderer/GridCellRenderer into ListCtrl as we did in grid?

Ehm. No it’s not. No physical system can hold infinite information. There’s not even such a thing as >infinite information and, above all, no one wants infinite information.

It’s different meaning between ‘showing infinite rows’ and ‘hold infinite rows’, The ‘infinite’, I used, is borrowed from ‘the infinite row model’ in javascript grid world. here the ‘infinite’ means ‘No limit’.

7M rows are not for the human eye to be scrolled on a GUI list.

Certainly not! even not on 10K, or 1K rows data.
But people want to make a look on raw data (especially when they touch raw data first time) not only seeing summarized or filtered information. System (even user selves) don’t not know what kind of summarized information satisfies their needs in advance.
Of course, I don’t denial the importance of filtering/searching capability.

Thank you again.

#4

…GDI… wxGTK… etc etc

see this first, then this, then this. And the link in my previous post. As I said, there’s nothing wxPython can do about it…

I wonder if we can add…

Well, a grid is not a list. However, there are many flavors of lists in wxPython, and many possible customizations. Filter the demo by “list” to get an overview of the possibilities.

It’s different meaning between ‘showing infinite rows’ and ‘hold infinite rows’

Yes, the same difference between knowing that there are infinite integer numbers and actually have to scroll a list of 7m integer numbers.
The point is not that you have, say, a database table of 7m+ rows… this is fairly common these days. Rather, you are loading your GUI widget with 7m rows of data, for the user to scroll through.
Now, every rule has its exception of course, but this is a common antipattern in GUI design, and I have seen my fair share of it.
Of course sometimes people may want to get the entire load to analyze, dig deeper, whatever. But then you’ll be better off dumping the table in a csv file for later consumption and leave users free to use their editor of choice.
A screen GUI is meant for ease of searching, filtering, selecting, sampling… If you’re just dumping the whole hot mess in a list for the user to deal with it, you are giving up GUI design entirely. As a rule of thumb, I’d recommend not showing more than 100 rows at a time.

#5

Thank you very much for your advice.

Well, a grid is not a list. However, there are many flavors of lists in wxPython, and many possible >customizations. Filter the demo by “list” to get an overview of the possibilities.

I want to have ‘the selection of range of cells’ as we do by excel, I also need to customize the column name (titles) bar, unfortunately, It seems ‘list’ has no such ability.

knowing that there are infinite integer numbers and actually have to scroll a list of 7m integer numbers.

I feel that our discussion is not in the same wavelength.
The ability of show raw data, does not equal
(1). force user scrolling all that data
(2). reject (or disable) the ability of searching, filtering, selection, sampling,…,etc

#6

selection of range of cells

I see… this could prove difficult with a list control. Basically, in a list the “item” is the entire row. So, you can select multiple rows (even non-consecutive), but not single cells. Maybe you can work your way around this by tinkering with the renderers, but… I don’t know… seems like a lot of work just to implement a poor man’s grid…

As for the rest, feel free to have it your way… at the end of the day, I won’t be the one to use your gui!

#7

ricpol:
Thanks for your comment and advise!

#8

in mathematics infinity is governed by axioms, like the Peano axioms for the set of natural numbers and its infinity proofed indirectly, i.e. assuming it is finite and showing bla bla bla
of course, the formulation of such a description may take some effort, but without it no one can be sure what the talk is all about
the urge for showing a large amount of undigested ‘info’? usually has its roots in a missing conceptual framework and some people are rushing into chaos theory or (a better seller these days) AI, but in both cases one doesn’t need a big GUI rather a lot of computing power: hardware beats software