Hi Chris,
Python threads are mapped directly to system threads. There is a limit
on the number of threads which can be active on one system, but it is
quite high (I have a test suite that starts 1000 python threads at
once, without trouble). On Windows, I believe that the system limit is
somewhere around 32k threads.
Thank you for your extensive and very clear explanation. Actually, I
was thinking about extreme cases, as it will be complicated for the
user to select that many files: the app itself requires the user to
input some data for every file, and this has to be done carefully, so
it is likely that the user will not select an exagerate number of
files.
This is especially true in Python, because the global interpreter lock
prevents true concurrency in CPU bound tasks, even on multiple core or
multiple CPU systems. Rewriting your algorithms to be event based and
non-blocking might actually gain you performance. However, writing
code in this style is also difficult (its a paradigm shift from normal
procedural programming) and sometimes the conceptual ease of "lots of
blocking tasks" is worth using threads. This is especially true when
the tasks are highly independent (little or no need to share state
between running threads), and it sounds like this might be.
Yes, this is exactly the case: there is no data sharing between
threads, as they perform completely disconnected actions.
Multiple process concurrency is also an option. Done properly this is
as fast as threads (often faster, in Python, on multiple CPU systems)
but communicating between processes (especially on Windows) is a bit
more awkward.
I thought about multiple processes also, but I am not a big expert in
this kind of things, so I just abandoned the idea from the beginning.
Thank you for your hints.
Andrea.
"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.virgilio.it/infinity77/
ยทยทยท
On 3/6/07, Chris Mellon wrote: