Using a newline character as a join character

Maybe you remember those days starting to code in a new
language when you overcomplicate things because they
look similar to the correct way to do it in the
previous language? (sprint in C) Anyway thanks for
again showing me how easy it is to do things in Python.

This app is for testing a glucose monitor.

--vicki

Uh, I don't entirely follow what you're doing, but

here

are some thoughts:

- os.path.join() is for file system paths.
- I doubt you're doing file system path stuff since

you

have "\n" as a component.
- You can put strings together like so:
  new = old + '\n' + returnedVal
Or directly as an arg:
  self.outputbox.SetLabel(old + '\n' + returnedVal)
- Stylistically, I recommend eval(s) instead of `s`.
eval() fits nicer with all the other cool functions

and

···

statements like dir(), hasattr(), exec, etc.

So what are you building? :slight_smile:

--
Chuck
http://ChuckEsterbrook.com

I don't seem to have gotten Chuck's comment in this thread myself, so I'm replying to this reply to it...

- You can put strings together like so:
new = old + '\n' + returnedVal
Or directly as an arg:
self.outputbox.SetLabel(old + '\n' + returnedVal)

Strings can indeed be added this way, but beware that repeated addition of strings tends to be slow and inefficient. Each addition requires creation of a new string object and copying of the contents of the old strings to the new locations. As the number and size of string objects grow, this can cause a significant bottleneck. It won't make much difference for a couple of additions on strings of a few hundred bytes. But consider that you're adding on to this string each time that your handler is called, meaning that the label can grow significantly over time.

Typically, when working with strings, I prefer to use string methods like join() or string formatting operations with % instead of addition. These have much better growth curves as the number and size of strings increase, and while they may not make much difference in this particular case, I feel that it's better to be in the habit of using them so that it's automatic in those cases where it *will* make a difference. For example, the code above can also be written as:

        new = '%s\n%s' % (old, returnedVal)

- Stylistically, I recommend eval(s) instead of `s`.
eval() fits nicer with all the other cool functions
   

and

statements like dir(), hasattr(), exec, etc.

First, backticks (`s`) are not equivalent to eval(), they're equivalent to repr(). I also prefer to avoid using the backticks, because the function usage is much less cryptic.

More importantly, though, I strongly recommend *not* using eval() in almost any context. This is a risky, black-magic function, and it can have all sorts of surprising side-effects. It's especially bad if you're eval()-ing a string that's created by a user -- what happens when that string contains "os.system('rm -s /')"?? -- but even if you're not, it opens the door for all manner of unexpected things. Using eval() is like playing with a loaded firearm. There are cases where eval() will do a job that no other tool will do, but those cases are quite rare, and it's best not to risk using it unless you really do have one of those cases. I can't recall a single time that I've seen eval() used, that I couldn't think of some other way to achieve the same ends that was cleaner and safer.

Jeff Shannon
Technician/Programmer
Credit International

I don't seem to have gotten Chuck's comment in this thread myself, so
I'm replying to this reply to it...

>>- You can put strings together like so:
>> new = old + '\n' + returnedVal
>>Or directly as an arg:
>> self.outputbox.SetLabel(old + '\n' + returnedVal)

Strings can indeed be added this way, but beware that repeated
addition of strings tends to be slow and inefficient. Each addition
requires creation of a new string object and copying of the contents
of the old strings to the new locations. As the number and size of
string objects grow, this can cause a significant bottleneck. It
won't make much difference for a couple of additions on strings of a
few hundred bytes. But consider that you're adding on to this string
each time that your handler is called, meaning that the label can
grow significantly over time.

Typically, when working with strings, I prefer to use string methods
like join() or string formatting operations with % instead of
addition. These have much better growth curves as the number and size
of strings increase, and while they may not make much difference in
this particular case, I feel that it's better to be in the habit of
using them so that it's automatic in those cases where it *will* make
a difference. For example, the code above can also be written as:

        new = '%s\n%s' % (old, returnedVal)

I think your intentions are good and your general point is sound, but
possibly applied to the wrong case.

I'm think that the event handler could be faster with:
     self.old = self.old + '\n' + new

Than with:
     self.lines.append(new)
     '\n'.join(self.lines)

The second case gets slower as len(lines) grows which is bad karma. :slight_smile:
The first case is always just two operations. Of course, the string
*is* getting bigger, but copying a single growing string is probably
faster than traversing a growing list.

I *do* use 'someString'.join() quite a bit, but the above case is
different because the lines keep growing and the join() is asked for
every single time they do.

Also, in some previous testing I found that simple formatted strings are
slower than adding them for the obvious reason that formatted strings
have to be interpreted.

But my most important point is that for most apps it doesn't matter.
Better to get your app features built and stable, then worry about
performance if you have to!

>>- Stylistically, I recommend eval(s) instead of `s`.
>>eval() fits nicer with all the other cool functions
>
>and
>
>>statements like dir(), hasattr(), exec, etc.

First, backticks (`s`) are not equivalent to eval(), they're
equivalent to repr(). I also prefer to avoid using the backticks,
because the function usage is much less cryptic.

Not on my box. :slight_smile:

$ python
Python 2.2.2 (#1, Nov 26 2002, 16:14:37)
[GCC 3.2 (Mandrake Linux 9.0 3.2-1mdk)] on linux2

print `3+4`

7

That's clearly an interpretation of "3" and "4" as ints and "+" as an
operator to execute. I used to print to avoid the implicit repr() call
that the interpreter makes, which would have just been confusing for
this discussion.

print repr('3+4')

'3+4'

repr() returns strings as-is. No interpretation.

More importantly, though, I strongly recommend *not* using eval() in
almost any context. This is a risky, black-magic function, and it
can have all sorts of surprising side-effects. It's especially bad
if you're eval()-ing a string that's created by a user -- what
happens when that string contains "os.system('rm -s /')"?? -- but
even if you're not, it opens the door for all manner of unexpected
things. Using eval() is like playing with a loaded firearm. There
are cases where eval() will do a job that no other tool will do, but
those cases are quite rare, and it's best not to risk using it unless
you really do have one of those cases. I can't recall a single time
that I've seen eval() used, that I couldn't think of some other way
to achieve the same ends that was cleaner and safer.

I agree to some extent. For example, int(), str(), float() and such are
more appropriately for converting values from one type to another. My
most common use of eval() is for sucking in a dictionary config file:

# my.config
{
  'numThis': 5,
  'numThat': 10,
  'filename': '/some/path',
}

You might find something safer, but you probably won't find something
"cleaner" than this:
  eval(open('my.config').read())

···

On Wednesday 30 April 2003 10:52 am, Jeff Shannon wrote:

--
Chuck
http://ChuckEsterbrook.com

Chuck Esterbrook wrote:

$ python
Python 2.2.2 (#1, Nov 26 2002, 16:14:37)
[GCC 3.2 (Mandrake Linux 9.0 3.2-1mdk)] on linux2

print `3+4`

7

That's clearly an interpretation of "3" and "4" as ints and "+" as an operator to execute. I used to print to avoid the implicit repr() call that the interpreter makes, which would have just been confusing for this discussion.

print repr('3+4')

'3+4'

repr() returns strings as-is. No interpretation.

But the first case is not a string. `3+4` is the same as repr(3+4)

···

--
Robin Dunn
Software Craftsman
http://wxPython.org Java give you jitters? Relax with wxPython!

Hmmm, I guess I thought of `3+4` as a string since it looks like a
string.

I guess whether `3+4` means eval('3+4') or repr(3+4) *does* have a
technical answer under the hood, although they feel equivalent from an
outside perspective. At least, I couldn't think of an example (short of
dumping bytecode) where `foo`, eval('foo') and repr(foo) would result
in differences.

I'll take your word for it.

It just goes to show that backticks are evil. :slight_smile:

···

On Wednesday 30 April 2003 02:08 pm, Robin Dunn wrote:

Chuck Esterbrook wrote:
> $ python
> Python 2.2.2 (#1, Nov 26 2002, 16:14:37)
> [GCC 3.2 (Mandrake Linux 9.0 3.2-1mdk)] on linux2
>
>>>>print `3+4`
>
> 7
>
> That's clearly an interpretation of "3" and "4" as ints and "+" as
> an operator to execute. I used to print to avoid the implicit
> repr() call that the interpreter makes, which would have just been
> confusing for this discussion.
>
>>>>print repr('3+4')
>
> '3+4'
>
> repr() returns strings as-is. No interpretation.

But the first case is not a string. `3+4` is the same as repr(3+4)

--
Chuck
http://ChuckEsterbrook.com

Chuck Esterbrook wrote:

Typically, when working with strings, I prefer to use string methods
like join() or string formatting operations with % instead of
addition. [...]

I think your intentions are good and your general point is sound, but possibly applied to the wrong case.

[...]
But my most important point is that for most apps it doesn't matter. Better to get your app features built and stable, then worry about performance if you have to!

As I pointed out, in this particular case the difference is probably negligable either way. However, I've seen too many people try to use something like:

result = ""
for each in mylist:
    result = result + '\n' + each

to allow this sort of string addition to pass without comment. In cases where an entire list is being processed, or where a series of additions can easily be converted to list.append()s, using join() is usually more efficient, sometimes by large orders of magnitude. In cases where addition is more efficient, it's usually by a pretty small amount. Similarly for string formatting, when combining two or three variables addition may be slightly more efficient, but when combining half a dozen or more variables then string formatting can be a very big win. On the average of all of these cases, addition is less efficient than the alternatives, therefore I consider using addition in cases where it might be faster to be one of those premature optimizations. :wink:

- Stylistically, I recommend eval(s) instead of `s`.
eval() fits nicer with all the other cool functions
       

and

statements like dir(), hasattr(), exec, etc.
       

First, backticks (`s`) are not equivalent to eval(), they're
equivalent to repr(). I also prefer to avoid using the backticks,
because the function usage is much less cryptic.
   
Not on my box. :slight_smile:

$ python
Python 2.2.2 (#1, Nov 26 2002, 16:14:37)
[GCC 3.2 (Mandrake Linux 9.0 3.2-1mdk)] on linux2

print `3+4`
       

7

That's clearly an interpretation of "3" and "4" as ints and "+" as an operator to execute. I used to print to avoid the implicit repr() call that the interpreter makes, which would have just been confusing for this discussion.

print repr('3+4')
       

'3+4'

repr() returns strings as-is. No interpretation.

Ah, but you're not repr()'ing the same thing that you printed.

>>> print repr(3+4)
7
>>> print `'3+4'`
'3+4'
>>>

In your first case, you're using backticks on the result of an arithmetic expression (3+4), in the second case you're using it on a string that represents an arithmetic expression ('3+4'). These are not at all the same thing, and that's why you're getting different results.

Using an intermediate variable makes this even clearer:

>>> spam = 3+4
>>> print `spam`
7
>>> print repr(spam)
7
>>> spam = '3+4'
>>> print `spam`
'3+4'
>>> print repr(spam)
'3+4'
>>>

In addition, the equivalence of backticks and repr() is commented on in the documentation. :wink:

I can't recall a single time
that I've seen eval() used, that I couldn't think of some other way
to achieve the same ends that was cleaner and safer.
   
I agree to some extent. For example, int(), str(), float() and such are more appropriately for converting values from one type to another. My most common use of eval() is for sucking in a dictionary config file:

# my.config
{
'numThis': 5,
'numThat': 10,
'filename': '/some/path',
}

You might find something safer, but you probably won't find something "cleaner" than this:
eval(open('my.config').read())

I would argue that you're perhaps confusing "clean" with "concise", and these are not necessarily the same thing, nor is conciseness always a virtue. To quote from Python Zen ('import this'), "Explicit is better than implicit" and "Readability counts". I'd really rather see this:

# ---- my.config
numThis 5
numThat 10
filename /some/path
# note that those are tabs separating key from value

configdict = {}
for line in file('my.config').readlines():
    key, value = line.split('\t')
    configdict[key] = value.strip()

This lets me know, without examining the config file, exactly what form the data ends up in -- I can see that I'm creating a dictionary. In your version, I have to look into the my.config file to determine what I can do with the result of that one-liner -- is it a dict? a string? a list? an entire module? Not only that, but the my.config file is easier to maintain, since it doesn't require that the sysadmin who alters it understand Python syntax, and a misplaced quote won't throw the entire file off. I also don't have to worry about the safety issues involved in the possibility of someone replacing the config file with something malicious (config files typically being considered lower-security than program files, and thus more likely for an arbitrary user to gain permissions to modify).

Actually, I'd *really* rather see the whole thing done using the standard module ConfigParser, or wxConfig in a wxPython app, but that's another story.

Jeff Shannon
Technician/Programmer
Credit International

···

On Wednesday 30 April 2003 10:52 am, Jeff Shannon wrote:

I would argue that you're perhaps confusing "clean" with "concise",
and these are not necessarily the same thing, nor is conciseness
always a virtue. To quote from Python Zen ('import this'),
"Explicit is better than implicit" and "Readability counts". I'd
really rather see this:

As a side note, "Explicit is better than implicit" is a silly maxim and
I wish the Python community would drop it. Consider this simple
expression:
  a.b

That's *a lot* of implicit behavior. In fact, it's TONS of implicit
behavior. Yet no one in the Python community is crying for it to be
removed or redesigned, which begs the question: Why not? My answer is:

    "Implicit is great when it fits your brain."

Garbage collection is another good example. Lots of implicit behavior
and MUCH better than explicit memory management! It fits our brains, so
we prefer it over explicit.

# ---- my.config
numThis 5
numThat 10
filename /some/path
# note that those are tabs separating key from value

configdict = {}
for line in file('my.config').readlines():
    key, value = line.split('\t')
    configdict[key] = value.strip()

This lets me know, without examining the config file, exactly what
form the data ends up in -- I can see that I'm creating a dictionary.

It also completely prevents you from having nested structures, which I
often find useful in config files. And it prevents you from some minor
logic like having an "isProduction" flag which in turn is used by other
settings ("showTracebacks = not isProduction").

Recently, I've been doing my configs as modules and just treating them
like objects:
  config.isProduction
  config.showTracebacks

And now users can say:

isProduction = 1
showTracebacks = 1

which is easy for just about everyone (after all Python is a CP4E
language).

I guess there are pros and cons with each config approach which is not
surprising. Recently my overall preference is to use simple modules.

···

On Wednesday 30 April 2003 02:51 pm, Jeff Shannon wrote:

--
Chuck
http://ChuckEsterbrook.com

Chuck Esterbrook wrote:

Hmmm, I guess I thought of `3+4` as a string since it looks like a
string.

`3+4~ is not a string, but it does evaluate to a string, by definition.

I guess whether `3+4` means eval('3+4') or repr(3+4) *does* have a
technical answer under the hood

You guess??? They are completely different.

eval(string) means: evaluate this string as a python expression, and
return the results.

`expression` means: return a string representation of the enclosed
expression

repr(expression) means: return a string representation of the enclosed
expression (same as above), so that, in general: eval( repr(x) ) == x

str(expression) means: return a string representation of the enclosed
expression in a form that looks pleasing to people.

For example:

3+4 returns an integer
`3+4` returns a string representation of an integer
eval("3+4") returns an integer (same as 3+4)
repr(3+4) returns an integer

To see the difference between str() and repr()

repr(1.1)

'1.1000000000000001'

str(1.1)

'1.1'

and for completeness

`1.1`

'1.1000000000000001'

And now we've gotten pretty far off topic.

-Chris

···

--
Christopher Barker, Ph.D.
Oceanographer
                                        
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@noaa.gov

Chuck Esterbrook wrote:

I would argue that you're perhaps confusing "clean" with "concise",
and these are not necessarily the same thing, nor is conciseness
always a virtue. To quote from Python Zen ('import this'), "Explicit is better than implicit" and "Readability counts". I'd
really rather see this:
   
As a side note, "Explicit is better than implicit" is a silly maxim and I wish the Python community would drop it. Consider this simple expression:
a.b

That's *a lot* of implicit behavior. In fact, it's TONS of implicit behavior. Yet no one in the Python community is crying for it to be removed or redesigned, which begs the question: Why not? My answer is:

   "Implicit is great when it fits your brain."

I will agree that there's a balance between explicitness and implicitness. They key is that things should be explicit enough that it's readily apparent at quick glance what's happening. The Python interpreter refuses to guess at ambiguous cases; ideally, a person reading a python program shouldn't need to guess either. The reason that the Python community emphasizes explicitness so much is because so many programmers seem so fond of implicitness, and of sweeping (possibly important) details under the rug.

In the general usage, your example "a.b" is actually not all *that* much implicit behavior. It indicates a variable lookup for the name 'a', then calling getattr(a, b) -- that's all. There's a mechanism for objects to be able to hook into getattr(), using the special __getattr__() method, and this provides an opportunity for implicit behavior -- but it's an *opportunity*, and while sometimes such things are needed, usually it's not a good idea. If I set up a __getattr__() that changes an object's state every time it's called, this is typically a very bad implicit behavior (simply looking at an object changes it). Users of this object are unlikely to expect this, and it will result in confusion and program bugs. It's much better to have the object only change state explicitly, when someone calls a method that's designed to changed state -- it's typically assumed that methods represent actions and data attributes do not. (I say "typically" because there may be some reasonable justification in some cases -- for example, it may be necessary to log all attribute lookups -- but these cases are relatively rare and are pretty obviously special cases.) *That* sort of reasoning is where the maxim applies most strongly, and why it is (and will remain) one of the most important parts of Python Zen. (Your argument, on the other hand, embodies one of the *other* most important parts of Python Zen, "Practicality beats purity" -- nobody denies that there's a balance to be found.)

Jeff Shannon
Technician/Programmer
Credit International

···

On Wednesday 30 April 2003 02:51 pm, Jeff Shannon wrote:

Well said.

BTW You forgot __getattribute__ and properties as part of your
explanation. There *is* a lot to know about "a.b", but it's easy to use
and usually works like you would expect.

···

On Wednesday 30 April 2003 03:46 pm, Jeff Shannon wrote:

Chuck Esterbrook wrote:
>On Wednesday 30 April 2003 02:51 pm, Jeff Shannon wrote:
>>I would argue that you're perhaps confusing "clean" with "concise",
>>and these are not necessarily the same thing, nor is conciseness
>>always a virtue. To quote from Python Zen ('import this'),
>>"Explicit is better than implicit" and "Readability counts". I'd
>>really rather see this:
>
>As a side note, "Explicit is better than implicit" is a silly maxim
> and I wish the Python community would drop it. Consider this simple
> expression:
> a.b
>
>That's *a lot* of implicit behavior. In fact, it's TONS of implicit
>behavior. Yet no one in the Python community is crying for it to be
>removed or redesigned, which begs the question: Why not? My answer
> is:
>
> "Implicit is great when it fits your brain."

I will agree that there's a balance between explicitness and
implicitness. They key is that things should be explicit enough that
it's readily apparent at quick glance what's happening. The Python
interpreter refuses to guess at ambiguous cases; ideally, a person
reading a python program shouldn't need to guess either. The reason
that the Python community emphasizes explicitness so much is because
so many programmers seem so fond of implicitness, and of sweeping
(possibly important) details under the rug.

In the general usage, your example "a.b" is actually not all *that*
much implicit behavior. It indicates a variable lookup for the name
'a', then calling getattr(a, b) -- that's all. There's a mechanism
for objects to be able to hook into getattr(), using the special
__getattr__() method, and this provides an opportunity for implicit
behavior -- but it's an *opportunity*, and while sometimes such
things are needed, usually it's not a good idea. If I set up a
__getattr__() that changes an object's state every time it's called,
this is typically a very bad implicit behavior (simply looking at an
object changes it). Users of this object are unlikely to expect this,
and it will result in confusion and program bugs. It's much better
to have the object only change state explicitly, when someone calls a
method that's designed to changed state -- it's typically assumed
that methods represent actions and data attributes do not. (I say
"typically" because there may be some reasonable justification in
some cases -- for example, it may be necessary to log all attribute
lookups -- but these cases are relatively rare and are pretty
obviously special cases.) *That* sort of reasoning is where the
maxim applies most strongly, and why it is (and will remain) one of
the most important parts of Python Zen. (Your argument, on the other
hand, embodies one of the *other* most important parts of Python Zen,
"Practicality beats purity" -- nobody denies that there's a balance
to be found.)

--
Chuck
http://ChuckEsterbrook.com