Chapter 3. Native Datatypes¶

You’ll get back to your first Python program in just a minute. But first, a short digression is in order, because you need to know about dictionaries, tuples, and lists (oh my!). If you’re a Perl hacker, you can probably skim the bits about dictionaries and lists, but you should still pay attention to tuples.

3.1. Introducing Dictionaries¶

One of Python’s built-in datatypes is the dictionary, which defines one-to-one relationships between keys and values.

Note: Python vs. Perl: Dictionaries A dictionary in Python is like a hash in Perl. In Perl, variables that store hashes always start with a % character. In Python, variables can be named anything, and Python keeps track of the datatype internally.

Note: Python vs. Java: Dictionaries A dictionary in Python is like an instance of the Hashtable class in Java.

Note: Python vs. Visual Basic: Dictionaries A dictionary in Python is like an instance of the Scripting.Dictionary object in Visual Basic.

3.1.1. Defining Dictionaries¶

Example 3.1. Defining a Dictionary¶

>>> d = {"server":"mpilgrim", "database":"master"} (1)
>>> d
{'server': 'mpilgrim', 'database': 'master'}
>>> d["server"]                                    (2)
'mpilgrim'
>>> d["database"]                                  (3)
'master'
>>> d["mpilgrim"]                                  (4)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
KeyError: mpilgrim

First, you create a new dictionary with two elements and assign it to the variable d. Each element is a key-value pair, and the whole set of elements is enclosed in curly braces.
‘server’ is a key, and its associated value, referenced by d[“server”], is ‘mpilgrim’.
‘database’ is a key, and its associated value, referenced by d[“database”], is ‘master’.
You can get values by key, but you can’t get keys by value. So d[“server”] is ‘mpilgrim’, but d[“mpilgrim”] raises an exception, because ‘mpilgrim’ is not a key.

3.1.2. Modifying Dictionaries¶

Example 3.2. Modifying a Dictionary¶

>>> d
{'server': 'mpilgrim', 'database': 'master'}
>>> d["database"] = "pubs" (1)
>>> d
{'server': 'mpilgrim', 'database': 'pubs'}
>>> d["uid"] = "sa"        (2)
>>> d
{'server': 'mpilgrim', 'uid': 'sa', 'database': 'pubs'}

You can not have duplicate keys in a dictionary. Assigning a value to an existing key will wipe out the old value.
You can add new key-value pairs at any time. This syntax is identical to modifying existing values. (Yes, this will annoy you someday when you think you are adding new values but are actually just modifying the same value over and over because your key isn’t changing the way you think it is.)

Note that the new element (key ‘uid’, value ‘sa’) appears to be in the middle. In fact, it was just a coincidence that the elements appeared to be in order in the first example; it is just as much a coincidence that they appear to be out of order now.

Note: Dictionaries are unordered Dictionaries have no concept of order among elements. It is incorrect to say that the elements are “out of order”; they are simply unordered. This is an important distinction that will annoy you when you want to access the elements of a dictionary in a specific, repeatable order (like alphabetical order by key). There are ways of doing this, but they’re not built into the dictionary.

When working with dictionaries, you need to be aware that dictionary keys are case-sensitive.

Example 3.3. Dictionary Keys Are Case-Sensitive¶

>>> d = {}
>>> d["key"] = "value"
>>> d["key"] = "other value" (1)
>>> d
{'key': 'other value'}
>>> d["Key"] = "third value" (2)
>>> d
{'Key': 'third value', 'key': 'other value'}

Assigning a value to an existing dictionary key simply replaces the old value with a new one.
This is not assigning a value to an existing dictionary key, because strings in Python are case-sensitive, so ‘key’ is not the same as ‘Key’. This creates a new key/value pair in the dictionary; it may look similar to you, but as far as Python is concerned, it’s completely different.

Example 3.4. Mixing Datatypes in a Dictionary¶

>>> d
{'server': 'mpilgrim', 'uid': 'sa', 'database': 'pubs'}
>>> d["retrycount"] = 3 (1)
>>> d
{'server': 'mpilgrim', 'uid': 'sa', 'database': 'master', 'retrycount': 3}
>>> d[42] = "douglas"   (2)
>>> d
{'server': 'mpilgrim', 'uid': 'sa', 'database': 'master',
42: 'douglas', 'retrycount': 3}

Dictionaries aren’t just for strings. Dictionary values can be any datatype, including strings, integers, objects, or even other dictionaries. And within a single dictionary, the values don’t all need to be the same type; you can mix and match as needed.
Dictionary keys are more restricted, but they can be strings, integers, and a few other types. You can also mix and match key datatypes within a dictionary.

3.1.3. Deleting Items From Dictionaries¶

Example 3.5. Deleting Items from a Dictionary¶

>>> d
{'server': 'mpilgrim', 'uid': 'sa', 'database': 'master',
42: 'douglas', 'retrycount': 3}
>>> del d[42] (1)
>>> d
{'server': 'mpilgrim', 'uid': 'sa', 'database': 'master', 'retrycount': 3}
>>> d.clear() (2)
>>> d
{}

del lets you delete individual items from a dictionary by key.
clear deletes all items from a dictionary. Note that the set of empty curly braces signifies a dictionary without any items.

3.2. Introducing Lists¶

Lists are Python’s workhorse datatype. If your only experience with lists is arrays in Visual Basic or (God forbid) the datastore in Powerbuilder, brace yourself for Python lists.

Note: Python vs. Perl: lists A list in Python is like an array in Perl. In Perl, variables that store arrays always start with the @ character; in Python, variables can be named anything, and Python keeps track of the datatype internally.

Note: Python vs. Java: lists A list in Python is much more than an array in Java (although it can be used as one if that’s really all you want out of life). A better analogy would be to the ArrayList class, which can hold arbitrary objects and can expand dynamically as new items are added.

3.2.1. Defining Lists¶

Example 3.6. Defining a List¶

>>> li = ["a", "b", "mpilgrim", "z", "example"] (1)
>>> li
['a', 'b', 'mpilgrim', 'z', 'example']
>>> li[0]                                       (2)
'a'
>>> li[4]                                       (3)
'example'

First, you define a list of five elements. Note that they retain their original order. This is not an accident. A list is an ordered set of elements enclosed in square brackets.
A list can be used like a zero-based array. The first element of any non-empty list is always li[0].
The last element of this five-element list is li[4], because lists are always zero-based.

Example 3.7. Negative List Indices¶

>>> li
['a', 'b', 'mpilgrim', 'z', 'example']
>>> li[-1] (1)
'example'
>>> li[-3] (2)
'mpilgrim'

A negative index accesses elements from the end of the list counting backwards. The last element of any non-empty list is always li[-1].
If the negative index is confusing to you, think of it this way: li[-n] == li[len(li) - n]. So in this list, li[-3] == li[5 - 3] == li[2].

Example 3.8. Slicing a List¶

>>> li
['a', 'b', 'mpilgrim', 'z', 'example']
>>> li[1:3]  (1)
['b', 'mpilgrim']
>>> li[1:-1] (2)
['b', 'mpilgrim', 'z']
>>> li[0:3]  (3)
['a', 'b', 'mpilgrim']

You can get a subset of a list, called a “slice”, by specifying two indices. The return value is a new list containing all the elements of the list, in order, starting with the first slice index (in this case li[1]), up to but not including the second slice index (in this case li[3]).
Slicing works if one or both of the slice indices is negative. If it helps, you can think of it this way: reading the list from left to right, the first slice index specifies the first element you want, and the second slice index specifies the first element you don’t want. The return value is everything in between.
Lists are zero-based, so li[0:3] returns the first three elements of the list, starting at li[0], up to but not including li[3].

Example 3.9. Slicing Shorthand¶

>>> li
['a', 'b', 'mpilgrim', 'z', 'example']
>>> li[:3] (1)
['a', 'b', 'mpilgrim']
>>> li[3:] (2) (3)
['z', 'example']
>>> li[:]  (4)
['a', 'b', 'mpilgrim', 'z', 'example']

If the left slice index is 0, you can leave it out, and 0 is implied. So li [:3] is the same as li[0:3] from Example 3.8, ??Slicing a List??.
Similarly, if the right slice index is the length of the list, you can leave it out. So li[3:] is the same as li[3:5], because this list has five elements.
Note the symmetry here. In this five-element list, li[:3] returns the first 3 elements, and li[3:] returns the last two elements. In fact, li[:n] will always return the first n elements, and li[n:] will return the rest, regardless of the length of the list.
If both slice indices are left out, all elements of the list are included. But this is not the same as the original li list; it is a new list that happens to have all the same elements. li[:] is shorthand for making a complete copy of a list.

3.2.2. Adding Elements to Lists¶

Example 3.10. Adding Elements to a List¶

>>> li
['a', 'b', 'mpilgrim', 'z', 'example']
>>> li.append("new")               (1)
>>> li
['a', 'b', 'mpilgrim', 'z', 'example', 'new']
>>> li.insert(2, "new")            (2)
>>> li
['a', 'b', 'new', 'mpilgrim', 'z', 'example', 'new']
>>> li.extend(["two", "elements"]) (3)
>>> li
['a', 'b', 'new', 'mpilgrim', 'z', 'example', 'new', 'two', 'elements']

append adds a single element to the end of the list.
insert inserts a single element into a list. The numeric argument is the index of the first element that gets bumped out of position. Note that list elements do not need to be unique; there are now two separate elements with the value ‘new’, li[2] and li[6].
extend concatenates lists. Note that you do not call extend with multiple arguments; you call it with one argument, a list. In this case, that list has two elements.

Example 3.11. The Difference between extend and append¶

>>> li = ['a', 'b', 'c']
>>> li.extend(['d', 'e', 'f']) (1)
>>> li
['a', 'b', 'c', 'd', 'e', 'f']
>>> len(li)                    (2)
6
>>> li[-1]
'f'
>>> li = ['a', 'b', 'c']
>>> li.append(['d', 'e', 'f']) (3)
>>> li
['a', 'b', 'c', ['d', 'e', 'f']]
>>> len(li)                    (4)
4
>>> li[-1]
['d', 'e', 'f']

Lists have two methods, extend and append, that look like they do the same thing, but are in fact completely different. extend takes a single argument, which is always a list, and adds each of the elements of that list to the original list.
Here you started with a list of three elements (‘a’, ‘b’, and ‘c’), and you extended the list with a list of another three elements (‘d’, ‘e’, and ‘f’), so you now have a list of six elements.
On the other hand, append takes one argument, which can be any data type, and simply adds it to the end of the list. Here, you’re calling the append method with a single argument, which is a list of three elements.
Now the original list, which started as a list of three elements, contains four elements. Why four? Because the last element that you just appended is itself a list. Lists can contain any type of data, including other lists. That may be what you want, or maybe not. Don’t use append if you mean extend.

3.2.3. Searching Lists¶

Example 3.12. Searching a List¶

>>> li
['a', 'b', 'new', 'mpilgrim', 'z', 'example', 'new', 'two', 'elements']
>>> li.index("example") (1)
5
>>> li.index("new")     (2)
2
>>> li.index("c")       (3)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
ValueError: list.index(x): x not in list
>>> "c" in li           (4)
False

index finds the first occurrence of a value in the list and returns the index.
index finds the first occurrence of a value in the list. In this case, ‘new’ occurs twice in the list, in li[2] and li[6], but index will return only the first index, 2.
If the value is not found in the list, Python raises an exception. This is notably different from most languages, which will return some invalid index. While this may seem annoying, it is a good thing, because it means your program will crash at the source of the problem, rather than later on when you try to use the invalid index.
To test whether a value is in the list, use in, which returns True if the value is found or False if it is not.

Note: What's True in Python?
Before version 2.2.1, Python had no separate boolean datatype. To
compensate for this, Python accepted almost anything in a boolean context
(like an if statement), according to the following rules:
  + 0 is false; all other numbers are true.
  + An empty string ("") is false, all other strings are true.
  + An empty list ([]) is false; all other lists are true.
  + An empty tuple (()) is false; all other tuples are true.
  + An empty dictionary ({}) is false; all other dictionaries are true.
These rules still apply in Python 2.2.1 and beyond, but now you can also
use an actual boolean, which has a value of True or False. Note the
capitalization; these values, like everything else in Python, are
case-sensitive.

3.2.4. Deleting List Elements¶

Example 3.13. Removing Elements from a List¶

>>> li
['a', 'b', 'new', 'mpilgrim', 'z', 'example', 'new', 'two', 'elements']
>>> li.remove("z")   (1)
>>> li
['a', 'b', 'new', 'mpilgrim', 'example', 'new', 'two', 'elements']
>>> li.remove("new") (2)
>>> li
['a', 'b', 'mpilgrim', 'example', 'new', 'two', 'elements']
>>> li.remove("c")   (3)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
ValueError: list.remove(x): x not in list
>>> li.pop()         (4)
'elements'
>>> li
['a', 'b', 'mpilgrim', 'example', 'new', 'two']

remove removes the first occurrence of a value from a list.
remove removes only the first occurrence of a value. In this case, ‘new’ appeared twice in the list, but li.remove(“new”) removed only the first occurrence.
If the value is not found in the list, Python raises an exception. This mirrors the behavior of the index method.
pop is an interesting beast. It does two things: it removes the last element of the list, and it returns the value that it removed. Note that this is different from li[-1], which returns a value but does not change the list, and different from li.remove(value), which changes the list but does not return a value.

3.2.5. Using List Operators¶

Example 3.14. List Operators¶

>>> li = ['a', 'b', 'mpilgrim']
>>> li = li + ['example', 'new'] (1)
>>> li
['a', 'b', 'mpilgrim', 'example', 'new']
>>> li += ['two']                (2)
>>> li
['a', 'b', 'mpilgrim', 'example', 'new', 'two']
>>> li = [1, 2] * 3              (3)
>>> li
[1, 2, 1, 2, 1, 2]

Lists can also be concatenated with the + operator. list = list + otherlist has the same result as list.extend(otherlist). But the + operator returns a new (concatenated) list as a value, whereas extend only alters an existing list. This means that extend is faster, especially for large lists.
Python supports the += operator. li += [‘two’] is equivalent to li.extend ([‘two’]). The += operator works for lists, strings, and integers, and it can be overloaded to work for user-defined classes as well. (More on classes in Chapter 5.)
The * operator works on lists as a repeater. li = [1, 2] * 3 is equivalent to li = [1, 2] + [1, 2] + [1, 2], which concatenates the three lists into one.

3.3. Introducing Tuples¶

A tuple is an immutable list. A tuple can not be changed in any way once it is created.

Example 3.15. Defining a tuple¶

>>> t = ("a", "b", "mpilgrim", "z", "example") (1)
>>> t
('a', 'b', 'mpilgrim', 'z', 'example')
>>> t[0]                                       (2)
'a'
>>> t[-1]                                      (3)
'example'
>>> t[1:3]                                     (4)
('b', 'mpilgrim')

A tuple is defined in the same way as a list, except that the whole set of elements is enclosed in parentheses instead of square brackets.
The elements of a tuple have a defined order, just like a list. Tuples indices are zero-based, just like a list, so the first element of a non-empty tuple is always t[0].
Negative indices count from the end of the tuple, just as with a list.
Slicing works too, just like a list. Note that when you slice a list, you get a new list; when you slice a tuple, you get a new tuple.

Example 3.16. Tuples Have No Methods¶

>>> t
('a', 'b', 'mpilgrim', 'z', 'example')
>>> t.append("new")    (1)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
AttributeError: 'tuple' object has no attribute 'append'
>>> t.remove("z")      (2)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
AttributeError: 'tuple' object has no attribute 'remove'
>>> t.index("example") (3)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
AttributeError: 'tuple' object has no attribute 'index'
>>> "z" in t           (4)
True

You can’t add elements to a tuple. Tuples have no append or extend method.
You can’t remove elements from a tuple. Tuples have no remove or pop method.
You can’t find elements in a tuple. Tuples have no index method.
You can, however, use in to see if an element exists in the tuple.

So what are tuples good for?

Tuples are faster than lists. If you’re defining a constant set of values and all you’re ever going to do with it is iterate through it, use a tuple instead of a list.

It makes your code safer if you “write-protect” data that does not need to be changed. Using a tuple instead of a list is like having an implied assert statement that shows this data is constant, and that special thought (and a specific function) is required to override that.

Remember that I said that dictionary keys can be integers, strings, and ” a few other types”? Tuples are one of those types. Tuples can be used as keys in a dictionary, but lists can’t be used this way.Actually, it’s more complicated than that. Dictionary keys must be immutable. Tuples themselves are immutable, but if you have a tuple of lists, that counts as mutable and isn’t safe to use as a dictionary key. Only tuples of strings, numbers, or other dictionary-safe tuples can be used as dictionary keys.

Tuples are used in string formatting, as you’ll see shortly.

Note: Tuples into lists into tuples Tuples can be converted into lists, and vice-versa. The built-in tuple function takes a list and returns a tuple with the same elements, and the list function takes a tuple and returns a list. In effect, tuple freezes a list, and list thaws a tuple.

3.4. Declaring variables¶

Now that you know something about dictionaries, tuples, and lists (oh my!), let’s get back to the sample program from Chapter 2, odbchelper.py.

Python has local and global variables like most other languages, but it has no explicit variable declarations. Variables spring into existence by being assigned a value, and they are automatically destroyed when they go out of scope.

Example 3.17. Defining the myParams Variable¶

if __name__ == "__main__":
    myParams = {"server":"mpilgrim", \
                "database":"master", \
                "uid":"sa", \
                "pwd":"secret" \
                }

Notice the indentation. An if statement is a code block and needs to be indented just like a function.

Also notice that the variable assignment is one command split over several lines, with a backslash (“”) serving as a line-continuation marker.

Note: Writing Multiline Commands When a command is split among several lines with the line-continuation marker (“”), the continued lines can be indented in any manner; Python’s normally stringent indentation rules do not apply. If your Python IDE auto-indents the continued line, you should probably accept its default unless you have a burning reason not to.

Strictly speaking, expressions in parentheses, straight brackets, or curly braces (like defining a dictionary) can be split into multiple lines with or without the line continuation character (“”). I like to include the backslash even when it’s not required because I think it makes the code easier to read, but that’s a matter of style.

Third, you never declared the variable myParams, you just assigned a value to it. This is like VBScript without the option explicit option. Luckily, unlike VBScript, Python will not allow you to reference a variable that has never been assigned a value; trying to do so will raise an exception.

3.4.1. Referencing Variables¶

Example 3.18. Referencing an Unbound Variable¶

>>> x
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
NameError: There is no variable named 'x'
>>> x = 1
>>> x
1

You will thank Python for this one day.

3.4.2. Assigning Multiple Values at Once¶

One of the cooler programming shortcuts in Python is using sequences to assign multiple values at once.

Example 3.19. Assigning multiple values at once¶

>>> v = ('a', 'b', 'e')
>>> (x, y, z) = v     (1)
>>> x
'a'
>>> y
'b'
>>> z
'e'

v is a tuple of three elements, and (x, y, z) is a tuple of three variables. Assigning one to the other assigns each of the values of v to each of the variables, in order.

This has all sorts of uses. I often want to assign names to a range of values. In C, you would use enum and manually list each constant and its associated value, which seems especially tedious when the values are consecutive. In Python, you can use the built-in range function with multi-variable assignment to quickly assign consecutive values.

Example 3.20. Assigning Consecutive Values¶

>>> range(7)                                                                    (1)
[0, 1, 2, 3, 4, 5, 6]
>>> (MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY) = range(7) (2)
>>> MONDAY                                                                      (3)
0
>>> TUESDAY
1
>>> SUNDAY
6

The built-in range function returns a list of integers. In its simplest form, it takes an upper limit and returns a zero-based list counting up to but not including the upper limit. (If you like, you can pass other parameters to specify a base other than 0 and a step other than 1. You can print range.__doc__ for details.)
MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, and SUNDAY are the variables you’re defining. (This example came from the calendar module, a fun little module that prints calendars, like the UNIX program cal. The calendar module defines integer constants for days of the week.)
Now each variable has its value: MONDAY is 0, TUESDAY is 1, and so forth.

You can also use multi-variable assignment to build functions that return multiple values, simply by returning a tuple of all the values. The caller can treat it as a tuple, or assign the values to individual variables. Many standard Python libraries do this, including the os module, which you’ll discuss in Chapter 6.

3.5. Formatting Strings¶

Python supports formatting values into strings. Although this can include very complicated expressions, the most basic usage is to insert values into a string with the %s placeholder.

Note: Python vs. C: String Formatting String formatting in Python uses the same syntax as the sprintf function in C.

Example 3.21. Introducing String Formatting¶

>>> k = "uid"
>>> v = "sa"
>>> "%s=%s" % (k, v) (1)
'uid=sa'

The whole expression evaluates to a string. The first %s is replaced by the value of k; the second %s is replaced by the value of v. All other characters in the string (in this case, the equal sign) stay as they are.

Note that (k, v) is a tuple. I told you they were good for something.

You might be thinking that this is a lot of work just to do simple string concatentation, and you would be right, except that string formatting isn’t just concatenation. It’s not even just formatting. It’s also type coercion.

Example 3.22. String Formatting vs. Concatenating¶

>>> uid = "sa"
>>> pwd = "secret"
>>> print pwd + " is not a good password for " + uid      (1)
secret is not a good password for sa
>>> print "%s is not a good password for %s" % (pwd, uid) (2)
secret is not a good password for sa
>>> userCount = 6
>>> print "Users connected: %d" % (userCount, )           (3) (4)
Users connected: 6
>>> print "Users connected: " + userCount                 (5)
Traceback (innermost last):
  File "<interactive input>", line 1, in ?
TypeError: cannot concatenate 'str' and 'int' objects

- is the string concatenation operator.
In this trivial case, string formatting accomplishes the same result as concatentation.
(userCount, ) is a tuple with one element. Yes, the syntax is a little strange, but there’s a good reason for it: it’s unambiguously a tuple. In fact, you can always include a comma after the last element when defining a list, tuple, or dictionary, but the comma is required when defining a tuple with one element. If the comma weren’t required, Python wouldn’t know whether (userCount) was a tuple with one element or just the value of userCount.
String formatting works with integers by specifying %d instead of %s.
Trying to concatenate a string with a non-string raises an exception. Unlike string formatting, string concatenation works only when everything is already a string.

As with printf in C, string formatting in Python is like a Swiss Army knife. There are options galore, and modifier strings to specially format many different types of values.

Example 3.23. Formatting Numbers¶

>>> print "Today's stock price: %f" % 50.4625   (1)
50.462500
>>> print "Today's stock price: %.2f" % 50.4625 (2)
50.46
>>> print "Change since yesterday: %+.2f" % 1.5 (3)
+1.50

The %f string formatting option treats the value as a decimal, and prints it to six decimal places.
The “.2” modifier of the %f option truncates the value to two decimal places.
You can even combine modifiers. Adding the + modifier displays a plus or minus sign before the value. Note that the “.2” modifier is still in place, and is padding the value to exactly two decimal places.

3.6. Mapping Lists¶

One of the most powerful features of Python is the list comprehension, which provides a compact way of mapping a list into another list by applying a function to each of the elements of the list.

Example 3.24. Introducing List Comprehensions¶

>>> li = [1, 9, 8, 4]
>>> [elem*2 for elem in li]      (1)
[2, 18, 16, 8]
>>> li                           (2)
[1, 9, 8, 4]
>>> li = [elem*2 for elem in li] (3)
>>> li
[2, 18, 16, 8]

To make sense of this, look at it from right to left. li is the list you’re mapping. Python loops through li one element at a time, temporarily assigning the value of each element to the variable elem. Python then applies the function elem*2 and appends that result to the returned list.
Note that list comprehensions do not change the original list.
It is safe to assign the result of a list comprehension to the variable that you’re mapping. Python constructs the new list in memory, and when the list comprehension is complete, it assigns the result to the variable.

Here are the list comprehensions in the buildConnectionString function that you declared in Chapter 2: [“%s=%s” % (k, v) for k, v in params.items()]

First, notice that you’re calling the items function of the params dictionary. This function returns a list of tuples of all the data in the dictionary.

Example 3.25. The keys, values, and items Functions¶

>>> params = {"server":"mpilgrim", "database":"master", "uid":"sa", "pwd":"secret"}
>>> params.keys()   (1)
['server', 'uid', 'database', 'pwd']
>>> params.values() (2)
['mpilgrim', 'sa', 'master', 'secret']
>>> params.items()  (3)
[('server', 'mpilgrim'), ('uid', 'sa'), ('database', 'master'), ('pwd', 'secret')]

The keys method of a dictionary returns a list of all the keys. The list is not in the order in which the dictionary was defined (remember that elements in a dictionary are unordered), but it is a list.
The values method returns a list of all the values. The list is in the same order as the list returned by keys, so params.values()[n] == params [params.keys()[n]] for all values of n.
The items method returns a list of tuples of the form (key, value). The list contains all the data in the dictionary.

Now let’s see what buildConnectionString does. It takes a list, params.items(), and maps it to a new list by applying string formatting to each element. The new list will have the same number of elements as params.items(), but each element in the new list will be a string that contains both a key and its associated value from the params dictionary.

Example 3.26. List Comprehensions in buildConnectionString, Step by Step¶

>>> params = {"server":"mpilgrim", "database":"master", "uid":"sa", "pwd":"secret"}
>>> params.items()
[('server', 'mpilgrim'), ('uid', 'sa'), ('database', 'master'), ('pwd', 'secret')]
>>> [k for k, v in params.items()]                (1)
['server', 'uid', 'database', 'pwd']
>>> [v for k, v in params.items()]                (2)
['mpilgrim', 'sa', 'master', 'secret']
>>> ["%s=%s" % (k, v) for k, v in params.items()] (3)
['server=mpilgrim', 'uid=sa', 'database=master', 'pwd=secret']

Note that you’re using two variables to iterate through the params.items() list. This is another use of multi-variable assignment. The first element of params.items() is (‘server’, ‘mpilgrim’), so in the first iteration of the list comprehension, k will get ‘server’ and v will get ‘mpilgrim’. In this case, you’re ignoring the value of v and only including the value of k in the returned list, so this list comprehension ends up being equivalent to params.keys().
Here you’re doing the same thing, but ignoring the value of k, so this list comprehension ends up being equivalent to params.values().
Combining the previous two examples with some simple string formatting, you get a list of strings that include both the key and value of each element of the dictionary. This looks suspiciously like the output of the program. All that remains is to join the elements in this list into a single string.

3.7. Joining Lists and Splitting Strings¶

You have a list of key-value pairs in the form key=value, and you want to join them into a single string. To join any list of strings into a single string, use the join method of a string object.

Here is an example of joining a list from the buildConnectionString function:: return “;”.join([“%s=%s” % (k, v) for k, v in params.items()])

One interesting note before you continue. I keep repeating that functions are objects, strings are objects... everything is an object. You might have thought I meant that string variables are objects. But no, look closely at this example and you’ll see that the string “;” itself is an object, and you are calling its join method.

The join method joins the elements of the list into a single string, with each element separated by a semi-colon. The delimiter doesn’t need to be a semi-colon; it doesn’t even need to be a single character. It can be any string.

Caution: You Can’t join Non-Strings join works only on lists of strings; it does not do any type coercion. Joining a list that has one or more non-string elements will raise an exception.

Example 3.27. Output of odbchelper.py¶

>>> params = {"server":"mpilgrim", "database":"master", "uid":"sa", "pwd":"secret"}
>>> ["%s=%s" % (k, v) for k, v in params.items()]
['server=mpilgrim', 'uid=sa', 'database=master', 'pwd=secret']
>>> ";".join(["%s=%s" % (k, v) for k, v in params.items()])
'server=mpilgrim;uid=sa;database=master;pwd=secret'

This string is then returned from the odbchelper function and printed by the calling block, which gives you the output that you marveled at when you started reading this chapter.

You’re probably wondering if there’s an analogous method to split a string into a list. And of course there is, and it’s called split.

Example 3.28. Splitting a String¶

>>> li = ['server=mpilgrim', 'uid=sa', 'database=master', 'pwd=secret']
>>> s = ";".join(li)
>>> s
'server=mpilgrim;uid=sa;database=master;pwd=secret'
>>> s.split(";")    (1)
['server=mpilgrim', 'uid=sa', 'database=master', 'pwd=secret']
>>> s.split(";", 1) (2)
['server=mpilgrim', 'uid=sa;database=master;pwd=secret']

split reverses join by splitting a string into a multi-element list. Note that the delimiter (“;”) is stripped out completely; it does not appear in any of the elements of the returned list.
split takes an optional second argument, which is the number of times to split. (“”Oooooh, optional arguments...” You’ll learn how to do this in your own functions in the next chapter.)

Tip: Searching with split anystring.split(delimiter, 1) is a useful technique when you want to search a string for a substring and then work with everything before the substring (which ends up in the first element of the returned list) and everything after it (which ends up in the second element).

3.7.1. Historical Note on String Methods¶

When I first learned Python, I expected join to be a method of a list, which would take the delimiter as an argument. Many people feel the same way, and there’s a story behind the join method. Prior to Python 1.6, strings didn’t have all these useful methods. There was a separate string module that contained all the string functions; each function took a string as its first argument. The functions were deemed important enough to put onto the strings themselves, which made sense for functions like lower, upper, and split. But many hard-core Python programmers objected to the new join method, arguing that it should be a method of the list instead, or that it shouldn’t move at all but simply stay a part of the old string module (which still has a lot of useful stuff in it). I use the new join method exclusively, but you will see code written either way, and if it really bothers you, you can use the old string.join function instead.

3.8. Summary¶

The odbchelper.py program and its output should now make perfect sense.

def buildConnectionString(params):
    """Build a connection string from a dictionary of parameters.

    Returns string."""
    return ";".join(["%s=%s" % (k, v) for k, v in params.items()])

if __name__ == "__main__":
    myParams = {"server":"mpilgrim", \
                "database":"master", \
                "uid":"sa", \
                "pwd":"secret" \
                }
    print buildConnectionString(myParams)

Here is the output of odbchelper.py:

server=mpilgrim;uid=sa;database=master;pwd=secret

Before diving into the next chapter, make sure you’re comfortable doing all of

these things:

Using the Python IDE to test expressions interactively

Writing Python programs and running them from within your IDE, or from the command line

Importing modules and calling their functions

Declaring functions and using doc strings, local variables, and proper indentation

Defining dictionaries, tuples, and lists

Accessing attributes and methods of any object, including strings, lists, dictionaries, functions, and modules

Concatenating values through string formatting

Mapping lists into other lists using list comprehensions

Splitting strings into lists and joining lists into strings

Navigation