Due to python3.x 's intentional backwards incompatibility, it is not an easy work to migrate a project's codebase from python2.x to python3.x. Guido has given a recommended development model:

  1. You should have excellent unit tests with close to full coverage.
  2. Port your project to Python2.6.
  3. Turn on the Py3k warnings mode.
  4. Test and edit until no warnings remain.
  5. Use the 2to3 tool to convert this source code to 3.0 syntax.
  6. Test the converted source code under 3.0.
  7. If problems are found, make corrections to the 2.6 version of the source code and go back to step 4.
  8. When it's time to release, release separate 2.6 and 3.0 tarballs.
Fortunately, py.test is already 2.4 and 2.6 compatible, and has sufficient unit tests. So, I can directly start from step 3. However, unfortunately, Holger requires the codebase ported to 3.1 should still be compatible with 2.4 and 2.6, i.e., maintain only one codebase which could be used through all main python versions. So, 2to3 tool can only be used to show the minimum places that need notification.

Thus, to make py.test compatible with python2.4, 2.6, and 3.1, the first thing is to write some wrapper functions. The functions I wrote are:
Print:
--use Print function instead of each print statement in py.test, so python3.1 will not throw SyntaxError. In Print function, execute correct print code according to current python version.
Raise:
--the only incompatible syntax is raise cls, value, tb (2.x) and raise cls(value).with_traceback(tb) (3.x). So, just simply call corresponding statement in Raise function.
isinstancemethod, isclassmethod, isfunction:
--suppose we defined a class "myclass" and a method "method" inside "myclass", then create an instance of myclass, myinstance. In 2.x, myclass.method and myinstance.method have the same attribute names. But in 3.x, myclass.method's attribute names are the same with the ones of normal function. This is a very annoying difference, because in many places code objects are got by obj.im_func.func_code. In 2.x, obj could be myinstance.method or myclass.method; In 3.x, obj.im_func.func_code must be changed to obj.__func__.__code__ and it only works for myinstance.method. For myclass.method or normal function, should use obj.__code__. So, I have to provide three functions to distinguish them respectively.
updatemethodattr, updatefunctionattr:
--method's attributes:
im_self ==> __self__
im_func ==> __func__
im_class ==> disappeared?
--function's attributes:
func_closure ==> __closure__
func_code ==> __code__
func_defaults ==> __defaults__
func_dict ==> __dict__
func_doc ==> __doc__
func_globals ==> __globals__
func_name ==> __name__
So, to keep codebase unchanged, if obj is 3.x's method or function, create 2.x's attributes.
CmpToKey:
--in 3.x, there is no cmp keyword in sort function. So, this wrapper will transform a cmp function to key function.
bytestostr, strtobytes:
--in 3.x, all strings are unicode, but lots of streams require bytes object instead of string. So, use these two functions to wrap stream arguments and when it's 3.x, transfer from bytes to str or str to bytes.

Second, for some well known incompatibilities, such as "except as" and module rename. For "except Error as e:", change it to "except Error:", and in except block, create e by "e = sys.exc_info()[1]". For module rename, add a try/except block. For example:
try: import StringIO
except ImportError: import io as StringIO

Third, 3.x has many incompatible mechanisms. For example: 3.x will not call __cmp__ when compareing two objects. So, __lt__, __gt__, __le__, __ge__, and __eq__ should be implemented. Furthermore, if you define __eq__, __hash__ must be provided.
In 2.x, dict.items() will return a copy list of key-value pairs. But in 3.x, dict.items() will return a view object, and if the dict's size changed during iteration, a RuntimeError will be thrown. So, although "for key,value in dict.items():" still works in 3.x, it should be changed to "for key, value in list(dict.items()):" in case of RuntimeError.

There are still lots of incompatibities need to be fixed. Currently the packages of py.test without test failures under 3.1 are:
builtin/
cmdline/
io/
log/
path/
process/
rest/
I will keep updating this post when porting. Hope this could be finished ASAP.

0 comments

Post a Comment