Spartan Sniper

Currently only some tests of execnet/ and test/ failed. The 2nd kind of failures in previous post can be bypassed using locals() dictionary. It seems that python3.1 intendedly suppress the variables generated by exec() can only be shown in locals() dictionary.

Current Status:
tests: 169 failed, 1283 passed.
passed folders: builtin/, code/, cmdline/, io/, log/, misc/, path/, process/, rest/, thread/, tool/, xmlobj/.

Currently, for all tests of py.test, 1277 passed, 183 failed under python3.1. There are mainly 3 kinds of failures:
1. magic/exprinfo.py relies on the compiler package which is no longer supported by python3.x. exprinfo.py is used to construct assertion exception information, and in python3.1 I just simply disable exprinfo.py feature. So, some tests failed because of assertion exception's message does not match the expected string.
2. code/source.py provides a substitute API for compile function, but this API seems does not work well under python3.1. I think the reason is that the compile function's mechanism seems changed. For example:

def foo():
  co = compile("x=3", '', 'exec')
  exec(co)
  print("x:", x)

foo()

The output is: NameError: global name 'x' is not defined. However, this program can be run successfully under python2.x.
3. execnet/ folder aims to support distributed testing feature, and uses io stream tunnel heavily. But, in python3.1, some io streams require bytes not strings. the differences between byte and string are really annoying. So, some tests of py.test failed due to the messages cannot be transported from the sender to the receiver.

Currently passed folders:
builtin/
cmdline/
io/
log/
misc/
path/
process/
rest/
thread/
tool/
xmlobj/

Due to python3.x 's intentional backwards incompatibility, it is not an easy work to migrate a project's codebase from python2.x to python3.x. Guido has given a recommended development model:

1. You should have excellent unit tests with close to full coverage.
2. Port your project to Python2.6.
3. Turn on the Py3k warnings mode.
4. Test and edit until no warnings remain.
5. Use the 2to3 tool to convert this source code to 3.0 syntax.
6. Test the converted source code under 3.0.
7. If problems are found, make corrections to the 2.6 version of the source code and go back to step 4.
8. When it's time to release, release separate 2.6 and 3.0 tarballs.

Fortunately, py.test is already 2.4 and 2.6 compatible, and has sufficient unit tests. So, I can directly start from step 3. However, unfortunately, Holger requires the codebase ported to 3.1 should still be compatible with 2.4 and 2.6, i.e., maintain only one codebase which could be used through all main python versions. So, 2to3 tool can only be used to show the minimum places that need notification.

Thus, to make py.test compatible with python2.4, 2.6, and 3.1, the first thing is to write some wrapper functions. The functions I wrote are:
Print:
--use Print function instead of each print statement in py.test, so python3.1 will not throw SyntaxError. In Print function, execute correct print code according to current python version.
Raise:
--the only incompatible syntax is raise cls, value, tb (2.x) and raise cls(value).with_traceback(tb) (3.x). So, just simply call corresponding statement in Raise function.
isinstancemethod, isclassmethod, isfunction:
--suppose we defined a class "myclass" and a method "method" inside "myclass", then create an instance of myclass, myinstance. In 2.x, myclass.method and myinstance.method have the same attribute names. But in 3.x, myclass.method's attribute names are the same with the ones of normal function. This is a very annoying difference, because in many places code objects are got by obj.im_func.func_code. In 2.x, obj could be myinstance.method or myclass.method; In 3.x, obj.im_func.func_code must be changed to obj.__func__.__code__ and it only works for myinstance.method. For myclass.method or normal function, should use obj.__code__. So, I have to provide three functions to distinguish them respectively.
updatemethodattr, updatefunctionattr:
--method's attributes:
im_self ==> __self__
im_func ==> __func__
im_class ==> disappeared?
--function's attributes:
func_closure ==> __closure__
func_code ==> __code__
func_defaults ==> __defaults__
func_dict ==> __dict__
func_doc ==> __doc__
func_globals ==> __globals__
func_name ==> __name__
So, to keep codebase unchanged, if obj is 3.x's method or function, create 2.x's attributes.
CmpToKey:
--in 3.x, there is no cmp keyword in sort function. So, this wrapper will transform a cmp function to key function.
bytestostr, strtobytes:
--in 3.x, all strings are unicode, but lots of streams require bytes object instead of string. So, use these two functions to wrap stream arguments and when it's 3.x, transfer from bytes to str or str to bytes.

Second, for some well known incompatibilities, such as "except as" and module rename. For "except Error as e:", change it to "except Error:", and in except block, create e by "e = sys.exc_info()[1]". For module rename, add a try/except block. For example:
try: import StringIO
except ImportError: import io as StringIO

Third, 3.x has many incompatible mechanisms. For example: 3.x will not call __cmp__ when compareing two objects. So, __lt__, __gt__, __le__, __ge__, and __eq__ should be implemented. Furthermore, if you define __eq__, __hash__ must be provided.
In 2.x, dict.items() will return a copy list of key-value pairs. But in 3.x, dict.items() will return a view object, and if the dict's size changed during iteration, a RuntimeError will be thrown. So, although "for key,value in dict.items():" still works in 3.x, it should be changed to "for key, value in list(dict.items()):" in case of RuntimeError.

There are still lots of incompatibities need to be fixed. Currently the packages of py.test without test failures under 3.1 are:
builtin/
cmdline/
io/
log/
path/
process/
rest/
I will keep updating this post when porting. Hope this could be finished ASAP.

1. self contained
The test case can be run either in isolation or in arbitrary combination with any number of other test cases.

2. In unittest of Python, If setUp() succeeded, the tearDown() method will be run whether runTest() succeeded or not.

3. In unittest of Python, there are some ways to create test suite.

--------------------
suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
--------------------
def suite():
    suite = unittest.TestSuite()
    suite.addTest(WidgetTestCase('testDefaultSize'))
    suite.addTest(WidgetTestCase('testResize'))
    return suite
--------------------
def suite():
    tests = ['testDefaultSize', 'testResize']
    return unittest.TestSuite(map(WidgetTestCase, tests))

4. To use old test without converting every old test function to a TestCase subclass, unittest provides a FunctionTestCase class. This subclass of TestCase can be used to wrap an existing test function. Set-up and tear-down functions can also be provided.

def testSomething():
    something = makeSomething()
    assert something.name is not None
    # ...

testcase = unittest.FunctionTestCase(testSomething)
or
testcase = unittest.FunctionTestCase(testSomething,
setUp=makeSomethingDB,
tearDown=deleteSomethingDB)

5. TestCase.setUp()
This is called immediately before calling the test method; any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing.

6. The methods of TestCase used by the test implementation to check conditions and report failures can be get here.

7. ad-hoc means that there are no installation requirements whatsoever on the remote side. (from here)

8. By default, py.test catches text written to stdout/stderr during the execution of each individual test. This output will only be displayed however if the test fails, each failing test that produced output during the running of the test will have its output displayed in the recorded stdout section.

9. Disabling a test class
If you want to disable a complete test class you can set the class-level attribute disabled. For example, in order to avoid running some tests on Win32:

class TestPosixOnly:
    disabled = sys.platform == 'win32'
    def test_xxx(self):
        ...

It is wonderful for me to know that my application was accepted by Google Summer of Code program (GSoC). This summer I will work for improving py.test compatibility with several versions of Python, especially 3k. Besides, better-formatted output of regression tests and code coverage will be shown regularly on Snakebite. This project, accepted into Python Software Foundation (PSF), was inspired by Titus Brown, and will be mentored by Holger Krekel. Titus and Holger have given me many precious advices and suggestions, which are greatly appreciated.

There is no need to keep my proposal from public view, for it is against the spirit of open source. Thus, I paste it as following, and all comments, advices, or suggestions are welcome.

----GSoC Application Proposal----

ABOUT ME
My name is Yang Yang, a Ph.D. student in Computer Science at Michigan State University. As a teaching assistant of a Python course for nearly two years, I found that Python is such an amazing programming language that freshman can learn programming concepts effectively without being trapped in lower level details, meanwhile it is very powerful and can be used in almost everywhere. I am using Python almost every day, ranging from my research work to courses projects. Compared with C++, Python could save me lots of time in programming, thanks to its fewer alternatives and elegant coding style.

I still remember the time when we were developing a P2P-VoD application using C++. We have to write our own trace function and every one took charge of testing his/her own modules. Lacking of testing experience, we had to spend excessive time on debugging. Thus, knowing that py.test is aiming to provide a powerful testing framework, I am eager to improve this tool to make it usable for Core Python3k developers. So, developers can run tests without thought and more concentrate on their own modules.

This project will be my main work in this summer, and I am able to be in contact with mentors and community. Furthermore, finishing this project does not mean I already reach the destination. I will still keep in contact within PSF community and keep making my contribution.

CONTACT INFO
email: yangyan5 AT msu DOT edu
yangyan5 AT gmail DOT com
cell: 517-614-6078
IRC nick: yangyang

TITLE
Improve Unit Testing Framework for Core Python3k

ABSTRACT
In this project, I will make py.test compatible to Core Python3k test files and introduce some features to py.test, such that Python developers can run unit testing easier. Besides, regression test suite will be run over multiple platforms regularly on Snakebite, integrated with build-bot or other facilities.

SUMMARY
Unit testing is very important in software development and Python has already provided a unit testing framework unittest. Although unittest supports some features such as test automation and collection, tests still cannot be run without thought, which means that we still need to register the tests we want to run. Thus, some unit testing framework arises, such as py.test and nose. py.test is already used in PyPy and it has many excellent features, such as automatically collecting and executing tests, selecting/unselecting tests by keyword, and distributing tests to multiple CPUs or machines. Similarly, nose provides a number of excellent features as well. For example, nose has the ability to run tests based on specific tags, i.e. tests can be run selectively without manually generating test suite. Besides, nose can transparently wrap tests with code coverage recording. However, the prerequisite of using these features of py.test and nose is that the tests must be discoverable and executable, which cannot be guaranteed in Core Python. Besides, py.test is still not compatible with Python3k. Thus, I plan to introduce several features that make py.test and nose compatible and convenient to run the hundreds of test files in Core Python.

My effort will mainly focus on Python3k and be seven-fold. The first feature I'd like to contribute is to make py.test compatible with Python3k as well as Python2.4, which will be the basis of my later work. Second, I will provide a py.test plug-in for Python3k, Python2.6, and Python2.5 that could "understand" the conditions under which regrtest.py, the current test runner, finds and runs tests. Third, as Jython is becoming more and more popular, I plan to port the plug-in to make py.test could be used in Jython. Fourth, I will set up regular test runs for Core Python3k on Snakebite such that regression test suite across multiple platforms can be recorded and posted. And, for daily run reporting on Snakebite, I will work on integrating with build-bot or other facilities. Fifth, to help developers test their code over different version of Python, I will check the latest features of unittest.py and doctest.py of Python3k and make them compatible with previous versions of Python. Sixth, aiming to increase the test coverage in Python3k, I plan to write some test cases and report current test coverage for python modules in stdlib and list achieved improvements. Although py.test has pytest_figleaf which offers coverage report, the report is raw and needs configuration to make it useful and meaningful for developers. Last but not least, to collaborate with nose, a plug-in for py.test will be provided to invoke and run nose-style tests, which means to make py.test could discover and execute nose-style tests.

Through all of my work, py.test will be made compatible with multiple versions of CPython and Jython when running regression test suite, without modifying tests too much. In addition, the regression tests and code coverage result over multiple platforms will be shown on Snakebite.

SCHEDULE
April 20 - May 22: Get to know mentors and community. Dive into the details of py.test, regrtest.py, and nose.
May 23 - June 7: Port py.test to Python3k, keeping compatible with Python2.4 as well.. (including debugging)
June 8 - June 21: Write a Plug-in to make py.test compatible with regrtest of Python3k, Python2.6, and Python2.5. (including debugging)
June 22 - June 28: Port py.test and plug-in to Jython. (including debugging)
June 29 - July 5: Port latest features of Python3k to previous versions. (including debugging)
July 6 - July 15: Set up regular regression test runs on Snakebite. (keeping in touch with python-dev)
July 16 - July 26: Write test cases, report current test coverage and list achieved improvements. (keeping in touch with python-dev)
July 27 - August 2: Write a plug-in for py.test to invoke/run nose-style tests.
August 3 - August 10: Documentation

DELIVERABLES
1. Port py.test to Python3k, keeping compatible with Python2.4 as well.
2. A plug-in to make py.test compatible with regrtest of Python3k, Python2.6, and Python2.5.
3. Port py.test and plug-in to Jython.
4. Port latest features of Python3k to previous versions.
5. Regression test suite regular report on Snakebite, integrated with build-bot or other facilities.
6. Report current test coverage and list achieved improvements. The report will be developer-friendly.
7. A plug-in for py.test to invoke/run nose-style tests.

BENEFITS TO COMMUNITY
1. Make unit testing on Core Python3k and other Python versions easier by allowing developers to use py.test with some convenient features.
2. Give developers a clear view on regression test and code coverage result over multiple platforms.
-------End of Proposal--------

The exciting and busy summer is coming!

Mercurial is a distributed source version control system. Compared with subversion, Mercurial is much more scalable, and everyone can commit his/her own effort. It is becoming more and more popular, and Python has switched to Mercurial recently. Here is a tutorial written by Bryan O'Sullivan and all notes below are extracted from it.

1. log
$ hg log -r [-v] [-p]
-v: verbose mode
-p: show content of a change

2. status
$ hg status
Show what Mercurial knows about the files in repository.

3. incoming, pull, update
We use pull to get the changesets from "remote" repository. But, to avoid blindly fetching, we should use incoming first to give us a clear view of what will be transferred.
$ hg incoming remote_repository
$ hg pull remote_repository
Because pull only bring changes into repository, we should use update to get a working copy. So, we should run:
$ hg update

4. outgoing, push, update
outgoing command will tell us what changes would be pushed into another repository. push command does the actual push. Then use update to get a working copy.
$ hg outgoing remote_repository
$ hg push remote_repository
$ hg update

5. heads, parents
We can view the heads in a repository using the heads command.
$ hg heads
Note: heads is different with parents. parents command is used to find out what revision the working directory is at.
$ hg parents

6. Removing a file does not affect its history. It has only two effects:

• It removes the current version of the file from the working directory.
• It stops Mercurial from tracking changes to the file, from the time of the next commit.

7. Mercurial considers a file that you have deleted, but not used hg remove to delete, to be missing. If you deleted the missing file by accident, give hg revert the name of the file to recover. It will reappear, in unmodified form.

8. Mercurial offers a combination command, hg addremove, that adds untracked files and marks missing files as removed.

Holger Krekel shared with me an interesting post which talks about some typical prototypes of programmers. It is so true that I found those characteristics in myself and teammates very often. For example, I thought it embarrassing me to let others see my iterative work before I reach a milestone. And the result is that I have no time to refactor my code and have to continue my work based on the fragile and awkward "code bomb". Thanks to that post, I am trying to put the advices into practice.