Spartan Sniper

It is wonderful for me to know that my application was accepted by Google Summer of Code program (GSoC). This summer I will work for improving py.test compatibility with several versions of Python, especially 3k. Besides, better-formatted output of regression tests and code coverage will be shown regularly on Snakebite. This project, accepted into Python Software Foundation (PSF), was inspired by Titus Brown, and will be mentored by Holger Krekel. Titus and Holger have given me many precious advices and suggestions, which are greatly appreciated.

There is no need to keep my proposal from public view, for it is against the spirit of open source. Thus, I paste it as following, and all comments, advices, or suggestions are welcome.

----GSoC Application Proposal----

ABOUT ME
My name is Yang Yang, a Ph.D. student in Computer Science at Michigan State University. As a teaching assistant of a Python course for nearly two years, I found that Python is such an amazing programming language that freshman can learn programming concepts effectively without being trapped in lower level details, meanwhile it is very powerful and can be used in almost everywhere. I am using Python almost every day, ranging from my research work to courses projects. Compared with C++, Python could save me lots of time in programming, thanks to its fewer alternatives and elegant coding style.

I still remember the time when we were developing a P2P-VoD application using C++. We have to write our own trace function and every one took charge of testing his/her own modules. Lacking of testing experience, we had to spend excessive time on debugging. Thus, knowing that py.test is aiming to provide a powerful testing framework, I am eager to improve this tool to make it usable for Core Python3k developers. So, developers can run tests without thought and more concentrate on their own modules.

This project will be my main work in this summer, and I am able to be in contact with mentors and community. Furthermore, finishing this project does not mean I already reach the destination. I will still keep in contact within PSF community and keep making my contribution.

CONTACT INFO
email: yangyan5 AT msu DOT edu
yangyan5 AT gmail DOT com
cell: 517-614-6078
IRC nick: yangyang

TITLE
Improve Unit Testing Framework for Core Python3k

ABSTRACT
In this project, I will make py.test compatible to Core Python3k test files and introduce some features to py.test, such that Python developers can run unit testing easier. Besides, regression test suite will be run over multiple platforms regularly on Snakebite, integrated with build-bot or other facilities.

SUMMARY
Unit testing is very important in software development and Python has already provided a unit testing framework unittest. Although unittest supports some features such as test automation and collection, tests still cannot be run without thought, which means that we still need to register the tests we want to run. Thus, some unit testing framework arises, such as py.test and nose. py.test is already used in PyPy and it has many excellent features, such as automatically collecting and executing tests, selecting/unselecting tests by keyword, and distributing tests to multiple CPUs or machines. Similarly, nose provides a number of excellent features as well. For example, nose has the ability to run tests based on specific tags, i.e. tests can be run selectively without manually generating test suite. Besides, nose can transparently wrap tests with code coverage recording. However, the prerequisite of using these features of py.test and nose is that the tests must be discoverable and executable, which cannot be guaranteed in Core Python. Besides, py.test is still not compatible with Python3k. Thus, I plan to introduce several features that make py.test and nose compatible and convenient to run the hundreds of test files in Core Python.

My effort will mainly focus on Python3k and be seven-fold. The first feature I'd like to contribute is to make py.test compatible with Python3k as well as Python2.4, which will be the basis of my later work. Second, I will provide a py.test plug-in for Python3k, Python2.6, and Python2.5 that could "understand" the conditions under which regrtest.py, the current test runner, finds and runs tests. Third, as Jython is becoming more and more popular, I plan to port the plug-in to make py.test could be used in Jython. Fourth, I will set up regular test runs for Core Python3k on Snakebite such that regression test suite across multiple platforms can be recorded and posted. And, for daily run reporting on Snakebite, I will work on integrating with build-bot or other facilities. Fifth, to help developers test their code over different version of Python, I will check the latest features of unittest.py and doctest.py of Python3k and make them compatible with previous versions of Python. Sixth, aiming to increase the test coverage in Python3k, I plan to write some test cases and report current test coverage for python modules in stdlib and list achieved improvements. Although py.test has pytest_figleaf which offers coverage report, the report is raw and needs configuration to make it useful and meaningful for developers. Last but not least, to collaborate with nose, a plug-in for py.test will be provided to invoke and run nose-style tests, which means to make py.test could discover and execute nose-style tests.

Through all of my work, py.test will be made compatible with multiple versions of CPython and Jython when running regression test suite, without modifying tests too much. In addition, the regression tests and code coverage result over multiple platforms will be shown on Snakebite.

SCHEDULE
April 20 - May 22: Get to know mentors and community. Dive into the details of py.test, regrtest.py, and nose.
May 23 - June 7: Port py.test to Python3k, keeping compatible with Python2.4 as well.. (including debugging)
June 8 - June 21: Write a Plug-in to make py.test compatible with regrtest of Python3k, Python2.6, and Python2.5. (including debugging)
June 22 - June 28: Port py.test and plug-in to Jython. (including debugging)
June 29 - July 5: Port latest features of Python3k to previous versions. (including debugging)
July 6 - July 15: Set up regular regression test runs on Snakebite. (keeping in touch with python-dev)
July 16 - July 26: Write test cases, report current test coverage and list achieved improvements. (keeping in touch with python-dev)
July 27 - August 2: Write a plug-in for py.test to invoke/run nose-style tests.
August 3 - August 10: Documentation

DELIVERABLES
1. Port py.test to Python3k, keeping compatible with Python2.4 as well.
2. A plug-in to make py.test compatible with regrtest of Python3k, Python2.6, and Python2.5.
3. Port py.test and plug-in to Jython.
4. Port latest features of Python3k to previous versions.
5. Regression test suite regular report on Snakebite, integrated with build-bot or other facilities.
6. Report current test coverage and list achieved improvements. The report will be developer-friendly.
7. A plug-in for py.test to invoke/run nose-style tests.

BENEFITS TO COMMUNITY
1. Make unit testing on Core Python3k and other Python versions easier by allowing developers to use py.test with some convenient features.
2. Give developers a clear view on regression test and code coverage result over multiple platforms.
-------End of Proposal--------

The exciting and busy summer is coming!