- Getting Started Developing With PyPy
- Running PyPy’s unit tests
- Special Introspection Features of the Untranslated Python Interpreter
- Additional Tools for running (and hacking) PyPy
- Getting involved
- Where to start reading the sources
The PyPy source tree comes with an inlined version of py.test which you can invoke by typing:
python pytest.py -h
This is usually equivalent to using an installed version:
If you encounter problems with the installed version make sure you have the correct version installed which you can find out with the --version switch.
Now on to running some tests. PyPy has many different test directories and you can use shell completion to point at directories or files:
py.test pypy/interpreter/test/test_pyframe.py # or for running tests of a whole subdirectory py.test pypy/interpreter/
See py.test usage and invocations for some more generic info on how you can run tests.
Beware trying to run “all” pypy tests by pointing to the root directory or even the top level subdirectory pypy. It takes hours and uses huge amounts of RAM and is not recommended.
To run CPython regression tests you can point to the lib-python directory:
This will usually take a long time because this will run the PyPy Python interpreter on top of CPython. On the plus side, it’s usually still faster than doing a full translation and running the regression test with the translated PyPy Python interpreter.
If you are interested in the inner workings of the PyPy Python interpreter, there are some features of the untranslated Python interpreter that allow you to introspect its internals.
To start interpreting Python with PyPy, install a C compiler that is supported by distutils and use Python 2.7 or greater to run PyPy:
cd pypy python bin/pyinteractive.py
After a few seconds (remember: this is running on top of CPython), you should be at the PyPy prompt, which is the same as the Python prompt, but with an extra “>”.
If you press <Ctrl-C> on the console you enter the interpreter-level console, a usual CPython console. You can then access internal objects of PyPy (e.g. the object space) and any variables you have created on the PyPy prompt with the prefix w_:
>>>> a = 123 >>>> <Ctrl-C> *** Entering interpreter-level console *** >>> w_a W_IntObject(123)
The mechanism works in both directions. If you define a variable with the w_ prefix on the interpreter-level, you will see it on the app-level:
>>> w_l = space.newlist([space.wrap(1), space.wrap("abc")]) >>> <Ctrl-D> *** Leaving interpreter-level console *** KeyboardInterrupt >>>> l [1, 'abc']
Note that the prompt of the interpreter-level console is only ‘>>>’ since it runs on CPython level. If you want to return to PyPy, press <Ctrl-D> (under Linux) or <Ctrl-Z>, <Enter> (under Windows).
You may be interested in reading more about the distinction between interpreter-level and app-level.
To list the PyPy interpreter command line options, type:
cd pypy python bin/pyinteractive.py --help
pyinteractive.py supports most of the options that CPython supports too (in addition to a large amount of options that can be used to customize pyinteractive.py). As an example of using PyPy from the command line, you could type:
python pyinteractive.py -c "from test import pystone; pystone.main(10)"
Alternatively, as with regular Python, you can simply give a script name on the command line:
python pyinteractive.py ../../lib-python/2.7/test/pystone.py 10
See our configuration sections for details about what all the commandline options do.
You can use a simple tracing mode to monitor the interpretation of bytecodes. To enable it, set __pytrace__ = 1 on the interactive PyPy console:
>>>> __pytrace__ = 1 Tracing enabled >>>> x = 5 <module>: LOAD_CONST 0 (5) <module>: STORE_NAME 0 (x) <module>: LOAD_CONST 1 (None) <module>: RETURN_VALUE 0 >>>> x <module>: LOAD_NAME 0 (x) <module>: PRINT_EXPR 0 5 <module>: LOAD_CONST 0 (None) <module>: RETURN_VALUE 0 >>>>
The example-interpreter repository contains an example interpreter written using the RPython translation toolchain.
We use some optional tools for developing PyPy. They are not required to run the basic tests or to get an interactive PyPy prompt but they help to understand and debug PyPy especially for the translation process.
graphviz and pygame are both necessary if you want to look at generated flow graphs:
The py.test testing tool drives all our testing needs.
We use the py library for filesystem path manipulations, terminal writing, logging and some other support functionality.
You don’t necessarily need to install these two libraries because we also ship them inlined in the PyPy source tree.
PyPy employs an open development process. You are invited to join our pypy-dev mailing list or look at the other contact possibilities. Usually we give out commit rights fairly liberally, so if you want to do something with PyPy, you can become a committer. We also run frequent coding sprints which are separately announced and often happen around Python conferences such as EuroPython or PyCon. Upcoming events are usually announced on the blog.
PyPy is made from parts that are relatively independent of each other. You should start looking at the part that attracts you most (all paths are relative to the PyPy top level directory). You may look at our directory reference or start off at one of the following points:
- pypy/interpreter contains the bytecode interpreter: bytecode dispatcher in pypy/interpreter/pyopcode.py, frame and code objects in pypy/interpreter/eval.py and pypy/interpreter/pyframe.py, function objects and argument passing in pypy/interpreter/function.py and pypy/interpreter/argument.py, the object space interface definition in pypy/interpreter/baseobjspace.py, modules in pypy/interpreter/module.py and pypy/interpreter/mixedmodule.py. Core types supporting the bytecode interpreter are defined in pypy/interpreter/typedef.py.
- pypy/interpreter/pyparser contains a recursive descent parser, and grammar files that allow it to parse the syntax of various Python versions. Once the grammar has been processed, the parser can be translated by the above machinery into efficient code.
- pypy/interpreter/astcompiler contains the compiler. This contains a modified version of the compiler package from CPython that fixes some bugs and is translatable.
- pypy/objspace/std contains the Standard object space. The main file is pypy/objspace/std/objspace.py. For each type, the files xxxtype.py and xxxobject.py contain respectively the definition of the type and its (default) implementation.