Thanks to the Object Space architecture, any feature that is based on proxying, extending, changing or otherwise controlling the behavior of objects in a running program is easy to implement on top of PyPy.
Here is what we have implemented so far, in historical order:
- Dump Object Space: dumps all operations performed on all the objects into a large log file. For debugging your applications.
- Transparent Proxies Extension: adds new proxy objects to the Standard Object Space that enable applications to control operations on application and builtin objects, e.g lists, dictionaries, tracebacks.
This is a feature that was tried experimentally long ago, and we found no really good use cases. The basic functionality is still there, but we don’t recommend using it. Some of the examples below might not work any more (e.g. you can’t tproxy a list object any more). The rest can be done by hacking in standard Python. If anyone is interested in working on tproxy again, he is welcome, but we don’t regard this as an interesting extension.
PyPy’s Transparent Proxies allow routing of operations on objects
to a callable. Application-level code can customize objects without
interfering with the type system -
type(proxied_list) is list holds true
proxied_list is a proxied built-in
list - while
giving you full control on all operations that are performed on the
See [D12.1] for more context, motivation and usage of transparent proxies.
The following example proxies a list and will return
42 on any addition
$ py.py --objspace-std-withtproxy >>>> from __pypy__ import tproxy >>>> def f(operation, *args, **kwargs): >>>> if operation == '__add__': >>>> return 42 >>>> raise AttributeError >>>> >>>> i = tproxy(list, f) >>>> type(i) list >>>> i + 3 42
Suppose we want to have a list which stores all operations performed on it for later analysis. We can use the lib_pypy/tputil.py module to help with transparently proxying builtin instances:
from tputil import make_proxy history =  def recorder(operation): history.append(operation) return operation.delegate() >>>> l = make_proxy(recorder, obj=) >>>> type(l) list >>>> l.append(3) >>>> len(l) 1 >>>> len(history) 2
make_proxy(recorder, obj=) creates a transparent list
proxy that allows us to delegate operations to the
type(l) does not lead to any operation being executed at all.
append() shows up as
__getattribute__() and that
type(l) does not show up at all - the type is the only aspect of the instance
which the proxy controller cannot change.
Returns a proxy object representing the given type and forwarding all operations on this type to the controller. On each operation,
controller(opname, *args, **kwargs)will be called.
Returns the responsible controller for a given object. For non-proxied objects
The lib_pypy/tputil.py module provides:
make_proxy(controller, type, obj)¶
Creates a transparent proxy controlled by the given
controllercallable. The proxy will appear as a completely regular instance of the given type, but all operations on it are sent to the specified controller - which receives a
ProxyOperationinstance on each operation. If
typeis not specified, it defaults to
ProxyOperation instances have the following attributes:
The transparent proxy object of this operation.
The name of this operation.
Any positional arguments for this operation.
Any keyword arguments for this operation.
A lot of tasks could be performed using transparent proxies, including, but not limited to:
- Remote versions of objects, on which we can directly perform operations (think about transparent distribution)
- Access to persistent storage such as a database (imagine an SQL object mapper which looks like any other object).
- Access to external data structures, such as other languages, as normal objects (of course some operations could raise exceptions, but since operations are executed at the application level, that is not a major problem)
PyPy’s standard object space allows us to internally have multiple implementations of a type and change the implementation at run-time, while application-level code consistently sees the exact same type and object. Multiple performance optimizations using these features have already been implemented: alternative object implementations. Transparent Proxies use this architecture to provide control back to application-level code.
Transparent proxies are implemented on top of the standard object
space, in pypy/objspace/std/proxy_helpers.py,
pypy/objspace/std/proxyobject.py and pypy/objspace/std/transparent.py.
To use them you will need to pass a –objspace-std-withtproxy option to
translate.py. This registers implementations named
- which usually correspond to an appropriate
W_XxxObject - and
includes some interpreter hacks for objects that are too close to the interpreter
to be implemented in the standard object space. The types of objects that can
be proxied this way are user created classes & functions,
lists, dicts, exceptions, tracebacks and frames.
|[D12.1]||High-Level Backends and Interpreter Feature Prototypes, PyPy EU-Report, 2007, https://bitbucket.org/pypy/extradoc/raw/tip/eu-report/D12.1_H-L-Backends_and_Feature_Prototypes-2007-03-22.pdf|