The Object Space¶
Contents
Introduction¶
The object space creates all objects in PyPy, and knows how to perform operations on them. It may be helpful to think of an object space as being a library offering a fixed API: a set of operations, along with implementations that correspond to the known semantics of Python objects.
For example, add()
is an operation, with implementations in the object
space that perform numeric addition (when add()
is operating on numbers),
concatenation (when add()
is operating on sequences), and so on.
We have some working object spaces which can be plugged into the bytecode interpreter:
- The Standard Object Space is a complete implementation
of the various built-in types and objects of Python. The Standard Object
Space, together with the bytecode interpreter, is the foundation of our Python
implementation. Internally, it is a set of interpreter-level classes
implementing the various application-level objects – integers, strings,
lists, types, etc. To draw a comparison with CPython, the Standard Object
Space provides the equivalent of the C structures
PyIntObject
,PyListObject
, etc. - various Object Space proxies wrap another object space (e.g. the standard one) and adds new capabilities, like lazily computed objects (computed only when an operation is performed on them), security-checking objects, distributed objects living on several machines, etc.
The various object spaces documented here can be found in pypy/objspace.
Note that most object-space operations take and return application-level objects, which are treated as opaque “black boxes” by the interpreter. Only a very few operations allow the bytecode interpreter to gain some knowledge about the value of an application-level object.
Object Space Interface¶
This is the public API that all Object Spaces implement:
Administrative Functions¶
-
getexecutioncontext
()¶ Return the currently active execution context. (pypy/interpreter/executioncontext.py).
-
getbuiltinmodule
(name)¶ Return a
Module
object for the built-in module given byname
. (pypy/interpreter/module.py).
Operations on Objects in the Object Space¶
These functions both take and return “wrapped” (i.e. application-level) objects.
The following functions implement operations with straightforward semantics that directly correspond to language-level constructs:
id, type, issubtype, iter, next, repr, str, len, hash,
getattr, setattr, delattr, getitem, setitem, delitem,
pos, neg, abs, invert, add, sub, mul, truediv, floordiv, div, mod, divmod, pow, lshift, rshift, and_, or_, xor,
nonzero, hex, oct, int, float, long, ord,
lt, le, eq, ne, gt, ge, cmp, coerce, contains,
inplace_add, inplace_sub, inplace_mul, inplace_truediv, inplace_floordiv, inplace_div, inplace_mod, inplace_pow, inplace_lshift, inplace_rshift, inplace_and, inplace_or, inplace_xor,
get, set, delete, userdel
-
call
(w_callable, w_args, w_kwds)¶ Calls a function with the given positional (
w_args
) and keyword (w_kwds
) arguments.
-
index
(w_obj)¶ Implements index lookup (as introduced in CPython 2.5) using
w_obj
. Will return a wrapped integer or long, or raise aTypeError
if the object doesn’t have an__index__()
special method.
-
is_
(w_x, w_y)¶ Implements
w_x is w_y
.
-
isinstance
(w_obj, w_type)¶ Implements
issubtype()
withtype(w_obj)
andw_type
as arguments.
Convenience Functions¶
The following functions are used so often that it was worthwhile to introduce them as shortcuts – however, they are not strictly necessary since they can be expressed using several other object space methods.
-
eq_w
(w_obj1, w_obj2)¶ Returns
True
whenw_obj1
andw_obj2
are equal. Shortcut forspace.is_true(space.eq(w_obj1, w_obj2))
.
-
is_w
(w_obj1, w_obj2)¶ Shortcut for
space.is_true(space.is_(w_obj1, w_obj2))
.
-
hash_w
(w_obj)¶ Shortcut for
space.int_w(space.hash(w_obj))
.
-
len_w
(w_obj)¶ Shortcut for
space.int_w(space.len(w_obj))
.
NOTE that the above four functions return interpreter-level objects, not application-level ones!
-
not_
(w_obj)¶ Shortcut for
space.newbool(not space.is_true(w_obj))
.
-
finditem
(w_obj, w_key)¶ Equivalent to
getitem(w_obj, w_key)
but returns an interpreter-level None instead of raising a KeyError if the key is not found.
-
call_function
(w_callable, *args_w, **kw_w)¶ Collects the arguments in a wrapped tuple and dict and invokes
space.call(w_callable, ...)
.
-
call_method
(w_object, 'method', ...)¶ Uses
space.getattr()
to get the method object, and thenspace.call_function()
to invoke it.
-
unpackiterable
(w_iterable[, expected_length=-1])¶ Iterates over
w_x
(usingspace.iter()
andspace.next()
) and collects the resulting wrapped objects in a list. Ifexpected_length
is given and the length does not match, raises an exception.Of course, in cases where iterating directly is better than collecting the elements in a list first, you should use
space.iter()
andspace.next()
directly.
-
unpacktuple
(w_tuple[, expected_length=None])¶ Equivalent to
unpackiterable()
, but only for tuples.
-
callable
(w_obj)¶ Implements the built-in
callable()
.
Creation of Application Level objects¶
-
wrap
(x)¶ Deprecated! Eventually this method should disappear. Returns a wrapped object that is a reference to the interpreter-level object
x
. This can be used either on simple immutable objects (integers, strings, etc) to create a new wrapped object, or on instances ofW_Root
to obtain an application-level-visible reference to them. For example, most classes of the bytecode interpreter subclassW_Root
and can be directly exposed to application-level code in this way - functions, frames, code objects, etc.
-
newint
(i)¶ Creates a wrapped object holding an integral value. newint creates an object of type W_IntObject.
-
newlong
(l)¶ Creates a wrapped object holding an integral value. The main difference to newint is the type of the argument (which is rpython.rlib.rbigint.rbigint). On PyPy3 this method will return an
int
(PyPy2 it returns along
).
-
newbytes
(t)¶ The given argument is a rpython bytestring. Creates a wrapped object of type
bytes
(both on PyPy2 and PyPy3).
-
newtext
(t)¶ The given argument is a rpython bytestring. Creates a wrapped object of type
str
. On PyPy3 this will return a wrapped unicode object. The object will hold a utf-8-nosg decoded value of t. The “utf-8-nosg” codec used here is slightly different from the “utf-8” implemented in Python 2 or Python 3: it is defined as utf-8 without any special handling of surrogate characters. They are encoded using the same three-bytes sequence that encodes any char in the range from'\u0800'
to'\uffff'
.PyPy2 will return a bytestring object. No encoding/decoding steps will be applied.
-
newbool
(b)¶ Creates a wrapped
bool
object from an interpreter-level object.
-
newtuple
([w_x, w_y, w_z, ...])¶ Creates a new wrapped tuple out of an interpreter-level list of wrapped objects.
-
newlist
([..])¶ Creates a wrapped
list
from an interpreter-level list of wrapped objects.
-
newdict
()¶ Returns a new empty dictionary.
-
newslice
(w_start, w_end, w_step)¶ Creates a new slice object.
-
newunicode
(ustr)¶ Creates a Unicode string from an rpython unicode string. This method may disappear soon and be replaced by :py:function::newutf8.
-
newutf8
(bytestr)¶ Creates a Unicode string from an rpython byte string, decoded as “utf-8-nosg”. On PyPy3 it is the same as :py:function::newtext.
Many more space operations can be found in pypy/interpreter/baseobjspace.py and pypy/objspace/std/objspace.py.
Conversions from Application Level to Interpreter Level¶
-
unwrap
(w_x)¶ Returns the interpreter-level equivalent of
w_x
– use this ONLY for testing, because this method is not RPython and thus cannot be translated! In most circumstances you should use the functions described below instead.
-
is_true
(w_x)¶ Returns a interpreter-level boolean (
True
orFalse
) that gives the truth value of the wrapped objectw_x
.This is a particularly important operation because it is necessary to implement, for example, if-statements in the language (or rather, to be pedantic, to implement the conditional-branching bytecodes into which if-statements are compiled).
-
int_w
(w_x)¶ If
w_x
is an application-level integer or long which can be converted without overflow to an integer, return an interpreter-level integer. Otherwise raiseTypeError
orOverflowError
.
-
bigint_w
(w_x)¶ If
w_x
is an application-level integer or long, return an interpreter-levelrbigint
. Otherwise raiseTypeError
.
-
ObjSpace.
bytes_w
(w_x)¶ Takes an application level
bytes
(on PyPy2 this equals str) and returns a rpython byte string.
-
ObjSpace.
text_w
(w_x)¶ PyPy2 takes either a
str
and returns a rpython byte string, or it takes anunicode
and uses the systems default encoding to return a rpython byte string.On PyPy3 it takes a
str
and it will return an utf-8 encoded rpython string.
-
str_w
(w_x)¶ Deprecated. use text_w or bytes_w instead If
w_x
is an application-level string, return an interpreter-level string. Otherwise raiseTypeError
.
-
unicode_w
(w_x)¶ Takes an application level :py:class::unicode and return an interpreter-level unicode string. This method may disappear soon and be replaced by :py:function::text_w.
-
float_w
(w_x)¶ If
w_x
is an application-level float, integer or long, return an interpreter-level float. Otherwise raiseTypeError
(or:py:exc:OverflowError in the case of very large longs).
-
getindex_w
(w_obj[, w_exception=None])¶ Call
index(w_obj)
. If the resulting integer or long object can be converted to an interpreter-levelint
, return that. If not, return a clamped result ifw_exception
is None, otherwise raise the exception at the application level.(If
w_obj
can’t be converted to an index,index()
will raise an application-levelTypeError
.)
-
interp_w
(RequiredClass, w_x[, can_be_None=False])¶ If
w_x
is a wrapped instance of the given bytecode interpreter class, unwrap it and return it. Ifcan_be_None
isTrue
, a wrappedNone
is also accepted and returns an interpreter-levelNone
. Otherwise, raises anOperationError
encapsulating aTypeError
with a nice error message.
-
interpclass_w
(w_x)¶ If
w_x
is a wrapped instance of an bytecode interpreter class – for exampleFunction
,Frame
,Cell
, etc. – return it unwrapped. Otherwise returnNone
.
Data Members¶
-
space.
builtin
¶ The
Module
containing the builtins.
-
space.
sys
¶ The
sys
Module
.
-
space.
w_None
¶ The ObjSpace’s instance of
None
.
-
space.
w_True
¶ The ObjSpace’s instance of
True
.
-
space.
w_False
¶ The ObjSpace’s instance of
False
.
-
space.
w_Ellipsis
¶ The ObjSpace’s instance of
Ellipsis
.
-
space.
w_NotImplemented
¶ The ObjSpace’s instance of
NotImplemented
.
-
space.
w_int
¶ -
space.
w_float
¶ -
space.
w_long
¶ -
space.
w_tuple
¶ -
space.
w_str
¶ -
space.
w_unicode
¶ -
space.
w_type
¶ -
space.
w_instance
¶ -
space.
w_slice
¶ Python’s most common basic type objects.
-
space.w_[XYZ]Error
Python’s built-in exception classes (
KeyError
,IndexError
, etc).
-
ObjSpace.
MethodTable
¶ List of tuples containing
(method_name, symbol, number_of_arguments, list_of_special_names)
for the regular part of the interface.NOTE that tuples are interpreter-level.
-
ObjSpace.
BuiltinModuleTable
¶ List of names of built-in modules.
-
ObjSpace.
ConstantTable
¶ List of names of the constants that the object space should define.
-
ObjSpace.
ExceptionTable
¶ List of names of exception classes.
-
ObjSpace.
IrregularOpTable
¶ List of names of methods that have an irregular API (take and/or return non-wrapped objects).
The Standard Object Space¶
Introduction¶
The Standard Object Space (pypy/objspace/std/) is the direct equivalent
of CPython’s object library (the Objects/
subdirectory in the distribution).
It is an implementation of the common Python types in a lower-level language.
The Standard Object Space defines an abstract parent class, W_Object
as well as subclasses like W_IntObject
, W_ListObject
,
and so on. A wrapped object (a “black box” for the bytecode interpreter’s main
loop) is an instance of one of these classes. When the main loop invokes an
operation (such as addition), between two wrapped objects w1
and
w2
, the Standard Object Space does some internal dispatching (similar
to Object/abstract.c
in CPython) and invokes a method of the proper
W_XYZObject
class that can perform the operation.
The operation itself is done with the primitives allowed by RPython, and the
result is constructed as a wrapped object. For example, compare the following
implementation of integer addition with the function int_add()
in
Object/intobject.c
:
def add__Int_Int(space, w_int1, w_int2):
x = w_int1.intval
y = w_int2.intval
try:
z = ovfcheck(x + y)
except OverflowError:
raise FailedToImplementArgs(space.w_OverflowError,
space.wrap("integer addition"))
return W_IntObject(space, z)
This may seem like a lot of work just for integer objects (why wrap them into
W_IntObject
instances instead of using plain integers?), but the
code is kept simple and readable by wrapping all objects (from simple integers
to more complex types) in the same way.
(Interestingly, the obvious optimization above has actually been made in PyPy, but isn’t hard-coded at this level – see Standard Interpreter Optimizations.)
Object types¶
The larger part of the pypy/objspace/std/ package defines and
implements the library of Python’s standard built-in object types. Each type
xxx
(int
, float
, list
,
tuple
, str
, type
, etc.) is typically
implemented in the module xxxobject.py
.
The W_AbstractXxxObject
class, when present, is the abstract base
class, which mainly defines what appears on the Python-level type
object. There are then actual implementations as subclasses, which are
called W_XxxObject
or some variant for the cases where we have
several different implementations. For example,
pypy/objspace/std/bytesobject.py defines W_AbstractBytesObject
,
which contains everything needed to build the str
app-level type;
and there are subclasses W_BytesObject
(the usual string) and
W_Buffer
(a special implementation tweaked for repeated
additions, in pypy/objspace/std/bufferobject.py). For mutable data
types like lists and dictionaries, we have a single class
W_ListObject
or W_DictMultiObject
which has an indirection to
the real data and a strategy; the strategy can change as the content of
the object changes.
From the user’s point of view, even when there are several
W_AbstractXxxObject
subclasses, this is not visible: at the
app-level, they are still all instances of exactly the same Python type.
PyPy knows that (e.g.) the application-level type of its
interpreter-level W_BytesObject
instances is str because there is a
typedef
class attribute in W_BytesObject
which points back to
the string type specification from pypy/objspace/std/bytesobject.py;
all other implementations of strings use the same typedef
from
pypy/objspace/std/bytesobject.py.
For other examples of multiple implementations of the same Python type, see Standard Interpreter Optimizations.
Object Space proxies¶
We have implemented several proxy object spaces, which wrap another object space (typically the standard one) and add some capabilities to all objects. To find out more, see Transparent Proxies (DEPRECATED).