PyPy v7.3.2: release of 2.7, 3.6, and 3.7 alpha

Note

This is a pre-release announcement. When the release actually happens, it will be announced on the morepypy blog

The PyPy team is proud to release the version 7.3.2 of PyPy, which includes three different interpreters:

  • PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.13
  • PyPy3.6: which is an interpreter supporting the syntax and the features of Python 3.6, including the stdlib for CPython 3.6.9.
  • PyPy3.7 alpha: which is our first release of an interpreter supporting the syntax and the features of Python 3.7, including the stdlib for CPython 3.7.9. We call this alpha quality software, but it is based off PyPy 3.6 so issues should be around compatibility and not stability. Please try it out and let us know what is broken or missing. We have not implemented the documented changes in the re module, and other pieces are also missing. For more information, see the PyPy 3.7 wiki page

The interpreters are based on much the same codebase, thus the multiple release. This is a micro release, all APIs are compatible with the 7.3.0 (Dec 2019) and 7.3.1 (April 2020) releases, but read on to find out what is new.

Conda Forge now supports PyPy as a python interpreter. The support is quite complete for linux and macOS. This is the result of a lot of hard work and good will on the part of the Conda Forge team. A big shout out to them for taking this on.

Development of PyPy has transitioning to https://foss.heptapod.net/pypy/pypy. This move was covered more extensively in this blog post. We have seen an increase in the number of drive-by contributors who are able to use gitlab + mercurial to create merge requests.

The CFFI backend has been updated to version 1.14.2. We recommend using CFFI rather than c-extensions to interact with C, and using cppyy for performant wrapping of C++ code for Python.

Numpy has begun shipping wheels on PyPI for PyPy, currently for linux 64-bit only. We fixed PyUnicode_* C-API handling on windows, so now NumPy passes all tests on PyPy 3.6 win32, and wheels for PyPy windows will be available from the next NumPy release.

A new contributor took us up on the challenge to get windows 64-bit` support. The work is proceeding on the win64 branch, more help in coding or sponsorship is welcome.

As always, this release fixed several issues and bugs. We strongly recommend updating. Many of the fixes are the direct result of end-user bug reports, so please continue reporting issues as they crop up.

You can find links to download the v7.3.2 releases here:

We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work.

We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: PyPy and RPython documentation improvements, tweaking popular modules to run on pypy, or general help with making RPython’s JIT even better. Since the previous release, we have accepted contributions from XXX new contributors, thanks for pitching in.

If you are a python library maintainer and use c-extensions, please consider making a cffi / cppyy version of your library that would be performant on PyPy. In any case both cibuildwheel and the multibuild system support building wheels for PyPy.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7, 3.6, and 3.7. It’s fast (PyPy and CPython 2.7.x performance comparison) due to its integrated tracing JIT compiler.

We also welcome developers of other dynamic languages to see what RPython can do for them.

This PyPy release supports:

  • x86 machines on most common operating systems (Linux 32/64 bits, Mac OS X 64 bits, Windows 32 bits, OpenBSD, FreeBSD)
  • big- and little-endian variants of PPC64 running Linux,
  • s390x running Linux
  • 64-bit ARM machines running Linux.

Unfortunately at the moment of writing our ARM32 buildbots are out of service, so for now we are not releasing any binaries for that architecture, although PyPy does support ARM 32 bit processors.

Changelog

Changes shared across versions

  • More sysconfig fixes for win32
  • Fix time.tm_gmtoff and tm_zone on win32 to be like CPython
  • Speed up the performance of matching unicode strings in the sre engine
  • Add jobs control to the Makefile (issue 3187)
  • encode unicodes even in (built-in) files opened in binary mode (issue 3178)
  • Refactor RSocket.xxx_into() methods and add .recvmsg_into()
  • Invoke pip via runpy in ensurepip (merge request 723)
  • Fix implementation of PEP 3118 in ctypes
  • Package macOS as a “portable package” including needed dylib files
  • Allow overridden .__int__() in subclasses of complex
  • More bitbucket -> heptapod documentation fixes
  • Fix path to tcl,tk on windows, issue 3247
  • Speed up list.pop, list.insert operations that shift many items (merge request 729)
  • Update win32 to openssl1.1
  • Assigning a class instance as __get__ should call the class’s __call__, not its __get__ (issue 3255)
  • Remove all implicit str-unicode conversions in RPython (merge request 732)
  • Need FORBID_TEMP_BOXES on aarch64, like on arm32, to prevent register spilling (issue 3188)
  • Initialize lock timeout on windows if timeout is infinite (impacted PyPy 2.7 only) (merge request 744, issue 3252)
  • Move lib_pypy/tools -> lib_pypy/pypy_tools: otherwise tools appears on the sys.path. This would override any tools in packages like NumPy.
  • Update libffi_msvc by copying the fixes done in cffi, and also fix when returning a struct in CFFI
  • Add full precision times from os.stat on macOS
  • PyPy 2.7 only: add SOABI to distutils.sysconfig so wheel gets the correct ABI tag
  • Use the full set of format strings in time.strftime on windows. After the transition to a more modern MSVC compiler, windows now supports strings like %D and others.
  • Raise like CPython when strftime() year outside 1 - 9999 on windows
  • Use wcsftime in strftime() on windows
  • Fix issue 3282 where iterating over a mmap should return an int, not a byte
  • Use _chsize_s for ftruncate on windows: CPython issue 23668
  • Allow untranslated tests to run on top of PyPY2.7, not only CPython2.7
  • Add missing os functions os.sched_rr_get_interval, os.sched_getscheduler, os.sched_setscheduler, os.sched_getparam
  • Set buffer to None on close of buffered io classes
  • Use the Suppres_IPH context manager wherever CPython uses _Py_BEGIN_SUPPRESS_IPH, _Py_END_SUPPRESS_IPH
  • Fix leaked string if an exception occurs in socket.settimeout on windows
  • close open mmap and zipfile resources in stdlib tests
  • Make stack 3MB on windows which aligns expectations with Linux
  • Add pypyjit.releaseall() that marks all current machine code objects as ready to release. They will be released at the next GC (unless they are currently in use in the stack of one of the threads).
  • Fix possible infinite loop in tarfile.py: CPython issue 39017
  • Reject control characters in http requests: CPython issue 39603
  • Fix regex in parsing http headers to reject infinite backtracking: CPyton issue 39503
  • Escape the server title when rendering from xmlrpc as HTML: CPython issue 38243

C-API (cpyext) and c-extensions

  • Add PyCFunction_Call, PyNumber_ToBase, contiguous part of PyMemoryView_GetContiguous
  • use space.getitem in PySequence_ITEM, fixes pybind11 2146
  • give preference to as_sequence.sq_item in PySequence_ITEM
  • In Py_EnterRecursiveCall, char* -> const char *, issue 3232
  • Fix PySet_Add for frozenset (issue 3251)
  • Support using sq_repeat and sq_inplace_repeat, issue 3281

Python 3.6 only

  • Fix _sqlite3.Connection with isolation_level=None in the constructor
  • Fix embedded mode for CFFI (CFFI issue 449)
  • Add socket.recvmsg_into
  • Fix return types in os.readlink() (issue 3177) and os.listdir()
  • Fix os.listdir()` for win32
  • Update _ssl to handle Post Handshake Authorization callbacks (PHA)
  • Fix oldcrc argument of binascii.crc_hqx to unsigned int
  • Implement socket.sethostname() (issue 3198)
  • Backport CPython 35519: “Rename test.bisect to test.bisect_cmd” from CPython 3.7
  • Fix the repr of SRE_Pattern and SRE_Match
  • Fix ill-defined behaviour with class.__init__ (issue 3239)
  • Improve pickling performance (issue 3230)
  • Forward port race condition fix from PyPy 2.7 Lib/weakref.py (issue 3243)
  • Implement bpo 30465: Fix lineno and col_offset in fstring AST nodes (merge request 730)
  • Implement bpo 29104: Fixed parsing backslashes in f-strings (merge request 736)
  • Fix time.sleep bug in win32
  • Expose missing lzma_stream_encoder in cffi _lzma module (issue 3242_)
  • Fix os.unsetenv on win32, bpo 39413 (CPython 3.7+, we can do 3.6+)
  • Add symlinks to pypy, pypy3.6 to linux, macOS tarball. Maybe needed for macOS and multibuild
  • The following sequence no longer makes any copy: b = StringBuilder(); b.append(some_large_string); s = b.build()
  • Add missing os constants: P_NOWAIT, P_NOWAITO, P_WAIT
  • Allow codec errorhandlers to modify the underlying str/bytes being converted
  • Do not import platform at startup (issue 3269)
  • Enable more extensive optimizations of list strategies on Python3, since int and long are the same (issue 3250)
  • Special case bytewise codec errorhandlers replace, ignore, surrogateescape to use the same logic as final == True
  • Allow CRTL-C to interrupt time.sleep on windows
  • Inhibit compiler tail-call optimization via PYPY_INHIBIT_TAIL_CALL on windows
  • When pypy -m pip fails to find pip, give an error message that hints at pypy -m ensurepip
  • Fix broken _socket.share on windows
  • Add missing os.{gs}et_handle_inheritable (PEP 446) on windows
  • Fix ip address hashing in ipaddress.py: CPython issue 41004_
  • Disallow CR/LF in email.headerregistry.Address: CPython issue 39073
  • Ban reuse_address parameter in loop.create_datagram_endpoint() in asyncio.base_events: CPython issue 37228
  • Preventing newline in encodongs.uu.filename from corrupting the output format: CPython issue 38945
  • Prevent backtracking in regexes in http.cookiejar: Cpython issue 38804
  • Sync email._header_value_parser, email._parseaddr and their respective tests: CPython 37461 and 34155

Python 3.6 C-API

  • Add PyType_GetFlags, PyType_GetSlot, PyUnicode_{En,De}code_Locale, PyUnicode_{Find,Read,Write}Char,
  • Fix PyUnicode_* handling on windows where wchar_t is 2 bytes