Python is undoubtedly one of the most popular programming languages in the world in recent years. The language is praised by developers for its simple syntax and the plethora of features it provides such as list comprehensions and context management, it powers many of the Fortune 100 software including Netflix, Uber, YouTube, and Spotify, and it is used for many use cases including web development, data science, A.I. and IoT implementations.

As of October 3^rd, 2022, the stable version of Python 3.11 was released for public usage, promising a handful of language improvements that developers can look forward to. This article scratches the surface of the most anticipated features of Python 3.11 according to the changelog.

Improved Error Messages

Python is an object-oriented programming language. Therefore, all built-in errors (Exceptions, as called officially by Python) are subclasses of BaseException class. By default, BaseException supports a set of positional arguments, as well as a method .with_traceback(tb) that sets tb as the new traceback for the exception. Following is an example of a built-in exception, ZeroDivisionException, raised due to a division-by-zero on Python 3,9:

The interpreter prints a traceback on the console saying that a division by zero occurred.

Traceback (most recent call last):
    file "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

Let us consider now the case of executing the same code from the Python module below (a module is simply a fancy way to refer to .py files):

# zero-div.py
if __name__ == "__main__":
  print(3 / 0) # raises ZeeroDivisionError

Since Python modules usually include more than 1 line, the output will be more elaborate as it prints the line where the error occurred:

In this situation, it is clear that the statement 3 / 0 is the one responsible for raising ZeroDivisionError. However, in certain situations where a single line may potentially raise the error from different statements, the traceback will not be very helpful for debugging. Let us consider the module below:

# index-out-of-range.py
if __name__ == "__main__":
    my_nested_list = [1, 2, 3, 4, [5, 6, 7, 8, 9, 10]]
  print(my_nested_list[4][6]) # index 6 is out of range

Running the code outputs:

The traceback is ambiguous as it is impossible to tell (just by looking at the error) which index is out of range.

Python improved this issue by introducing a clever way to indicate the column which caused the exception. Following is the output of running the index-out-of-range.py module using Python 3.11:

The interpreter adds a squiggly line using tilde signs (~)underneath the statement which raised the exception, while also indicating the exact column using carets (^). This is a very welcome addition to error messages in Python as it speeds up spotting the reason behind some bugs that were historically daunting to spot.

TOML Support

Tom's Obvious Minimal Language, or TOML for short, is an open-source file format for configuration files that is intended to be obvious to read and write. The language is adopted by many projects such as Jekyll and Hugo, Rust's manifest file Cargo, and Python's project and package manager Poetry

Python 3.11 introduces a new package in the standard library named tomllib that allows parsing TOML files into dictionaries, using an API that is relatively similar to json library.

# config.toml
language = "python"
version = "3.11"
release_date = "2022-10-22"
features = ["exception_output", "exception_notes", "tomllib"]

# main.py
import tomllib

if __name__ == '__main__':
  # assuming that config.toml file lives in the same folder
   with open('config.toml', 'rb') as toml:
        config = tomllib.load(toml)
        print(f'{config["language"]} v{config["version"]} is released on {config["release_date"]} with the following features: {", ".join(config["features"])}')

As of the time of writing this article, tomllib does not support writing TOML configurations

Exception Improvements:

Dealing with exceptions is an important part of software development, and as shown before, having your code spelling out exactly what went wrong is crucial to minimize debugging time and ensure high-quality code. Python provides the built-in class Exception both as a generic exception class and a super-class for user-defined exceptions.

Python 3.11 introduces a new Exception subclass called ExceptionGroup that groups several Exception objects and prints them (beautifully) to the standard error, or wherever you configured your code to print them.

Here's an example of how ExceptionGroups are generated:

raise ExceptionGroup("my exception group", [
  Exception('exception 1'),
  Exception('exception 2'),
  Exception('exception 3')
])

ExceptionGroup takes a title/description as the first parameter and a list of Exception objects as the second argument. Furthermore, since ExceptionGroup is a subclass of Exception, the list can also include an ExceptionGroup (although I personally am not aware of any possible use case).

The line of code outputs the following:

Exception groups are especially useful when running asynchronous tasks to catch all exceptions and handle them collectively on the main thread.

You may be wondering now: "how do I catch an ExceptionGroup? and should I use a series of if-statements to check for the type of exceptions inside the group?". Luckily, we do not have to do it spaghetti-style, as Python 3.11 introduced the except* syntax specifically to destructure the content of exception groups and handle them as you would any other exception:

try:
  raise ExceptionGroup("group", [ValueError(4), IndexError(6)])
except* ValueError as e:
  print(f"ValueError caught: " + e.__str__())
except* IndexError as e:
  print("IndexError caught: " + e.__str__())
except* ArithmeticError as e:
  print("ArithmeticError caught: " + e.__str__())

which outputs:

Another welcomed addition to the Exception class is the method .add_note(note) which allows adding user-defined messages for more clarity:

try:
  raise IndexError(6)
except IndexError as e:
  e.add_note("'your_list' has only 4 items")
  raise

which outputs:

The Self Type

Python is a dynamically-typed language, which means variables are not bound to a specific type. As this may be convenient, it may sometimes make sense to give weak types to variables to increase code readability and prevent unexpected types of data circulating your code, Python introduced typing since version 3.5, which allows developers to annotate (i.e. weakly type) variables in their code, and employ IDEs to intercept type mismatch ahead-of-time.

Following are examples of type annotations in Python +3.5:

# Example of "primitive" types
blog: str = "DevDog"
year: int = 2022
pi: float = 3.14

# Example of "composite" types
from typing import Optional, Union, List, Dict

names: List[str] = ['Albert', 'Jack', 'Alex']
colors_to_french: Dict[str, str] = {
  'red': 'rouge',
  'green': 'vert',
  'blue': 'bleu'
}

optional_var: Optional[str] = None  

# pre-3.10
number_or_string: Union[str, int] = 3

# +3.10
number_or_string: str | int = 'string is accepted as well'

# function with return type
def greet(name: str) -> str:
  return f"Hello ${name}"

Before Python 3.10, typing supported all kinds of type hints except for methods that return the class instance type. Therefore, Python 3.11 introduced the type Self which indicates that a method returns an instance of its class:

from typing import Self

INTEREST_PERCENTAGE = 0.12

class Employee:
  def __init__(self, name, salary):
    self.name = name
    self.salary = salary

  def set_salary_with_interest(base_salary) -> Self:
    return Employee(self.name, base_salary - (base_salary * INTEREST_PERCENTAGE))

Performance Improvements

To put Python 3.11 to the test, I have used Python Benchmark Suite, which runs a series of ~20 benchmarking algorithms and calculates the mean and the standard deviation for each set of executions per algorithm. Python v3.11.0 is tested against the latest iterations of its last 3 predecessors, namely v3.8.15, v3.9.14, and v3.10.8.

Note: Tests are performed on a 14-inch MacBook Pro with the M1 Pro chip and 16Gb of RAM

Full benchmark results are available in this GitHub repository.

Python 3.11 is outperforming all of its predecessors by margins reaching 60% in some cases. This is due to the improvements and optimizations performed on CPython 3.11 such as lazy bootup loading and PEP 659 – Specializing Adaptive Interpreter

Verdict

Python 3.11 is delivering key changes to the programming language that both improve the development experience and the overall performance of the language. Should you update your projects to 3.11? I believe this depends on how large your codebase is and which version you're running. Furthermore, if the migration costs and risks are minimal, it is worth the upgrade as you will be gaining a performance boost as well as better debugging tools.

For more articles, tutorials, and news about open-source projects, visit my blog at devdog.co