Python 3.15: lazy imports and UTF-8 by default

Python 3.15.0 final is being released today. Most releases bring a pile of nice but forgettable improvements. This one contains two changes I have been waiting a long time for, which is explicit lazy imports for faster startup and UTF-8 finally becoming the default encoding everywhere.

The startup time problem

If you have ever written a command line tool in Python, you will probably know the feeling. You run mycooltool --help and the terminal just sits there for a second or two before showing anything. The tool has not done any actual work yet, but it has spent all that time importing modules.

That is because imports in Python are eager. The moment the interpreter hits an import statement, it loads the module, executes it top to bottom and then does the same for everything that module imports. For a small script it is usually nothing, but for an application that pulls in something heavy like a cloud SDK, a data library or a large automation framework, the import chain can easily reach hundreds of modules before your main() even runs.

The classic workaround has been to hide imports inside functions, so they only happen when the code path actually needs them. It works, but a downside is that it scatters imports all over the codebase and linters will complain about it. It also kind of feels like fighting against the language.

PEP 810 solves the imports

PEP 810 fixes this problem with imports properly. Python 3.15 introduces a lazy keyword that you put in front of an import:

lazy import json
lazy from pathlib import Path

print("Starting up...")          # nothing loaded yet

data = json.loads('{"key": 1}')  # json gets imported here
p = Path(".")                    # and then pathlib here

So the name is bound immediately, but the module is not loaded and executed until the first time you actually touch it. If the code path does not use it, the import never happens. For a CLI tool where --help or a simple subcommand only needs a fraction of the dependencies, this is exactly the behavior you want there and now it is a single keyword instead of a whole refactoring exercise.

A few sensible restrictions will apply if you want to use this. Lazy imports are only allowed at module scope, which mean that errors like a missing module show up at first use instead of at import time, with a traceback that points to both the usage and the original import statement. It is also worth noting that lazy from module import * will throw a syntax error.

If you are testing something to know how it would benefit you of using vs not using lazy imports, you can flip the whole interpreter into lazy mode without touching any code:

python -X lazy_imports=all mycooltool.py
# or
export PYTHON_LAZY_IMPORTS=all

For libraries that need to support older Python versions, there is a __lazy_modules__ list that 3.15 treats as lazy and older interpreters simply ignore. Opt-in, explicit and backward compatible.

The people behind the PEP are not guessing about the benefits either. The design is based on years of experience with lazy imports at Meta, where the same idea in Cinder cut startup time for real applications by up to 70 percent. I do not expect my own tools to hit numbers like that, but shaving half a second off every invocation of my CLI apps will add up the numbers over time.

PEP 686 makes UTF-8 the standard

Another change is less flashy but has probably caused more real-world pain over the years. As of 3.15, PEP 686 UTF-8 mode will become the default, so this line finally does the same thing on every platform:

with open("inventory.txt") as f:
    data = f.read()

Until now, open() without an explicit encoding argument used the systems local encoding. On Linux and macOS that has effectively meant UTF-8 for years. On Windows it meant a legacy code page like cp1252, which is how you end up with code that works perfectly on your laptop and then mangles every Swedish character the first time a colleague runs it on their Windows machine.

The fix is simply that Python now does what everything else already does. Source files, JSON, TOML and YAML are all UTF-8. Now plain open(), subprocess.run(text=True) and friends are too.

If you genuinely need the old behavior, the escape hatches exists. Set the environment variable PYTHONUTF8=0, which turns the mode off entirely and encoding="locale" keeps using the system locale for a specific call. If you want to find the places in your codebase that relied on the implicit default, run your test suite with -W error::EncodingWarning and Python will point them out for you.