The expected end of support for Python 2.7 is 1st January 2020, at least according to Guido van Rossum’s blog post. Starting now, you should consider developing all new Python applications in Python 3, and migrating existing code to Python 3 as and when time and workload permit.
Contents
Moving to Python 3
If you are unaware of the changes introduced in Python 3 that broke backward compatibility with Python 2 then there is a good summary on this What’s New In Python 3.0 web page.
The biggest difference you will notice moving to Python 3 is that the print statement is now a print function. But there are plenty of other changes that you should be aware of. This and subsequent blogs will look at aspects of Python has been added or improved in Python 3.
Migrating existing Python 2 code normally starts with running the 2to3.py script (supplied since Python 2.7). This script automates the mechanical process of converting syntax: primarily changing all print statements to function calls, renaming deprecated functions and methods (such as xrange to range), and updating import statements to match the refactored standard library modules. Unless you have used deprecated or out of date functions or modules the converted code should now compile.
One downside to this approach is that it is all too easy to continue using Python 2 features you are familiar with when there are better alternatives available. Learning the features of Python 3 is a case of reading the What’s New page for each version of Python 3 as it is released, looking for additional features or changed APIs.
Use Type Hints for Static Type Checking
Type hints were introduced in Python 3.5 and have been refined in later versions. They are intended to provide static type support for Python during code development and are similar to the type checking capabilities you get from compilers with languages such as C/C++, Java and C#. You can add type hints to function/method parameters and return types (Python 3.5), and variables used in assignment (effectively declarations – Python 3.6).
While type hint information is included in the compiled binary code (the .pyc file) it is not used for run-time type checking so does not introduce a runtime overhead other than a small amount of extra memory usage.
To get the benefits of adding type hints to your code you will need to use an IDE that understands type hints or pass your code through a static type checker either manually or as part of a Continuous Integration (CI) workflow.
An IDE currently supporting type hint checking on method parameters is PyCharm
from Jetbrains (there is a free Community version). Neither of the other popular Python editors Eclipse/PyDev or Spyder support type hints at this time.
Static analyzers supporting type hints are the experimental mypy and pytype (not available for windows). The popular static analyzer Pylint does not yet support type hints.
Both PyCharm, mypy and ptype use the Typeshed collection of library stubs defining type hints for the standard python library and some 3rd party libraries.
If you like strongly typed languages you will love type hints because you can now annotate variables and parameters with their type and have a static checker highlight language misuse without imposing the overhead of a runtime check.
If you like code completion (code assist/intellisense) features like a popup list of methods for an object then type hints will give you a list targeted to the type of the object.
Type Hint Syntax
As a simple example for using type hints we’ll look at a function that takes a string parameter called text and returns an integer. Lets say we use it to extract the integer value from a line such as “MAXSIZE=512”. Here is a function with a stubbed out body.
def extract(text):
return None
If you edit this code with PyCharm and start typing code at the beginning of the method by entering the characters text. the popup list of completion options will not normally include string methods – you’ll see just standard object methods like bit_length.
We can now add the type hint str to the parameter to show that it expects a string object, and include an int return type as follows:
def extract(text: str) -> int:
return None
The first thing you will see in PyCharm is the None return value highlighted as an error because the function does not return an integer – we will correct this later. If again we type the characters text. the popup will use the type hint to show a list of string methods making it easier to develop the function body.
Refactoring the stub function to provide simple functionality (ignoring error handling for clarity) will remove the type warning on the return statement:
def extract(text: str) -> int:
key, _, value = text.partition('=')
return int(value)
Calling a function does not change when we use type hints:
maxsize = extract('MAXSIZE=512')
PyCharm and static analysis tools will identify that the maxsize variable references an integer and provide appropriate popup help and/or static type checking.
Since Python 3.6 (Dec 2016) we can also use type hints on the first assignment to a variable as in:
maxsize: int = extract('MAXSIZE=512')Effectively type hints add variable declarations to the Python language: note that it is an error to add a type hint to a variable that has already been assigned a value.
This isn’t a type hints tutorial but it’s worth taking this example a little further to introduce
the typing module.
If we wanted to allow our stub code to allow None as well as an integer we would wrap the return type inside typing.Optional – as an aside these are often called nullable objects from the terminology introduced in C#.
Updating our function to permit a return of None we refactor the return type to Optional[int]:
from typing import Tuple
def extract(text: str) -> Optional[int]:
key, sep, value = text.partition('=')
if sep isNone:
returnNone
return int(value)The typing module defines several additional type objects that can be used to capture most type constructs used on a regular basis in Python. Simple examples being:
- List[int] – a list of integers
- Tuple[bool, float] – a tuple with two items: a boolean and a float
- Dict[str, int] – a dictionary accessed using a string key and holding an integer
- Dict[str, List[int]] – a dictionary with a string key holding a list of integers
Returning to our function example we can update the function return an optional tuple containing the keyword and its integer value:
from typing import Optional, Tuple
def extract(text: str) -> Optional[Tuple[str, int]]:
key, sep, value = text.partition('=')
if sep is None:
return None
return key.strip(), int(value)Ideally we’d now like to see PyCharm highlight our current assignment as an error:
n: int = extract('MAXSIZE=512')But at the moment PyCharm does not support type hints on assignment.
Whereas mypy does support assignment type hints and will show the error on line 9 of our blog.py file:
blog.py:9: error: Incompatible types in assignment
(expression has type "Optional[Tuple[str, int]]", variable has type "int")At this point you might be thinking that this isn’t Python anymore – it’s too complex and unreadable and has lost sight of the quick development aspects of simple Python scripts.
Hints are an optional language feature: so with simple scripts you can continue to use Python without cluttering up the language, just consider adding type hints to larger Python projects or libraries to improve static analysis.
Summary
Type hints can help you improve code quality and reduce development time by using static analysis to identify mismatched types as you write the code. With a type hint aware IDE (like PyCharm) you’ll get type specific code completion popup help as well.
Remember that if you are developing library code and add type hints to your functions and methods not all developers will be aware of, or make use, of, type hints; which means you may still want to include runtime type checks using isinstance or hasattr functions if you think this is required. Type hints do not replace dynamic type checks.
If you want to develop robust high quality Python code making best use of the development toolchain then type hints are a valuable addition to the language.
- CMake Presets - August 1, 2023
- C++20 Coroutine Iterators - September 23, 2021
- C++20 Coroutines - September 16, 2021
An independent IT trainer Martin has over 40 years academic and commercial experience in open systems software engineering. He has worked with a range of technologies from real time process controllers, through compilers, to large scale parallel processing systems; and across multiple sectors including industrial systems, semi-conductor manufacturing, telecomms, banking, MoD, and government.
