Any program you start developing, which turns into a multi-file project has the same problem. How do you use code from a different file? In Python, this is done with the import
keyword. But it isn’t as easy as you think.
The problem
Once you get to the complexity which requires not simply using modules but creating an entire package, things get messy. But let’s start with a simple script.
Imports aren’t necessary but they’re inevitable
Take a look at this script:
def run(): print("This script has no imports")
It does not need anything but standard language features, therefore it has no imports. It’s pretty boring.
Let’s say, however, that it needs to print the message in multiple languages. We can make it process a parameter, which defines the language and then open a file to read the text from there – a standard technique for internationalization. Like this:
import sys def run(): print("This script has no imports") if __name__ == "__main__": if len(sys.argv) > 1: language_code = sys.argv[1] else: language_code = "en" with open("i18n_" + language_code + ".properties") as f: translation = f.readlines()[0] # the only line we need is the first one in the file run(translation)
You can see that we already have an import here. So it is just a matter of time to use one. In any case, this isn’t that hard and everything works out of the box. But now we have a pretty complex module with a bunch of different responsibilities in one place.
Bringing more complexity
We know that different responsibilities should not be mixed, so we want to split off reading the file into a separate module. We can move the file operations to a separate file called fileops.py
def read_translation(language_code): with open("i18n_" + language_code + ".properties") as f: translation = f.readlines()[0] # the only line we need is the first one in the file return translation
With this action, we do not have access to the file I/O operations in our main file. Therefore we need to import the new module into our main.py:
import fileops
and then access the new function there like that:
translation = fileops.read_translation(language_code)
Introducing packages
By now we have only two modules in a single directory, but we’re getting into a stage where we would need to organize them into an entire package with submodules.
To do this we would have to introduce a __init__.py
file in the directory where our modules stay. Then if needed, we can create subdirectories where separate modules would be located. Each subdirectory will have its own __init__.py
file. Our package would look something like this:
awesomepackage |- __init__.py |- main.py |-io |-__init__.py |-fileops.py
And here start the problems
Just like that, the project is already complex enough by itself to start messing it up. On top of that, if you are just starting with Python, it starts fighting you back.
The symptoms
If we now try and run our simple application it will, naturally, fail. This is because we moved the fileops
module to another submodule. So we need to change the import statement:
import io.fileops
But if we run our program now, there is an error message:
ModuleNotFoundError: No module named 'io.fileops'; 'io' is not a package
So this doesn’t look good. But Python offers more than one way to import a module. Perhaps there is a difference in the interworkings which would allow us to finally import our module.
Let’s use this:
from io.fileops import read_translation
And then of course remove the dot-notation navigation to this function – use it directly:
translation = read_translation(language_code)
All these exercises, however, lead to the same result:
ModuleNotFoundError: No module named 'io.fileops'; 'io' is not a package
The documentation
At this point, it is time to hit the documentation and read carefully about Python’s import system. So I went directly to docs.python.org and looked for “import”.
The reference
This leads me to the following page: 5. Imports. This is the official reference to the import system in Python. Unfortunately, it is not written for beginners. It is a reference, after all, its job is to explain everything in detail, but it also assumes the reader has extensive knowledge of Python. I, on the other hand, am only a beginner. This combination makes the reference a very boring and frustrating read. There is just a single section which I understand and which may be connected with my problem: 5.7 Package Relative Imports But even if this section has something to do with my problem, it is written in such a formal way, that I miss the point in it.
The tutorial
Luckily, docs.python.org also has some helpful tutorials like this one: 6. Modules where I found a very detailed and easy to understand explanation of how to create and use modules, packages and other stuff. This includes the use of imports.
The knowledge
In this tutorial, I read that the import
keyword is used for absolute imports, meaning that you have to specify the absolute path to the submodule you want to use. In case you want to use a module from a neighbouring submodule, you can use the other syntax form: from path.to.submodule import thing
. Using this form also means you do not need to specify the module name when using the thing because it is added to the name table of your module.
But all this is quite intuitive and it does not tell us what our problem is and how to solve it.
The Package section
Inside this tutorial, however, there is a section describing the structure and interworkings of a package. This includes different ways of performing imports as well. There we can see how exactly we need to specify the module we want to import with the different forms of imports, as well as referencing modules inside of our own package, etc.
A particularly interesting piece of information there is the one about relative and absolute imports.
The solution
The unsuccessful attempts
Knowing about absolute and relative imports, we can try a slightly different from-import:
from .io.fileops import read_translation
But this one fails with an import error:
ImportError: attempted relative import with no known parent package
Reading the tutorial a bit more carefully would show us that a relative import is not allowed in the module with __name__ = '__main__'
, so this is not the way to go. However it explains why if you try to import a module into the interactive prompt everything is fine, but executing it using python -m
would make it fail. Simply the module name changes depending on how you use it.
That means if you ever use a relative import in your module, you cannot later use it as a standalone application.
Clearly using relative imports is not the right way to go here. But what is then making our application not recognize the new io
submodule we just created? So we need to go with absolute imports. This means we need to change the import from:
from .io.fileops import read_translation
to:
from awesomepackage.io.fileops import read_translation
The results are unimpressive. The same ModuleNotFoundError pops up on the screen.
Where you run your package from
Up until now, I have been executing the package using:
. This command was executed in the root of our package. But if I instead try to run it from outside the package, I end up with a different result. This time there is an error, but it doesn’t look like an import error, it looks like our i18n_en.properties file is missing. I guess the import finally worked!python -m main
Of course, this is for the price of ugly execution command: python -m awesomepackage.main
It looks like I’m executing a file with the
extension, instead of a python module. However, a quick search points me to moving the entry point of the package to the special file .main
_
. Now I can execute the package by referencing it instead of its _main__.py
module.main
Conclusions
Python has a complex import syntax with many forms and nuances. The different import statements are obviously there to help you in different situations, but for the beginner, they are somewhat perplexing. However, if you read a bit about them, it becomes clear which one is needed where.
Often, when talking about imports, creating a package is at the beginning of this journey. Knowledge of how packages work in Python is undoubtedly fundamental for anyone who wants to create a sophisticated application.
So go on and read a bit and try out what you’ve learned. Then rinse and repeat. That’s how one becomes a master, right? Right?
Cheers,
This article was inspired by my work on this other one: How To Easily Reuse Code With Python Packages. You’re free to have a look at it :)