Problem
I want to import a function from another file in the same directory.
It works for me with from time to time. but sometimes I get a: mymodule import myfunction
SystemError: Parent module '' not loaded, cannot perform relative import
Sometimes it works using import myfunction from mymodule, but other times I get a:
SystemError: Parent module '' not loaded, cannot perform relative import
I’m not sure what’s going on here, and I couldn’t locate an explanation. This appears to be absolutely random.
Could someone explain to me what’s the logic behind all this?
Asked by John Smith Optional
Solution #1
It’s not uncommon to see a design like this…
main.py
mypackage/
__init__.py
mymodule.py
myothermodule.py
…with something like this in mymodule.py…
#!/usr/bin/env python3
# Exported function
def as_int(a):
return int(a)
# Test function for module
def _test():
assert as_int('1') == 1
if __name__ == '__main__':
_test()
…something along the lines of myothermodule.py…
#!/usr/bin/env python3
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
…as well as a main.py that looks like this…
#!/usr/bin/env python3
from mypackage.myothermodule import add
def main():
print(add('1', '1'))
if __name__ == '__main__':
main()
…which works great when running main.py or mypackage/mymodule.py, but fails when running mypackage/myothermodule.py because of the relative import…
from .mymodule import as_int
It’s intended to be done this way…
python3 -m mypackage.myothermodule
…but it’s somewhat verbose, and doesn’t mix well with a shebang line like #!/usr/bin/env python3.
If the name mymodule is globally unique, the simplest solution is to avoid using relative imports and instead use…
from mymodule import as_int
…but, if it isn’t unique or your package structure is more complicated, you’ll need to add the directory containing your package directory in PYTHONPATH and do it like this…
from mypackage.mymodule import as_int
…otherwise, if you want it to work right now, you can use this to frob the PYTHONPATH in code…
import sys
import os
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))
from mypackage.mymodule import as_int
It’s a pain, but in an email written by a certain Guido van Rossum, there’s a hint as to why…
Running scripts inside a package may or may not be considered an antipattern, but I find it quite handy in a package I have that has some custom wxPython widgets, because I can execute the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.
Answered by Aya
Solution #2
From PEP 328
PEP 338 and PEP 328 clashed at one point:
and to address the issue, PEP 366 introduced the top level variable __package__:
(emphasis mine)
name .rpartition(‘.’)[0] yields empty string if the name is ‘ main ‘. This is why the error explanation contains an empty string literal:
SystemError: Parent module '' not loaded, cannot perform relative import
The relevant part of the CPython’s PyImport_ImportModuleLevelObject function:
if (PyDict_GetItem(interp->modules, package) == NULL) {
PyErr_Format(PyExc_SystemError,
"Parent module %R not loaded, cannot perform relative "
"import", package);
goto error;
}
If package (the package name) could not be found in interp->modules, CPython throws this exception (accessible as sys.modules). Because sys.modules is a “dictionary that maps module names to modules that have previously been loaded,” it’s evident that the parent module must be explicitly absolute-imported before relative import can take place.
Note that the fix for issue 18018 included another if block that will run before the code above:
if (PyUnicode_CompareWithASCIIString(package, "") == 0) {
PyErr_SetString(PyExc_ImportError,
"attempted relative import with no known parent package");
goto error;
} /* else if (PyDict_GetItem(interp->modules, package) == NULL) {
...
*/
The error message will be if package (same as above) is empty string.
ImportError: attempted relative import with no known parent package
However, this is only visible in Python 3.6 and newer.
Consider the following directory (a Python package):
.
├── package
│ ├── __init__.py
│ ├── module.py
│ └── standalone.py
The first two lines of code in every file in the package are the same:
from pathlib import Path
print('Running' if __name__ == '__main__' else 'Importing', Path(__file__).resolve())
These two lines are just used to demonstrate the order of operations. We can fully disregard them because they have no bearing on the execution.
Only those two lines are in init .py and module.py.
standalone.py additionally attempts to import module.py via relative import:
from . import module # explicit relative import
We know that /path/to/python/interpreter package/standalone.py will not work. The -m command line option, on the other hand, will “search sys.path for the named module and execute its contents as the __main__ module”:
vaultah@base:~$ python3 -i -m package.standalone
Importing /home/vaultah/package/__init__.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/module.py
>>> __file__
'/home/vaultah/package/standalone.py'
>>> __package__
'package'
>>> # The __package__ has been correctly set and module.py has been imported.
... # What's inside sys.modules?
... import sys
>>> sys.modules['__main__']
<module 'package.standalone' from '/home/vaultah/package/standalone.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
-m automates all the importing for you and sets __package__ for you, but you can do it yourself in the command line.
Please consider it more of a proof of concept than a complete solution. It’s unsuitable for usage in real-world programs.
PEP 366 has a workaround to this problem, however, it’s incomplete, because setting __package__ alone is not enough. You’re going to need to import at least N preceding packages in the module hierarchy, where N is the number of parent directories (relative to the directory of the script) that will be searched for the module being imported.
Thus,
I’ll use some of the files from Solution #1 and add a few more subpackages:
package
├── __init__.py
├── module.py
└── subpackage
├── __init__.py
└── subsubpackage
├── __init__.py
└── standalone.py
This time, standalone.py will use the relative import to import module.py from the package package.
from ... import module # N = 3
To make it work, we’ll need to put the boilerplate code before that line.
import sys
from pathlib import Path
if __name__ == '__main__' and __package__ is None:
file = Path(__file__).resolve()
parent, top = file.parent, file.parents[3]
sys.path.append(str(top))
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
import package.subpackage.subsubpackage
__package__ = 'package.subpackage.subsubpackage'
from ... import module # N = 3
It allows us to run standalone.py by specifying the filename:
vaultah@base:~$ python3 package/subpackage/subsubpackage/standalone.py
Running /home/vaultah/package/subpackage/subsubpackage/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/subpackage/__init__.py
Importing /home/vaultah/package/subpackage/subsubpackage/__init__.py
Importing /home/vaultah/package/module.py
You can find a more general solution wrapped in a function here. Use this as an example:
if __name__ == '__main__' and __package__ is None:
import_parents(level=3) # N = 3
from ... import module
from ...module.submodule import thing
The steps are as follows:
For example, consider the following directory structure.
.
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module.py
│ │ └── standalone.py
│ └── setup.py
where setup.py is
from setuptools import setup, find_packages
setup(
name = 'your_package_name',
packages = find_packages(),
)
The remainder of the files were taken from Solution #1.
Installing the package will allow you to import it regardless of your working directory (assuming no naming conflicts).
We may utilize this advantage by modifying standalone.py (step 1):
from package import module # absolute import
Change your working directory to project and run /path/to/python/interpreter setup.py install –user (–user installs the package in your site-packages directory) (step 2):
vaultah@base:~$ cd project
vaultah@base:~/project$ python3 setup.py install --user
Let’s verify that it’s now possible to run standalone.py as a script:
vaultah@base:~/project$ python3 -i package/standalone.py
Running /home/vaultah/project/package/standalone.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py
Importing /home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/.local/lib/python3.6/site-packages/your_package_name-0.0.0-py3.6.egg/package/module.py'>
Note: If you go this approach, you’ll be better off installing packages in isolation using virtual environments.
To be honest, you don’t need to install anything; you could just add some boilerplate code to your script to make absolute imports work.
I’m going to borrow files from Solution #1 and change standalone.py:
standalone.py works flawlessly:
vaultah@base:~$ python3 -i package/standalone.py
Running /home/vaultah/package/standalone.py
Importing /home/vaultah/package/__init__.py
Importing /home/vaultah/package/module.py
>>> module
<module 'package.module' from '/home/vaultah/package/module.py'>
>>> import sys
>>> sys.modules['package']
<module 'package' from '/home/vaultah/package/__init__.py'>
>>> sys.modules['package.module']
<module 'package.module' from '/home/vaultah/package/module.py'>
I feel compelled to caution you: avoid doing so, especially if your project has a complex structure.
PEP 8 promotes the use of absolute imports, but states that explicit relative imports are permissible in several situations:
Answered by vaultah
Solution #3
Put this in the init .py file of your package:
# For relative imports to work in Python 3.6
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
Assume the following for your package:
├── project
│ ├── package
│ │ ├── __init__.py
│ │ ├── module1.py
│ │ └── module2.py
│ └── setup.py
Use regular imports in your package now, like as:
# in module2.py
from module1 import class1
This works in Python 2 and Python 3.
Answered by am5
Solution #4
This is a problem I encountered. Importing with an if/else block, as shown below, is a hack workaround:
#!/usr/bin/env python3
#myothermodule
if __name__ == '__main__':
from mymodule import as_int
else:
from .mymodule import as_int
# Exported function
def add(a, b):
return as_int(a) + as_int(b)
# Test function for module
def _test():
assert add('1', '1') == 2
if __name__ == '__main__':
_test()
Answered by goffer
Solution #5
This suggests you’re using a script to launch a module from within the package. It’s risky to mix scripts inside packages, and it should be avoided if at all possible. Instead, use a wrapper script that imports the package and executes your scripty function.
If your top-level directory is called foo, which is on your PYTHONPATH module search path, and you have a package bar there (it is a directory you’d expect an __init__.py file in), scripts should not be placed inside bar, but should live on in foo at best.
Note that scripts differ from modules here in that they are used as a filename argument to the python command, either by using python or via a #! (shebang) line. It is loaded directly as the __main__ module (this is why if __name__ == “__main__”: works in scripts), and there is no package context to build on for relative imports.
All of the preceding assumes that your package may be imported, which implies it must be located in one of the sys.path folders (or zipfiles). There are a few alternatives here as well:
Answered by Martijn Pieters
Post is based on https://stackoverflow.com/questions/16981921/relative-imports-in-python-3