Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
refreshing imports
#1
I have an environment (tinycore linux) where as the system boots, python modules get loaded. The program needs to start up quickly, and it takes a while to load some modules like scipy.

In python2, this was not a problem, I could:
try:
    import scipy
except:
    return
and try again later.

In python3, this does not work. I have to restart the process, it always fails even long after the module is available if it was not available when python3 started.

I tried
import importlib
importlib.invalidate_caches()

but this does not work either, it doesn't solve the problem.

I know it's possible to use multiprocessing and communicate over pipes to get around this issue but it's really making the code complicated and less efficient.

So, how do I get the interpreter to rescan the available modules after it's started since invalidate_caches doesn't do what it claims to?
Reply
#2
What causes the initial failure? Are fileystems not mounted so you get a ModuleNotFoundError, or does something else happen?

I would suspect that importlib.reload(module) would be sufficient, but that depends on what the initial failure is.
Reply
#3
ModuleNotFoundError: No module named 'scipy'

you cannot reload it because the module is not found so from importlib.reload(scipy)
NameError: name 'scipy' is not defined

Should I install a stub module that exists at boot then reload it later?
Reply
#4
Are filesystems getting mounted? I don't see why a later import would fail here if the modules are getting introduced into the correct path.

>>> import os
>>> os.rename("tmdbsimple", "tmdbsimple.moved")
>>> import tmdbsimple
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'tmdbsimple'
>>> os.rename("tmdbsimple.moved", "tmdbsimple")
>>> import tmdbsimple
>>>
Reply
#5
It turns out this only affects modules that use a .pth file to redirect where they are located.

so I have scipy.pth in /usr/local/lib/python3.6/site-packages
and in scipy.pth the path to the scipy

When I rearranged it so the module was located in the scipy directory and did not need this file, it now works as I needed it to.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Imports that work with Python 3.8 fail with 3.9 and 3.10 4slam 1 2,551 Mar-11-2022, 01:50 PM
Last Post: snippsat
  Imports in my first package cuppajoeman 1 1,912 Jun-28-2021, 09:06 AM
Last Post: snippsat
  script with imports works but pytest gives "ModuleNotFoundError"? Hpao 0 1,544 Jun-27-2021, 08:30 PM
Last Post: Hpao
  Help wanted with python imports petros21 3 2,481 Apr-07-2021, 07:16 PM
Last Post: snippsat
Question How to include Modules not found (conditional imports) in my setup.py when I want to cff 0 3,767 Mar-17-2021, 11:57 AM
Last Post: cff
  threading across imports Nickd12 2 2,105 Nov-09-2020, 01:59 AM
Last Post: Nickd12
  Multimode imports fine as script but not after compiling into exe chesschaser 0 2,374 Aug-13-2020, 01:28 PM
Last Post: chesschaser
  absolute imports between folders mikisDW 0 1,507 Aug-05-2020, 12:26 PM
Last Post: mikisDW
  Understanding the concept ( Modules , imports ) erfanakbari1 1 2,140 Nov-25-2019, 01:59 AM
Last Post: Larz60+
  How Do I Get the IDE to Find My Imports petec 18 9,044 Feb-24-2019, 02:50 AM
Last Post: petec

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020