Python Forum
using self-developed modules
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
using self-developed modules
#1
I use Win10 and Python 3.7.4.
I wrote a program that contains a number of inline functions and classes followed by a main program. It imports a module that I wrote that contains some functions and classes that are used in multiple programs.
My programs look as follows:

main.py
import sys
from selfwrittenmodule import f1, f2
def if1:
    some lines of code
def if2:
    some lines of code
b = 2
a = if1(b)
c = if2 (b)
print (a, b)
selfwrittenmodule
def f1:
Reply
#2
Sorry: I'm not yet used to this forum, I managed to file an incomplete question. Here is the completed question:

I use Win10 and Python 3.7.4. I am rather new to Python.

I wrote a python program that contains a number of inline functions and classes followed by a main program. It imports some “standard” modules (sys, decimal, tkinter, …) and also a home-grown module that contains functions and classes that are also used in other modules.

My modules look (simplified) like:

main.py
import sys
from selfwrittenmodule import f1, f2
def if1(x):
….a = f1(x);return(a)
def if2(x):
….a = f2(x);return(a)
b = 2
a = if1(b)
c = if2(b)
print (a, c)

selfwrittenmodule.py
def f1(x):
….some lines of code
def f2(x)
….some lines of code

I stored the “main.py” module in a folder dedicated to “work in progress”. Because “selfwrittenmodule,py” is for use by many other scripts to, it is stored in a folder dedicated to “general use” modules.
In the manuals I found three ways to ensure that the python interpreter finds modules in other folders:
1) Add a “PYTHONPATH” variable to the system variables and have it point to the “general use” folder.
2) Add the “general use” folder to the “%PATH%” system variable.
3) Move the “general use” folder, with the “selfwrittenmodule.py” in it, into C:\Users\actname\AppData\Local\Programs\Python\Python37-32\Lib

I tried all three methods, no joy: when I press “F5” in idlex, I receive an error message that the “selfwrittenmodule.py” is not found.

Then I moved “selfwrittenmodule.py” to the folder that contains “main.py” and press f5 in idlex. Now the python interpreter finds “selfwrittenmodule.py” and “main” is “compiled” and executed fine.

Question: how can I store “selfwrittenmodule.py” in a folder different from my “work in progress” folder and be able to import it correctly in another module by the python interpreter?
Reply
#3
Use Code tag.
Look at this post,where i show use of site.

Looking at from outside it work like this.
Python find modules/packages(also .py files) in sys.path
>>> import sys 
>>> 
>>> sys.path
OS find Python bye looking in Environment Variables Path.

3)... should work for you if Python is in your Windows Path.
I explain more in here Python 3.6 and pip installation under Windows.
Also i not so happy with default install path C:\Users\actname\AppData\Local\Programs\Python\Python37-32\Lib,look at link over if want to change.
Reply
#4
Thx for your suggestions.
T
First some more info: some time ago I have installed Python 3.7.4 and all the “packages” that I need using the standard installation (“Install Now” without customizing folder names). I selected the “Install Launcher” and “Add PATH” as recommended.

I have tried, as you suggested, with “solution 3”, but it definitely looks as if my Python does not “follow the book”.
I have documented both modules with some more comments here”)

# main.py
#
# is stored in G:\\UserData\\actname\\Documents\\Programmas\\Python_3-7-4\\
#
import sys
from selfwrittenmodule import f1, f2
def if1(x):
    a = f1(x);return(a)
def if2(x):
    a = f2(x);return(a)

b = 2
a = if1(b)
c = if2(b)
print (a, c)
# selfwrittenmodule.py
#
# is stored in C:\\Users\\actname\\AppData\\Local\\Programs\\Python\\Python37-32\\Lib\\generaluse\\
#
def f1(x):
    x = x + 1
def f2(x)
    x = x + 10
When I “edit” main with IdleX and press F5 (run), then I receive this error msg:

Error:
Traceback (most recent call last): File "G:\UserData\actname\Documents\Programs\Python_3-7-4\main.py", line 6, in <module> from selfwrittenmodule import f1, f2 ModuleNotFoundError: No module named 'selfwrittenmodule'
Then I renamed the foldername from C:\\Users\\actname\\AppData\\Local\\Programs\\Python\\Python37-32\\Lib\\generaluse\\ to C:\\Users\\actname\\AppData\\Local\\Programs\\Python\\Python37-32\\Lib\\selfwrittenmodule\\, and executed “main” again with IdleX. Then I get this error message:
Error:
Traceback (most recent call last): File "G:\UserData\actname\Documents\Programs\Python_3-7-4\main.py", line 6, in <module> from selfwrittenmodule import f1, f2 ImportError: cannot import name 'f1' from 'selfwrittenmodule' (unknown location) >>>
Please tell me what I did wrong.
Reply
#5
(Aug-29-2019, 10:36 AM)wlp Wrote: Please tell me what I did wrong.

It will not address the main issue, but you did wrong here:

def f2(x)
I don't observe :
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#6
Read my post again,you should not save in python folder and you still have the long path.
I make it simple and have python in this path C:\python37,as i show in install link.

In my first link ,see that i have added E:\\div_code to path,just a place i have my python files.
If i save selfwrittenmodule.py in E:\\div_code,it will work from anywhere in cmd or cmder as i use.
# selfwrittenmodule.py
def f1(x):
    x = x + 1
    return x
Now can i test dos not matter which folder i am in.
C:\Users\Tom
λ ptpython
>>> from selfwrittenmodule import f1

>>> f1(42)
43
Reply
#7
Sorry, I misunderstood your reply. I thought with "3)..." you were referring to MY number 3), but you probably referred to your sys.path?

I cannot add your code (print (sys.path)?) in my program to see what the variable contains because the execution of main is stopped already during the initialization of the pseudo code.

Yes, I still am using the long paths. I agree that that is ugly and requires a lot of typing, but to organize the many files on the disk in a number of folders, it's not bad. And in the python manuals I see no mentioning of a limit on the path length? Additionally since the path is constructed by the python installer itself, I assume that also the long path should work? If that long pathname would not work, then there is an error in the default installer logic that creates the pathname, and that should then be corrected asap.

You were correct, a colon was missing in "selfwrittenmodule.py". It "fell of" while I copied the lines of code from the real python code into the reply on this forum. The real python code has that colon.

I will test with the other 2 methods and come back here.

If I understand your comment correctly, then you save the "main.py" module and the "selfwrittenmodule.py" in the same Win10 folder. Initially I worked in that same way: the modules dedicated to the system I was writing and the "general use" modules were in the same folder, and everything worked fine. But then I wanted to change the way I organize the folders so that they beter reflect their content. Because "main.py" is code dedicated to a specific application, and "selfwrittenmodule.py" contains code that is intended to be used by many different systems (= NOT dedicated to one particular system), I would like to store them in 2 different folders: 1 for modules dedicated to the specific application, and one for "general use" modules.

To be sure that the length of the path is not causing the problem: does anybody here on this forum know about a limit on the pathname in python?
Reply
#8
(Aug-29-2019, 04:18 PM)wlp Wrote: If I understand your comment correctly, then you save the "main.py" module and the "selfwrittenmodule.py" in the same Win10 folder.
No and no,if look at my post again so if i save selfwrittenmodule.py in E:\div_code(which i added to sys.path as shown linked 2 times) then it work from any folder as long as Python37 is in Windows Environment Variables Path.

(Aug-29-2019, 04:18 PM)wlp Wrote: To be sure that the length of the path is not causing the problem: does anybody here on this forum know about a limit on the pathname in python?
You should uninstall and start a new install,as i think you messing this up Python 3.6/3.7 and pip installation under Windows .
This picture is in link,see what you should click to disable Path lenght.
[Image: tlZRit.jpg]
Reply
#9
I uninstalled and reinstalled Python 3.7.4, no change: I still get error messages telling that the to-be-imported module (selfwrittenmodule.py) is not found.

Then I did read your replies and links to previous messages in great detail and I have been "trying" a lot. Adding the "sitecustomize.py" code in the importing module, prior to importing "selfwrittenmodule.py", seems to resolve the problem.

But first I started testing on what the effect could be of adding the path to the folder that contains the "to-be-imported" module to the "System Variables" or/and to the "User Variables for actname". Whether it is added to the "path", or whether it is added using an added "PYTHONPATH" does not matter, the error re-occurs. After all these changes to the Windows "PATH", "pprint (sys.path)" does not show the manually added path. Python seems not to be influenced by changes to the "System Variables" of the Win PC. This seems to contradict the manuals of Python 3.7.4! Or I misunderstand the statements that explain how and where to change the "System Variables".

Now I first want to find a way to avoid hardcoding, in the "importing" module, the path to the folder that contains the "selfwrittenmodule.py" module (hardcoding external data should ALWAYS be avoided!). If I find a way to avoid that in an elegant way, I will let you know if and how that can be done.

Thanks for your patience and for your excellent suggestions.
Reply
#10
I don't know what your problem is now.

Can show a example with virtual environment,this is now build into Python trough venv.
Then use selfwrittenmodule.py my version that return something,your should also return something.
# selfwrittenmodule.py
def f1(x):
    x = x + 1
    return x
This will be "like a" new python installation,separate from OS Python.
Can maybe help looking at it from an other prescriptive.
# Test python
C:\data
λ python -c "import sys; print(sys.executable)"
C:\python37\python.exe

# Test pip
C:\data
λ pip -V
pip 19.2.1 from c:\python37\lib\site-packages\pip (python 3.7)

# Make virtual environment 
C:\data
λ python -m venv my_code

# Cd in
C:\data
λ cd my_code

# Activate see now (my_code)
C:\data\my_code
λ C:\data\my_code\Scripts\activate

# Now will python and pip only point to this folder,as long as active
(my_code) C:\data\my_code
λ python -c "import sys; print(sys.executable)"
C:\data\my_code\Scripts\python.exe

(my_code) C:\data\my_code
λ pip -V
pip 19.0.3 from c:\data\my_code\lib\site-packages\pip (python 3.7)
Now if save selfwrittenmodule.py in c:\data\my_code\lib\site-packages

See that work fine from my_code folder.
(my_code) C:\data\my_code
λ python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import selfwrittenmodule
>>>
>>> selfwrittenmodule.f1(999)
1000

# Placement
>>> selfwrittenmodule.__file__
'C:\\data\\my_code\\lib\\site-packages\\selfwrittenmodule.py'
>>> exit()
Install 3-party package eg Requests will aslo go into site-packages folder as all 3-party packages/modules dos.
(my_code) C:\data\my_code
λ pip install requests
Collecting requests .....  
Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3
Usage test:
(my_code) C:\data\my_code
λ python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
>>> r = requests.get('http://python-forum.io')
>>> r.status_code
200

>>> r.text[494:520]
'Welcome to python-forum.io'

# Placement
>>> requests.__file__
'C:\\data\\my_code\\lib\\site-packages\\requests\\__init__.py'
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Developed App in VS2019 - Help with pyinstaller fioranosnake 0 1,573 Nov-11-2019, 09:12 PM
Last Post: fioranosnake
  Modules issue, pip3 download modules only to pyhton3.5.2 not the latest 3.6.1 bmohanraj91 6 8,351 May-25-2017, 08:15 AM
Last Post: wavic

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020