Python Forum
I'm stuck with the reinforcement learning function in my Code
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
I'm stuck with the reinforcement learning function in my Code
#1
Sad 
Hello everybody! I've got a problem with my code for my "VoiceAssistent", the normal code with speak function works well and makes everything right but i tried to implement the reinforcement function in it and i need to get the Environment class but it's not really defined in the code and i cannot find a file with the name Environment.py like the GPT AI said it should be... Since i made from environment import environment, it comes with the error:

Error:
Exception has occurred: AttributeError 'Environment' object has no attribute 'observation_space' KeyError: 'observation_space' During handling of the above exception, another exception occurred: File "C:\Users\oski1\AI + Algorithm (Reinforcement learning).py", line 22, in <module> Q = np.zeros([env.observation_space.n, env.action_space.n]) ^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'Environment' object has no attribute 'observation_space'
Could anyone eventually help me further? The code were:

import webbrowser
import openai
import pyttsx3
import speech_recognition as sr
from gtts import gTTS
import os
import subprocess as sp
from random import choice
import requests
import wikipedia
import pywhatkit as kit
from email.message import EmailMessage
import smtplib
import decouple
import numpy as np
from environment import Environment ,is_yesish

env = Environment()
alpha = 0.1  # learning rate
gamma = 0.9  # discount factor

Q = np.zeros([env.observation_space.n, env.action_space.n])

# Iterate over episodes
for episode in range(1, 1000):
    state = env.reset()
    done = False
    rewards = 0

    while not done:
        action = np.argmax(Q[state, :] + np.random.randn(1, env.action_space.n) * (1. / (episode + 1)))
        new_state, reward, done, info = env.step(action)

        Q[state, action] = Q[state, action] + alpha * (reward + gamma * np.max(Q[new_state, :]) - Q[state, action])
        rewards += reward
        state = new_state

    print('Episode: {}, Rewards: {}'.format(episode, rewards))

USERNAME = ('Oscar')
BOTNAME = ('Samantha')

paths = {
    'notepad': "C:\\Program Files\\Notepad++\\notepad++.exe",
    'discord': "C:\\Users\\ashut\\AppData\\Local\\Discord\\app-1.0.9003\\Discord.exe",
    'calculator': "C:\\Windows\\System32\\calc.exe"
}

def open_browser():
    webbrowser.open('https://www.google.com')

def speak(text):
    engine.say(text)
    engine.runAndWait()

def open_camera():
    sp.run('start microsoft.windows.camera:', shell=True)

def open_notepad():
    os.startfile(paths['notepad'])

def open_discord():
    os.startfile(paths['discord'])

def open_cmd():
    os.system('start cmd')

def open_calculator():
    sp.Popen(paths['calculator'])
    
def get_weather_report(city):
    res = requests.get(
        f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={OPENWEATHER_APP_ID}&units=metric").json()
    weather = res["weather"][0]["main"]
    temperature = res["main"]["temp"]
    feels_like = res["main"]["feels_like"]
    return weather, f"{temperature}℃", f"{feels_like}℃"

EMAIL = ("MY EMAIL")
PASSWORD = ("MY PASSWORD")


def send_email(receiver_address, subject, message):
    search_url = f"https://outlook.live.com/owa/"
    webbrowser.open(search_url)
    try:
        email = EmailMessage()
        email['To'] = receiver_address
        email["Subject"] = subject
        email['From'] = EMAIL
        email.set_content(message)
        s = smtplib.SMTP("smtp.gmail.com", 587)
        s.starttls()
        s.login(EMAIL, PASSWORD)
        s.send_message(email)
        s.close()
        return True
    except Exception as e:
        print(e)
        return False
    
def search_on_google(query):
    search_url = f"https://www.google.com/search?q={query}"
    webbrowser.open(search_url, {query})
    kit.search(query)
    
def play_on_youtube(video):
    search_url = f"https://www.youtube.com/results?search_query={video}"
    webbrowser.open(search_url)
    kit.playonyt(video)
    
def search_on_wikipedia(query):
    search_url = f"https://en.wikipedia.org/wiki/search?q={query}"
    webbrowser.open(search_url)
    results = wikipedia.summary(query, sentences=2)
    return results

def find_my_ip():
    search_url = f"https://api64.ipify.org?format=json"
    webbrowser.open(search_url)
    ip_address = requests.get('https://api64.ipify.org?format=json').json()
    return ip_address["ip"]

def get_random_joke():
    search_url = f"https://icanhazdadjoke.com/"
    webbrowser.open(search_url)
    headers = {
        'Accept': 'application/json'
    }
    res = requests.get("https://icanhazdadjoke.com/", headers=headers).json()
    return res["joke"]

def get_random_advice():
    res = requests.get("https://api.adviceslip.com/advice").json()
    return res['slip']['advice']


engine = pyttsx3.init()

# Get the available voices
voices = engine.getProperty('voices')

# Print the available voices
for voice in voices:
    print("Voice:")
    print(" - ID: ", voice.id)
    print(" - Name: ", voice.name)
    print(" - Languages: ", voice.languages)
    print(" - Gender: ", voice)
    print(" - Age: ", voice.age)
    
    # Set the voice ID you want to use
voice_id = 'com.apple.speech.synthesis.voice.samantha'

# Set the voice property
engine.setProperty('voice', voice_id)

# Set your OpenAI API key
openai.api_key = "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"

# Initialize the Text-to-Speech engine
engine = pyttsx3.init()

def transcribe_audio_to_text(filename):
    recognizer = sr.Recognizer()
    with sr.AudioFile(filename) as source:
        audio = recognizer.record(source)
        try:
            return recognizer.recognize_google(audio)
        except sr.UnknownValueError:
            print("Google Speech Recognition could not understand audio")
        except sr.RequestError as e:
            print(f"Could not request results from Google Speech Recognition service; {e}")

def generate_response(prompt):
    response = openai.Completion.create(
        engine="text-davinci-003",
        prompt=prompt,
        max_tokens=4000,
        n=1,
        stop=None,
        temperature=0.5,
    )
    return response.choices[0].text

def speak_text(text):
    engine.say(text)
    engine.runAndWait()

def main():
    while True:
        
        # Wait for the user to say "Hello"
        print("Say 'Hello' to start recording your question...")
        with sr.Microphone() as source:
            recognizer = sr.Recognizer()
            audio = recognizer.listen(source)
            try:
                transcription = recognizer.recognize_google(audio)
                if transcription.lower() == "hello":
                    # Record audio
                    filename = "input.wav"
                    print("Say your question...")
                    with sr.Microphone() as source:
                        recognizer = sr.Recognizer()
                        source.pause_threshold = 1
                        audio = recognizer.listen(source, phrase_time_limit=None, timeout=None)
                        with open(filename, "wb") as f:
                            f.write(audio.get_wav_data())

                    # Transcribe audio to text
                    text = transcribe_audio_to_text(filename)
                    if text:
                        print(f"You said: {text}")

                        # Generate response using GPT-3
                        response = generate_response(text)
                        print(f"GPT-3 says: {response}")

                        # Record audio with gTTS for speech output
                        tts = gTTS(text=response, lang='en')
                        tts.save("sample.mp3")

                        # Read response using text to speech
                        speak_text(response)
                        
                        if 'open notepad' in text:
                            open_notepad()
                        elif 'open discord' in text:
                            open_discord()
                        elif 'open command prompt' in text or 'open cmd' in text:
                            open_cmd()
                        elif 'open camera' in text:
                            open_camera()
                        elif 'open calculator' in text:
                            open_calculator()
                        elif 'find my ip address' in text:
                            ip_address = find_my_ip()
                            speak(f'Your IP Address is {ip_address}.\n For your convenience, I am printing it on the screen sir.')
                            print(f'Your IP Address is {ip_address}')
                        elif 'search on wikipedia' in text:
                            speak('What do you want to search on Wikipedia, sir?')
                            search_text = input().lower()
                            results = search_on_wikipedia(search_text)
                            speak(f"According to Wikipedia, {results}")
                            speak("For your convenience, I am printing it on the screen sir.")
                            print(results)
                        elif 'search on youtube' in text:
                            speak('What do you want to play on Youtube, sir?')
                            video = input().lower()
                            play_on_youtube(video)
                        elif 'search on google' in text:
                            speak('What do you want to search on Google, sir?')
                            query = input().lower()
                            search_on_google(query)
                        elif "send an email" in text:
                            speak("On what email address do I send sir? Please enter in the console: ")
                            receiver_address = input("Enter email address: ")
                            speak("What should be the subject sir?")
                            subject = input().capitalize()
                            speak("What is the message sir?")
                            message = input().capitalize()
                            if send_email(receiver_address, subject, message):
                                speak("I've sent the email sir.")
                            else:
                                speak("Something went wrong while I was sending the mail. Please check the error logs sir.")
                        elif 'I wanna hear a joke' in text:
                            speak(f"Hope you like this one sir")
                            joke = get_random_joke()
                            speak(joke)
                            speak("For your convenience, I am printing it on the screen sir.")
                            print(joke)
                        elif "advice" in text:
                            speak(f"Here's an advice for you, sir")
                            advice = get_random_advice()
                            speak(advice)
                            speak("For your convenience, I am printing it on the screen sir.")
                            print(advice)
                        elif 'weather' in text:
                            ip_address = find_my_ip()
                            city = requests.get(f"https://ipapi.co/{ip_address}/city/").text
                            speak(f"Getting weather report for your city {city}")
                            weather, temperature, feels_like = get_weather_report(city)
                            speak(f"The current temperature is {temperature}, but it feels like {feels_like}")
                            speak(f"Also, the weather report talks about {weather}")
                            speak("For your convenience, I am printing it on the screen sir.")
                            print(f"Description: {weather}\nTemperature: {temperature}\nFeels like: {feels_like}")         
            except sr.UnknownValueError:
                print("Google Speech Recognition could not understand audio")
            except sr.RequestError as e:
                print(f"Could not request results from Google Speech Recognition service; {e}")

if __name__ == "__main__":
    main()
Thank for your time and i hope anyone can help me further!
buran write May-20-2023, 08:17 PM:
Please, use proper tags when post code, traceback, output, etc. This time I have added tags for you.
See BBcode help for more info.
Reply
#2
which package is environment?
I found https://pypi.org/project/environment/ on PyPI, which is rather old one.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
Thank you for your time, i couldn't find the environment package for my code, so i got one from gym, i'm not a professional but i try what i can for my HobbyProject and i hope i can make it better with more knowledge :) After a few hours on my laptop i made some steps further, now i need to learn more to come further with my code and all the functions that i want to build in, it works and it makes something but i'm not really sure what, it's able to speak and to reply to all my answers and i can get advices and it can mathe, it goes so far, that it search for codes when i ask for, i got them at the Terminal of visual studio and it reads those too. But it still cannot make everything i want, cause it's not able to search for a video on youtube or make a search on google, i must to work a liltle bit more on that code untill it works with all the functions of a "VoiceAssisst". I could get the environment with CartPoleEnv from gym. I'm making that as my Hobby and to learn so much stuff i can, if someone likes it or has ideas for these code, i'm very happy to hear about it! Now that's the actually "working" code:

import webbrowser
import openai
import pyttsx3
import speech_recognition as sr
from gtts import gTTS
import os
import subprocess as sp
from random import choice
import requests
import wikipedia
import pywhatkit as kit
from email.message import EmailMessage
import smtplib
import decouple
import numpy as np
import gym
from gym.envs.classic_control import CartPoleEnv

env = CartPoleEnv()
alpha = 0.1 # learning rate
gamma = 0.9 # discount factor

# Update the lines to get the dimensions of the observation and action spaces
obs_space_dim = env.observation_space.shape[0]
action_space_dim = env.action_space.n

Q = np.zeros((obs_space_dim, action_space_dim))

# Iterate over episodes
for episode in range(1, 1000):
state = env.reset()
done = False
rewards = 0

while not done:
state_str = str(tuple(state))
state_hash = hash(state_str)
state_index = state_hash % obs_space_dim # Map the state hash to a valid index

action = np.argmax(Q[state_index, :] + np.random.randn(action_space_dim) * (1. / (episode + 1)))
result = env.step(action)
new_state, reward, done, _ = result[:4]

new_state_str = str(tuple(new_state))
new_state_hash = hash(new_state_str)
new_state_index = new_state_hash % obs_space_dim # Map the new state hash to a valid index

Q[state_index, action] = Q[state_index, action] + alpha * (
reward + gamma * np.max(Q[new_state_index, :]) - Q[state_index, action]
)
rewards += reward
state = new_state

print('Episode: {}, Rewards: {}'.format(episode, rewards))

USERNAME = ('Oscar')
BOTNAME = ('Samantha')

paths = {
'notepad': "C:\\Program Files\\Notepad++\\notepad++.exe",
'discord': "C:\\Users\\ashut\\AppData\\Local\\Discord\\app-1.0.9003\\Discord.exe",
'calculator': "C:\\Windows\\System32\\calc.exe"
}

def open_browser():
webbrowser.open('https://www.google.com')

def speak(text):
engine.say(text)
engine.runAndWait()

def open_camera():
sp.run('start microsoft.windows.camera:', shell=True)

def open_notepad():
os.startfile(paths['notepad'])

def open_discord():
os.startfile(paths['discord'])

def open_cmd():
os.system('start cmd')

def open_calculator():
sp.Popen(paths['calculator'])

def get_weather_report(city):
res = requests.get(
f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={OPENWEATHER_APP_ID}&units=metric").json()
weather = res["weather"][0]["main"]
temperature = res["main"]["temp"]
feels_like = res["main"]["feels_like"]
return weather, f"{temperature}℃", f"{feels_like}℃"

EMAIL = ("YOUR EMAIL")
PASSWORD = ("YOUR PASSWORD")


def send_email(receiver_address, subject, message):
search_url = f"https://outlook.live.com/owa/"
webbrowser.open(search_url)
try:
email = EmailMessage()
email['To'] = receiver_address
email["Subject"] = subject
email['From'] = EMAIL
email.set_content(message)
s = smtplib.SMTP("smtp.gmail.com", 587)
s.starttls()
s.login(EMAIL, PASSWORD)
s.send_message(email)
s.close()
return True
except Exception as e:
print(e)
return False

def search_on_google(query):
search_url = f"https://www.google.com/search?q={query}"
webbrowser.open(search_url, {query})
kit.search(query)

def play_on_youtube(video):
search_url = f"https://www.youtube.com/results?search_query={video}"
webbrowser.open(search_url)
kit.playonyt(video)

def search_on_wikipedia(query):
search_url = f"https://en.wikipedia.org/wiki/search?q={query}"
webbrowser.open(search_url)
results = wikipedia.summary(query, sentences=2)
return results

def find_my_ip():
search_url = f"https://api64.ipify.org?format=json"
webbrowser.open(search_url)
ip_address = requests.get('https://api64.ipify.org?format=json').json()
return ip_address["ip"]

def get_random_joke():
search_url = f"https://icanhazdadjoke.com/"
webbrowser.open(search_url)
headers = {
'Accept': 'application/json'
}
res = requests.get("https://icanhazdadjoke.com/", headers=headers).json()
return res["joke"]

def get_random_advice():
res = requests.get("https://api.adviceslip.com/advice").json()
return res['slip']['advice']


engine = pyttsx3.init()

# Get the available voices
voices = engine.getProperty('voices')

# Print the available voices
for voice in voices:
print("Voice:")
print(" - ID: ", voice.id)
print(" - Name: ", voice.name)
print(" - Languages: ", voice.languages)
print(" - Gender: ", voice)
print(" - Age: ", voice.age)

# Set the voice ID you want to use
voice_id = 'com.apple.speech.synthesis.voice.samantha'

# Set the voice property
engine.setProperty('voice', voice_id)

# Set your OpenAI API key
openai.api_key = "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"

# Initialize the Text-to-Speech engine
engine = pyttsx3.init()

def transcribe_audio_to_text(filename):
recognizer = sr.Recognizer()
with sr.AudioFile(filename) as source:
audio = recognizer.record(source)
try:
return recognizer.recognize_google(audio)
except sr.UnknownValueError:
print("Google Speech Recognition could not understand audio")
except sr.RequestError as e:
print(f"Could not request results from Google Speech Recognition service; {e}")

def generate_response(prompt):
response = openai.Completion.create(
engine="text-davinci-003",
prompt=prompt,
max_tokens=4000,
n=1,
stop=None,
temperature=0.5,
)
return response.choices[0].text

def speak_text(text):
engine.say(text)
engine.runAndWait()

def main():
while True:

# Wait for the user to say "Hello"
print("Say 'Hello' to start recording your question...")
with sr.Microphone() as source:
recognizer = sr.Recognizer()
audio = recognizer.listen(source)
try:
transcription = recognizer.recognize_google(audio)
if transcription.lower() == "hello":
# Record audio
filename = "input.wav"
print("Say your question...")
with sr.Microphone() as source:
recognizer = sr.Recognizer()
source.pause_threshold = 1
audio = recognizer.listen(source, phrase_time_limit=None, timeout=None)
with open(filename, "wb") as f:
f.write(audio.get_wav_data())

# Transcribe audio to text
text = transcribe_audio_to_text(filename)
if text:
print(f"You said: {text}")

# Generate response using GPT-3
response = generate_response(text)
print(f"GPT-3 says: {response}")

# Record audio with gTTS for speech output
tts = gTTS(text=response, lang='en')
tts.save("sample.mp3")

# Read response using text to speech
speak_text(response)

if 'open notepad' in text:
open_notepad()
elif 'open discord' in text:
open_discord()
elif 'open command prompt' in text or 'open cmd' in text:
open_cmd()
elif 'open camera' in text:
open_camera()
elif 'open calculator' in text:
open_calculator()
elif 'find my ip address' in text:
ip_address = find_my_ip()
speak(f'Your IP Address is {ip_address}.\n For your convenience, I am printing it on the screen sir.')
print(f'Your IP Address is {ip_address}')
elif 'search on wikipedia' in text:
speak('What do you want to search on Wikipedia, sir?')
search_text = input().lower()
results = search_on_wikipedia(search_text)
speak(f"According to Wikipedia, {results}")
speak("For your convenience, I am printing it on the screen sir.")
print(results)
elif 'search on youtube' in text:
speak('What do you want to play on Youtube, sir?')
video = input().lower()
play_on_youtube(video)
elif 'search on google' in text:
speak('What do you want to search on Google, sir?')
query = input().lower()
search_on_google(query)
elif "send an email" in text:
speak("On what email address do I send sir? Please enter in the console: ")
receiver_address = input("Enter email address: ")
speak("What should be the subject sir?")
subject = input().capitalize()
speak("What is the message sir?")
message = input().capitalize()
if send_email(receiver_address, subject, message):
speak("I've sent the email sir.")
else:
speak("Something went wrong while I was sending the mail. Please check the error logs sir.")
elif 'I wanna hear a joke' in text:
speak(f"Hope you like this one sir")
joke = get_random_joke()
speak(joke)
speak("For your convenience, I am printing it on the screen sir.")
print(joke)
elif "advice" in text:
speak(f"Here's an advice for you, sir")
advice = get_random_advice()
speak(advice)
speak("For your convenience, I am printing it on the screen sir.")
print(advice)
elif 'weather' in text:
ip_address = find_my_ip()
city = requests.get(f"https://ipapi.co/{ip_address}/city/").text
speak(f"Getting weather report for your city {city}")
weather, temperature, feels_like = get_weather_report(city)
speak(f"The current temperature is {temperature}, but it feels like {feels_like}")
speak(f"Also, the weather report talks about {weather}")
speak("For your convenience, I am printing it on the screen sir.")
print(f"Description: {weather}\nTemperature: {temperature}\nFeels like: {feels_like}")
except sr.UnknownValueError:
print("Google Speech Recognition could not understand audio")
except sr.RequestError as e:
print(f"Could not request results from Google Speech Recognition service; {e}")

if __name__ == "__main__":
main()
buran write May-21-2023, 02:49 AM:
Please post all code, output and errors (it it's entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Putting code into a function breaks its functionality, though the code is identical! PCesarano 1 2,003 Apr-05-2021, 05:40 PM
Last Post: deanhystad
  Question on None function in a machine learning algorithm Livingstone1337 1 2,363 Mar-17-2021, 10:12 PM
Last Post: supuflounder
  a complicated randomized function which I'm stuck on jojo77m 11 3,558 Aug-27-2020, 09:24 AM
Last Post: DPaul
  Translation of R Code to Python for Statistical Learning Course SterlingAesir 2 2,142 Aug-27-2020, 08:46 AM
Last Post: ndc85430
  Learning python SyntaxError: 'return' outside function Grale1953 3 2,526 Aug-03-2020, 06:55 AM
Last Post: buran
  learning to code python nhan 5 2,557 May-23-2020, 03:35 PM
Last Post: nhan
  my function is stuck on loop - even when it not supposed to be korenron 2 2,365 May-26-2019, 12:31 PM
Last Post: korenron
  Learning python, stuck on some code. stanceworksv8 2 3,489 Apr-02-2019, 01:51 AM
Last Post: stanceworksv8
  Still learning - code efficiency, which of these is better? shelzmike 2 3,289 Oct-14-2017, 04:47 AM
Last Post: shelzmike
  I finsh the basics of python but I'm stuck on how to link code to GUI Qubayel 5 4,359 Apr-04-2017, 07:18 AM
Last Post: Ofnuts

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020