Python Forum

Hello,

Earlier this year, I downloaded and modified a code that helps you to analyze your whatsapp chat data.
I used Anaconda and Jupyter notebook to run the codes and they ran without any issues.

My problem started when I upgraded my MacBook operating system to the new version, Catalina. My codes have stopped working since the upgrade, giving me different types of errors. I have done lot of research and have not been able tp sort this out. From what I have learned, the problem is not with the code but with the platforms of Anaconda and Jupyter notebooks. To tackle this, I uninstalled and installed Anaconda but that hasn't help.

For now, I have put my plans to fix Anaconda/Jupiter problem on MacBook Pro on hold. I have instead focus my attention to running the codes on Repl.it .

I took part of my code and ran it at Repl.it but got an error. I have researched and modified my code but that hasn't helped yet. Wondering if anyone can tell me why I am getting this error.

#Very good file. Third revision!
import os
import pandas as pd
pd.set_option('display.max_rows', 500)
import itertools
import datetime as dt
from matplotlib import pyplot as plt
from IPython import get_ipython
get_ipython().run_line_magic('matplotlib', 'inline')
import seaborn as sns

import re

from collections import Counter

import string
import emoji

import pickle
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.cbook as cbook

import pandas as pd 
import time
import sys
from wordcloud import WordCloud, STOPWORDS
from PIL import Image

files_groups = os.listdir('data/')
def read_history(file,conv_type):
    f = open('data/{}/{}'.format(conv_type,file), 'r',)
 
    # Feed the file text into findall(); it returns a list of all the found strings
    messages = re.findall('\[(\d+-\d+-\d+, \d+:\d+:\d+ [A-Z]*)\] (.*?): (.*)', f.read())
    f.close()

    
    #Convert list to a dataframe and name columns
    history = pd.DataFrame(messages,columns =['date','name','msg'])
    history['date'] = pd.to_datetime(history['date'],format="%Y-%m-%d, %I:%M:%S %p")
    history['date1'] = history['date'].apply(lambda x: x.date())
    history['msg_len'] = history['msg'].str.len()
    history['conv_name'] = file[19:-4]
    history['conv_name'] = file[19:-4]
# Get Media shared in the Message
    history['Media']=history['msg'].str.contains('omitted')
    return history
history['Media']
all = []

for file in files_groups:
    history = read_history(file,'')
    history['tipo'] = 'g'
    all.append(history)
    history = pd.concat(all).reset_index()
    history_clean = history[history['msg']!=' <Media omitted>'].sort_values(by=['conv_name','name','date1'])
history_clean.shape
history.columns

The error I am getting is this

Error:
Matplotlib created a temporary config/cache directory at /tmp/matplotlib-9v71tol1 because the default path (/config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Traceback (most recent call last):
  File "main.py", line 9, in <module>
    get_ipython().run_line_magic('matplotlib', 'inline')
AttributeError: 'NoneType' object has no attribute 'run_line_magic'

I got line 9 from stackoverflow. I am using this line to replace the original code of

%matplotlib inline

(this line was also giving me an error)

I am sorry there is lot of writing here. I am sure this is not clear to everyone. Please do ask questions and I will answer.

I would appreciate any help.

thank you

-YK

The ipython stuff looks like it's for showing matplot info in a jupyter notebook. I don't think repl.it will support ipython stuff.

(Dec-18-2020, 09:17 AM)bowlofred Wrote: [ -> ]The ipython stuff looks like it's for showing matplot info in a jupyter notebook. I don't think repl.it will support ipython stuff.

Thank you for pointing this out. I googled online and found a matplot code that works at repl.it :)

Everyone,

I want to move this thread forward. bowlofred pointed me in the right direction with my problem.

I now have another problem my code. Can you please have a look. I am pasting the code again and the error I am receiving.

#Very good file. Third revision!
import os
import pandas as pd
pd.set_option('display.max_rows', 500)
import itertools
import datetime as dt
from matplotlib import pyplot as plt
import matplotlib as mpl
mpl.use('Agg')
import numpy as np
import matplotlib.pyplot as plt
#%matplotlib inline
#from IPython import get_ipython
#get_ipython().run_line_magic('matplotlib', 'inline')
import seaborn as sns

import re

from collections import Counter

import string
import emoji

import pickle
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import matplotlib.cbook as cbook

import pandas as pd 
import time
import sys
from wordcloud import WordCloud, STOPWORDS
from PIL import Image

files_groups = os.listdir('data/')
def read_history(file,conv_type):
    f = open('data/{}/{}'.format(conv_type,file), 'r',)
 
    # Feed the file text into findall(); it returns a list of all the found strings
    messages = re.findall('\[(\d+-\d+-\d+, \d+:\d+:\d+ [A-Z]*)\] (.*?): (.*)', f.read())
    f.close()

    
    #Convert list to a dataframe and name columns
    history = pd.DataFrame(messages,columns =['date','name','msg'])
    history['date'] = pd.to_datetime(history['date'],format="%Y-%m-%d, %I:%M:%S %p")
    history['date1'] = history['date'].apply(lambda x: x.date())
    history['msg_len'] = history['msg'].str.len()
    history['conv_name'] = file[19:-4]
    history['conv_name'] = file[19:-4]
# Get Media shared in the Message
    history['Media']=history['msg'].str.contains('omitted')
    return history
history['Media']
all = []

for file in files_groups:
    history = read_history(file,'')
    history['tipo'] = 'g'
    all.append(history)
    history = pd.concat(all).reset_index()
    history_clean = history[history['msg']!=' <Media omitted>'].sort_values(by=['conv_name','name','date1'])
history_clean.shape
history.columns

Error:
Matplotlib created a temporary config/cache directory at /tmp/matplotlib-4oa8sevy because the default path (/config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Traceback (most recent call last):
  File "main.py", line 55, in <module>
    history['Media']
NameError: name 'history' is not defined
 

Please note, I have a folder called Data in Repl that I have created. Inside this folder is the text file that contains the whatsapp chat data.

Thank you in advance.

YK

The problem is quite self-explanatory really: on line 55, you try to use a variable called history which hasn't been declared in that scope. Is that line supposed to be there? Even if they variable had been declared, it wouldn't do anything useful as all you do is retrieve the value associated with 'Media' and then throw it away.

Unless you have special reasons, consider putting all your functions at the top of your code and then everything outside a function below it. Right now you have function definitions interspersed with other code. That may give the wrong impression of how the execution flow works.

(Dec-18-2020, 01:25 PM)ndc85430 Wrote: [ -> ]The problem is quite self-explanatory really: on line 55, you try to use a variable called history which hasn't been declared in that scope. Is that line supposed to be there? Even if they variable had been declared, it wouldn't do anything useful as all you do is retrieve the value associated with 'Media' and then throw it away.

The line is not suppose to be there. I added it to make sure everything is working ok upto that point.

I like you to know I did declare the history variable in line 46. Is that not sufficient. Line 46 pulls data from the text file. Is the problem that the code is not picking up the data from the text file.

Also, if I can add, if I remove line 55, I get similar error down at aline 64.

Kindly let me know your thoughts if possible.

thanks,

Yasser

As I said before, interspersing function definitions with other code will make it harder to follow the execution.

Line 46 is inside the function read_history(), so the variable assigned there is local to the function. It is not accessible outside the function. So when line 55 attempts to reference it, there is an error.

(Dec-19-2020, 07:02 AM)bowlofred Wrote: [ -> ]As I said before, interspersing function definitions with other code will make it harder to follow the execution.

Line 46 is inside the function read_history(), so the variable assigned there is local to the function. It is not accessible outside the function. So when line 55 attempts to reference it, there is an error.

I don't know how to fix this. sorry. Should I break read_history() function.

In this case you only have one line above it. Move that line below the function, and leave a couple of blank lines below the function to serve as a visual break.

Now you can start reading your code execution from that point (and mentally ignore the stuff inside the function).

You can't use a variable (like history) unless you've assigned a value to it. The assignment inside read_history() doesn't count.

You call the function later in the for loop, and assign history at that time (line 59). You can refer to or print history anytime after that.