Posts: 13
Threads: 8
Joined: Apr 2020
I have four lists named designName,creatorName,fabric_names and data the data list consist of multiple duplicate values which I want to map it to it's asscociated design name and creator name like:
Output: Catching Fireflies thestorysmith FABRIC_PETAL_SIGNATURE_COTTON 1.75 10.58 18.22
Catching Fireflies thestorysmith FABRIC_SATIN 1.75 11.85 19.71
Catching Fireflies thestorysmith FABRIC_COTTON_POPLIN_BRAVA 1.75 11.85 19.71....
Spoonflower Color Map spoonflower_help FABRIC_PETAL_SIGNATURE_COTTON N/A N/A 16.4
Spoonflower Color Map spoonflower_help FABRIC_SATIN N/A N/A 17.74
Spoonflower Color Map spoonflower_help FABRIC_COTTON_POPLIN_BRAVA N/A N/A 17.74...
Night Sky Stars Midnight Blue at_the_cottage FABRIC_PETAL_SIGNATURE_COTTON 1.75 10.58 18.22
Night Sky Stars Midnight Blue at_the_cottage FABRIC_SATIN 1.75 11.85 19.71
Night Sky Stars Midnight Blue at_the_cottage FABRIC_COTTON_POPLIN_BRAVA 1.75 11.85 19.71
Data:
desigName = ['Catching Fireflies', 'Spoonflower Color Map', 'Night Sky Stars Midnight Blue']
creatorName = ['thestorysmith', 'spoonflower_help', 'at_the_cottage']
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
data = [('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),... ('FABRIC_SATIN', 1.75, 11.85, 19.71),('FABRIC_SATIN', 1.75, 11.85, 19.71),('FABRIC_SATIN', 1.75, 11.85, 19.71),... ('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),...]I am trying to figure out the best way to turn this data into one ordered dict like the following.
Output: {('Catching Fireflies', 'thestorysmith'): {'fabric_name_00': 'FABRIC_PETAL_SIGNATURE_COTTON', 'test_swatch_meter_00': 1.75, 'fat_quarter_meter_00': 10.58, 'meter_00': 18.22, 'fabric_name_01': 'FABRIC_SATIN', 'test_swatch_meter_01': 1.75, 'fat_quarter_meter_01': 11.85, 'meter_01': 19.71, 'fabric_name_02': 'FABRIC_COTTON_POPLIN_BRAVA', 'test_swatch_meter_02': 1.75, 'fat_quarter_meter_02': 11.85, 'meter_02': 19.71}}
I've tried so far:
for fab in fabric_names:
print(fab)
for name, creator in zip(designName, creatorName):
for fab_type in fabric_names:
Design_Name = name
Creator_Name = creator
test_swatch_meter = data[1]
fat_quarter_meter = data[2]
meter = data[3]
if (name, creator) not in items_dict.keys():
items_dict[(name, creator)] = {}
itemCount = len(items_dict[(name, creator)].values()) / 4
items_dict[(name, creator)].update({'fabric_name_%02d' %itemCount: fab_type,
'test_swatch_meter_%02d' %itemCount: test_swatch_meter,
'fat_quarter_meter_%02d' %itemCount: fat_quarter_meter,
'meter_%02d' %itemCount: meter})but cannot get it to format like above.
Posts: 11,873
Threads: 474
Joined: Sep 2016
Jun-23-2021, 05:21 PM
(This post was last modified: Jun-24-2021, 01:24 AM by Larz60+.)
you are showing ellipsis in your data. In the following solution, I am assuming that this is an error.
from CreateDict import CreateDict
desigName = ['Catching Fireflies', 'Spoonflower Color Map', 'Night Sky Stars Midnight Blue']
creatorName = ['thestorysmith', 'spoonflower_help', 'at_the_cottage']
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
data = [
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71)
]
def create_new_dict():
newdict = {}
cd = CreateDict()
for n, item in enumerate(desigName):
dnode = cd.add_node(newdict, (item, creatorName[n]))
cd.add_cell(dnode, f"fabric_name{n:02}", fabric_names[n])
for element in data:
if element[0] == fabric_names[n]:
cd.add_cell(dnode, f"test_swatch_meter_{n:02}", element[1])
cd.add_cell(dnode, f"fat_quarter_meter__{n:02}", element[2])
cd.add_cell(dnode, f"meter_{n:02}", element[3])
break
cd.display_dict(newdict)
create_new_dict() this creates (which is what you ask for):
Output: {
('Catching Fireflies', 'thestorysmith'): {
'fabric_name00': 'FABRIC_PETAL_SIGNATURE_COTTON',
'test_swatch_meter_00': 1.75,
'fat_quarter_meter__00': 10.58,
'meter_00': 18.22
},
('Spoonflower Color Map', 'spoonflower_help'): {
'fabric_name01': 'FABRIC_SATIN',
'test_swatch_meter_01': 1.75,
'fat_quarter_meter__01': 11.85, 'meter_01': 19.71
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'fabric_name02': 'FABRIC_COTTON_POPLIN_BRAVA',
'test_swatch_meter_02': 1.75,
'fat_quarter_meter__02': 11.85, 'meter_02': 19.71
}
}
Also, why is the data repeated for each fabric? I take care of this, but only one entry is necessary, and would simplify the code
To make a better dictionary:
if data were: data = [('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22)('FABRIC_SATIN', 1.75, 11.85, 19.71),('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71)]
then this code would work:
from CreateDict import CreateDict
desigName = ['Catching Fireflies', 'Spoonflower Color Map', 'Night Sky Stars Midnight Blue']
creatorName = ['thestorysmith', 'spoonflower_help', 'at_the_cottage']
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
data = [
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71)
]
def create_new_dict():
newdict = {}
cd = CreateDict()
for n, item in enumerate(desigName):
dnode = cd.add_node(newdict, (item, creatorName[n]))
cnode = cd.add_node(dnode, fabric_names[n])
element = data[n]
cd.add_cell(cnode, f"test_swatch_meter", element[1])
cd.add_cell(cnode, f"fat_quarter_meter", element[2])
cd.add_cell(cnode, f"meter_", element[3])
return newdict
newdict = create_new_dict()
print(newdict)which creates a much better dictionary:
Output: {
('Catching Fireflies', 'thestorysmith'): {
'FABRIC_PETAL_SIGNATURE_COTTON': {
'test_swatch_meter': 1.75,
'fat_quarter_meter': 10.58,
'meter_': 18.22
}
},
('Spoonflower Color Map', 'spoonflower_help'): {
'FABRIC_SATIN': {
'test_swatch_meter': 1.75,
'fat_quarter_meter': 11.85,
'meter_': 19.71
}
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'FABRIC_COTTON_POPLIN_BRAVA': {
'test_swatch_meter': 1.75,
'fat_quarter_meter': 11.85,
'meter_': 19.71
}
}
}
For either of these solutions, you will have to add the following module to the same directory as your script
you are free to use this as I have published it before please use the name indicated:
CreateDict.py:
#
import os
class CreateDict:
"""
Generic Software tools used by Trailmapper.
CreateDict.py - Contains methods to simplify node and cell creation within
a dictionary
Usage:
The best way to learn what can be done is to examine the testit function
included in this module.
new_dict(dictname) - Creates a new dictionary instance with the name
contained in dictname
add_node(parent, nodename) - Creates a new node (nested dictionary)
named in nodename, in parent dictionary.
add_cell(nodename, cellname, value) - Creates a leaf node within node
named in nodename, with a cell name of cellname, and value of value.
display_dict(dictname) - Recursively displays a nested dictionary.
Requirements:
Trailmapper software:
None
Python standard library:
os
Author: Larry McCaig -- May 2019.
"""
def __init__(self):
os.chdir(os.path.abspath(os.path.dirname(__file__)))
def new_dict(self, dictname):
setattr(self, dictname, {})
def add_node(self, parent, nodename):
node = parent[nodename] = {}
return node
def add_cell(self, nodename, cellname, value):
cell = nodename[cellname] = value
return cell
def display_dict(self, dictname, level=0):
indent = " " * (4 * level)
for key, value in dictname.items():
if isinstance(value, dict):
print(f'\n{indent}{key}')
level += 1
self.display_dict(value, level)
else:
print(f'{indent}{key}: {value}')
if level > 0:
level -= 1
def testit():
# instantiate class
cd = CreateDict()
# create new dictionary named CityList
cd.new_dict('CityList')
# add node Boston
boston = cd.add_node(cd.CityList, 'Boston')
# add sub node Resturants
bos_resturants = cd.add_node(boston, 'Resturants')
# Add subnode 'Spoke Wine Bar' to parent bos_resturants
spoke = cd.add_node(bos_resturants, 'Spoke Wine Bar')
cd.add_cell(spoke, 'Addr1', '89 Holland St')
cd.add_cell(spoke, 'City', 'Sommerville')
cd.add_cell(spoke, 'Addr1', '02144')
cd.add_cell(spoke, 'Phone', '617-718-9463')
# Add subnode 'Highland Kitchen' to parent bos_resturants
highland = cd.add_node(bos_resturants, 'Highland Kitchen')
cd.add_cell(highland, 'Addr1', '150 Highland Ave')
cd.add_cell(highland, 'City', 'Sommerville')
cd.add_cell(highland, 'ZipCode', '02144')
cd.add_cell(highland, 'Phone', '617-625-1131')
# display dictionary
print(f'\nCityList Dictionary')
cd.display_dict(cd.CityList)
print(f'\nraw data: {cd.CityList}')
if __name__ == '__main__':
testit()
Posts: 13
Threads: 8
Joined: Apr 2020
Jun-25-2021, 04:43 AM
(This post was last modified: Jun-25-2021, 04:43 AM by rhat398.)
(Jun-23-2021, 05:21 PM)Larz60+ Wrote: you are showing ellipsis in your data. In the following solution, I am assuming that this is an error.
from CreateDict import CreateDict
desigName = ['Catching Fireflies', 'Spoonflower Color Map', 'Night Sky Stars Midnight Blue']
creatorName = ['thestorysmith', 'spoonflower_help', 'at_the_cottage']
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
data = [
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71)
]
def create_new_dict():
newdict = {}
cd = CreateDict()
for n, item in enumerate(desigName):
dnode = cd.add_node(newdict, (item, creatorName[n]))
cd.add_cell(dnode, f"fabric_name{n:02}", fabric_names[n])
for element in data:
if element[0] == fabric_names[n]:
cd.add_cell(dnode, f"test_swatch_meter_{n:02}", element[1])
cd.add_cell(dnode, f"fat_quarter_meter__{n:02}", element[2])
cd.add_cell(dnode, f"meter_{n:02}", element[3])
break
cd.display_dict(newdict)
create_new_dict() this creates (which is what you ask for):
Output: {
('Catching Fireflies', 'thestorysmith'): {
'fabric_name00': 'FABRIC_PETAL_SIGNATURE_COTTON',
'test_swatch_meter_00': 1.75,
'fat_quarter_meter__00': 10.58,
'meter_00': 18.22
},
('Spoonflower Color Map', 'spoonflower_help'): {
'fabric_name01': 'FABRIC_SATIN',
'test_swatch_meter_01': 1.75,
'fat_quarter_meter__01': 11.85, 'meter_01': 19.71
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'fabric_name02': 'FABRIC_COTTON_POPLIN_BRAVA',
'test_swatch_meter_02': 1.75,
'fat_quarter_meter__02': 11.85, 'meter_02': 19.71
}
}
Also, why is the data repeated for each fabric? I take care of this, but only one entry is necessary, and would simplify the code
To make a better dictionary:
if data were: data = [('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22)('FABRIC_SATIN', 1.75, 11.85, 19.71),('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71),('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71)]
then this code would work:
from CreateDict import CreateDict
desigName = ['Catching Fireflies', 'Spoonflower Color Map', 'Night Sky Stars Midnight Blue']
creatorName = ['thestorysmith', 'spoonflower_help', 'at_the_cottage']
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
data = [
('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22),
('FABRIC_SATIN', 1.75, 11.85, 19.71),
('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71)
]
def create_new_dict():
newdict = {}
cd = CreateDict()
for n, item in enumerate(desigName):
dnode = cd.add_node(newdict, (item, creatorName[n]))
cnode = cd.add_node(dnode, fabric_names[n])
element = data[n]
cd.add_cell(cnode, f"test_swatch_meter", element[1])
cd.add_cell(cnode, f"fat_quarter_meter", element[2])
cd.add_cell(cnode, f"meter_", element[3])
return newdict
newdict = create_new_dict()
print(newdict)which creates a much better dictionary:
Output: {
('Catching Fireflies', 'thestorysmith'): {
'FABRIC_PETAL_SIGNATURE_COTTON': {
'test_swatch_meter': 1.75,
'fat_quarter_meter': 10.58,
'meter_': 18.22
}
},
('Spoonflower Color Map', 'spoonflower_help'): {
'FABRIC_SATIN': {
'test_swatch_meter': 1.75,
'fat_quarter_meter': 11.85,
'meter_': 19.71
}
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'FABRIC_COTTON_POPLIN_BRAVA': {
'test_swatch_meter': 1.75,
'fat_quarter_meter': 11.85,
'meter_': 19.71
}
}
}
For either of these solutions, you will have to add the following module to the same directory as your script
you are free to use this as I have published it before please use the name indicated:
CreateDict.py:
#
import os
class CreateDict:
"""
Generic Software tools used by Trailmapper.
CreateDict.py - Contains methods to simplify node and cell creation within
a dictionary
Usage:
The best way to learn what can be done is to examine the testit function
included in this module.
new_dict(dictname) - Creates a new dictionary instance with the name
contained in dictname
add_node(parent, nodename) - Creates a new node (nested dictionary)
named in nodename, in parent dictionary.
add_cell(nodename, cellname, value) - Creates a leaf node within node
named in nodename, with a cell name of cellname, and value of value.
display_dict(dictname) - Recursively displays a nested dictionary.
Requirements:
Trailmapper software:
None
Python standard library:
os
Author: Larry McCaig -- May 2019.
"""
def __init__(self):
os.chdir(os.path.abspath(os.path.dirname(__file__)))
def new_dict(self, dictname):
setattr(self, dictname, {})
def add_node(self, parent, nodename):
node = parent[nodename] = {}
return node
def add_cell(self, nodename, cellname, value):
cell = nodename[cellname] = value
return cell
def display_dict(self, dictname, level=0):
indent = " " * (4 * level)
for key, value in dictname.items():
if isinstance(value, dict):
print(f'\n{indent}{key}')
level += 1
self.display_dict(value, level)
else:
print(f'{indent}{key}: {value}')
if level > 0:
level -= 1
def testit():
# instantiate class
cd = CreateDict()
# create new dictionary named CityList
cd.new_dict('CityList')
# add node Boston
boston = cd.add_node(cd.CityList, 'Boston')
# add sub node Resturants
bos_resturants = cd.add_node(boston, 'Resturants')
# Add subnode 'Spoke Wine Bar' to parent bos_resturants
spoke = cd.add_node(bos_resturants, 'Spoke Wine Bar')
cd.add_cell(spoke, 'Addr1', '89 Holland St')
cd.add_cell(spoke, 'City', 'Sommerville')
cd.add_cell(spoke, 'Addr1', '02144')
cd.add_cell(spoke, 'Phone', '617-718-9463')
# Add subnode 'Highland Kitchen' to parent bos_resturants
highland = cd.add_node(bos_resturants, 'Highland Kitchen')
cd.add_cell(highland, 'Addr1', '150 Highland Ave')
cd.add_cell(highland, 'City', 'Sommerville')
cd.add_cell(highland, 'ZipCode', '02144')
cd.add_cell(highland, 'Phone', '617-625-1131')
# display dictionary
print(f'\nCityList Dictionary')
cd.display_dict(cd.CityList)
print(f'\nraw data: {cd.CityList}')
if __name__ == '__main__':
testit()
I tried the first solution
import asyncio
import aiohttp
import json
import requests
from bs4 import BeautifulSoup
import pandas as pd
from CreateDict import CreateDict
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0',
'Accept': '*/*',
'Accept-Language': 'en-US,en;q=0.5',
'X-Spoonflower-Window-UUID': 'a9bc37a2-9eb2-4a1e-8ea1-fcee89347364',
'Content-Type': 'application/json',
'Origin': 'https://www.spoonflower.com',
'Connection': 'keep-alive',
'Referer': 'https://www.spoonflower.com/',
'Sec-GPC': '1',
'If-None-Match': 'W/95d6572c326b81ce98c7ae27ac449d42',
'TE': 'Trailers',
}
def get_fabric_names():
res = requests.get('https://www.spoonflower.com/spoonflower_fabrics')
soup = BeautifulSoup(res.text, 'lxml')
fabrics = [fabric.find('h2').text.strip() for fabric in soup.find_all('div', {'class': 'product_detail medium_text'})]
fabric = [("_".join(fab.upper().replace(u"\u2122", '').split())) for fab in fabrics]
for index in range(len(fabric)):
if 'COTTON_LAWN_(BETA)' in fabric[index]:
fabric[index] = 'COTTON_LAWN_APPAREL'
elif 'COTTON_POPLIN' in fabric[index]:
fabric[index] = 'COTTON_POPLIN_BRAVA'
elif 'ORGANIC_COTTON_KNIT' in fabric[index]:
fabric[index] = 'ORGANIC_COTTON_KNIT_PRIMA'
elif 'PERFORMANCE_PIQUÉ' in fabric[index]:
fabric[index] = 'PERFORMANCE_PIQUE'
elif 'CYPRESS_COTTON' in fabric[index]:
fabric[index] = 'CYPRESS_COTTON_BRAVA'
return fabric
async def get_designEndpoint(session, url):
"""
Get Design End Point
:param url:
"""
async with session.get(url) as response:
response = await response.read()
# print(response)
json_response = json.loads(response.decode("utf-8"))
extracting_endpoint = json_response['page_results']
# extracting designId
design_Id = [item['designId'] for item in extracting_endpoint]
# extracting designName
design_Name = [item['name'] for item in extracting_endpoint]
# extracting creator_Name
creator_Name = [item['user']['screenName'] for item in extracting_endpoint]
return design_Id, design_Name, creator_Name
async def get_Fabric_Pricing_Data(session, url):
"""
Extract all the pricing data with respect to Fabric type
:param url: detail url
:Return: json data
"""
async with session.get(url) as response:
response = await response.read()
json_response = json.loads(response)
#print(json_response)
# Extracting Data
try:
fabric = json_response['data']['fabric_code']
except:
fabric = 'N/A'
try:
test_swatch_meter = json_response['data']['pricing']['TEST_SWATCH_METER']['price']
except:
test_swatch_meter = 'N/A'
try:
fat_quarter_meter = json_response['data']['pricing']['FAT_QUARTER_METER']['price']
except:
fat_quarter_meter = 'N/A'
try:
meter = json_response['data']['pricing']['METER']['price']
except:
meter = 'N/A'
# summary = fabric + "|" + str(test_swatch_meter) + "|" + str(fat_quarter_meter) + "|" + str(meter)
return fabric, test_swatch_meter, fat_quarter_meter, meter
async def main():
urls = []
tasks = []
async with aiohttp.ClientSession(headers=headers) as session:
fabrics = get_fabric_names()
design_Id, design_Name, creator_Name = await get_designEndpoint(session, 'https://pythias.spoonflower.com/search/v1/designs?lang=en&page_offset=0&sort=bestSelling&product=Fabric&forSale=true&showMatureContent=false&page_locale=en')
for item in design_Id:
for fab_type in fabrics[0:-3]:
price_url = 'https://api-gateway.spoonflower.com/alpenrose/pricing/fabrics/FABRIC_'+ fab_type +'?quantity=1&shipping_country=PK¤cy=EUR&measurement_system=METRIC&design_id='+str(item)+'&page_locale=en'
print(price_url)
urls.append(price_url)
for url in urls:
tasks.append(asyncio.create_task(get_Fabric_Pricing_Data(session, url)))
results = await asyncio.gather(*tasks)
print(len(design_Name))
print(len(creator_Name))
print(len(results))
return design_Name, creator_Name, results
if __name__ == '__main__':
loop = asyncio.get_event_loop()
fabrics = get_fabric_names()[0:-3]
design_Name, creator_Name, results = loop.run_until_complete(main())
#print(creator_Name)
def create_new_dict():
newdict = {}
cd = CreateDict()
for n, item in enumerate(design_Name):
dnode = cd.add_node(newdict, (item, creator_Name[n]))
cd.add_cell(dnode, f"fabric_name{n:02}", fabrics[n])
for element in results:
if element[0] == fabrics[n]:
cd.add_cell(dnode, f"test_swatch_meter_{n:02}", element[1])
cd.add_cell(dnode, f"fat_quarter_meter__{n:02}", element[2])
cd.add_cell(dnode, f"meter_{n:02}", element[3])
break
cd.display_dict(newdict)
create_new_dict()
# df = pd.DataFrame.from_dict(items_dict, orient='index').reset_index(drop=False)
# df = df.rename(columns={'level_0':'designName','level_1':'screenName'})
# df.to_csv('scraped_data_1.csv', index=False)it is giving me list index out of range:
Error: Traceback (most recent call last):
File "test-try.py", line 142, in <module>
create_new_dict()
File "test-try.py", line 134, in create_new_dict
cd.add_cell(dnode, f"fabric_name{n:02}", fabrics[n])
IndexError: list index out of range
Posts: 11,873
Threads: 474
Joined: Sep 2016
Jun-25-2021, 07:42 PM
(This post was last modified: Jun-25-2021, 07:42 PM by Larz60+.)
It's assuming that fabric_names is defined as in your first example:
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
and that the same applies for all other lists.
Any deviation from this will cause an error.
add after line 133: and report results
print(f"\ndesigName: {desigName}\ncreatorName: {creatorName}"
print(f"\nfabric_names: {fabric_names}\ndata: {data}\n"Also, try running code I provided stand alone, so you can be sure it works before using.
Posts: 13
Threads: 8
Joined: Apr 2020
Jun-25-2021, 11:37 PM
(This post was last modified: Jun-25-2021, 11:40 PM by rhat398.)
(Jun-25-2021, 07:42 PM)Larz60+ Wrote: It's assuming that fabric_names is defined as in your first example:
fabric_names = ['FABRIC_PETAL_SIGNATURE_COTTON', 'FABRIC_SATIN', 'FABRIC_COTTON_POPLIN_BRAVA']
and that the same applies for all other lists.
Any deviation from this will cause an error.
add after line 133: and report results
print(f"\ndesigName: {desigName}\ncreatorName: {creatorName}"
print(f"\nfabric_names: {fabric_names}\ndata: {data}\n"Also, try running code I provided stand alone, so you can be sure it works before using.
after adding line 133 results:
print(f"\ndesigName: {design_Name}\ncreatorName: {creator_Name}")
print(f"\nfabric_names: {fabrics}\ndata: {results}\n")Output: desigName: ['Catching Fireflies', 'Spoonflower Color Map', 'Night Sky Stars Midnight Blue', 'Soft Meadow Floral', 'Just Jellies - Jellyfish', 'Bees & Lemons - Large - Blue (original colors)', 'Ruth Bader Ginsburg RBG Bust - Black', 'Mid Century Kaleidoscope', 'Native
Eucalyptus Leaves || Edition 1 || Fabric Wallpaper', 'genevieve floral', 'William Morris ~ Pimpernel ~ Original on Black', 'Wild grasses ', 'DND pattern', 'Scattered Earth Tones Watercolor Rainbows', 'Mermaid Scales', "Eame's Wildflower Meadow", 'A Nod to the House Bird', 'elephant march', 'Bats and Hearts with Pink Background', 'Forest Animal Hot Air Balloon Night Adventure', 'cherry blossom watercolor // cherry blossom floral', 'art nouveau poppy red wallpaper', 'Dear Clementine oranges - teal', 'Art Deco Swans - Cream on Black', 'Call of the Mountains (evergreen) MED', 'Sunflowers and cream 7x7', 'Botanic Garden', 'Save The Honey Bees - Large - New', 'midcentury sunflowers', 'Sierra Floral', 'Under the water', 'Rummy Royal Game Mat', 'Mod Triangles Gold Indigo', 'Galaxy deep space seamless, thousands of stars, starry night sky', 'Colorful geometric crystals Small', 'Fable Floral (teal) JUMBO', 'Navy Floral', 'Southdown
tartan - 6" tan/black/white', 'Black cat in the butterfly garden', 'Flight of Feathers Painted', 'Hanging out', 'Napoleonic Bees ~ Faux Gilt on Blackest Black', 'Mermaid Music', 'Bohemian summer', 'Blackest Black Solid', 'Woodland Gnomes', 'Dear Clementine oranges on
white', 'Fable Floral (blush) MED', 'tiger and peacock (large scale)', 'SEAMLESS watercolor Larger leaves Pattern-1', 'Plain white solid white plain unprinted fabric', 'Turtles in Aqua and Blue', 'Freehand hearts || Bone on dusty pink', 'EMERALD EDEN', 'Golden Girls Illustration in Peach', 'Scandinavian sweet hedgehog illustration for kids gender neutral black and white', 'Stained Glass Rose Windows', 'Rainbow boho with stars on blush pink ', 'Art Deco Cranes', 'hearts on grey linen || valentines day', 'Fable Floral (black) JUMBO', 'Spectacular Cats', 'Sandstone desert', 'Mod Triangles Emerald Teal', 'Dark Floral Black roses on black moody floral JUMBO size', 'Cute kawaii sushi small size', 'Flowers & Honey Bees - Large - Blue', 'Opal', 'Watercolor Skull Moth Damask', 'Hidden place ', 'Soft Eucalyptus Watercolor Smaller Leaves Pattern', 'Dragon fire', ' EW, COVID- Grey', 'Tropical Sunrise', 'japanese water garden - black', 'Mushroom forest damask wallpaper dark blue ', 'Black and White Dogs', 'daisy print fabric - daisies, daisy fabric, baby fabric, spring fabric, baby girl, earthy - tan', 'Indigo Rain', "1950's Mid Century Modern", 'Octopus Oasis in Sea', 'Rosie the Riveter', 'molecular divergence in sea blue and coral ', 'Chinoiserie - "Whimsy" - Blue and White']
creatorName: ['thestorysmith', 'spoonflower_help', 'at_the_cottage', 'sweeterthanhoney', 'katerhees', 'fernlesliestudio', 'katerhees', 'ceciliamok', 'erin__kendal', 'crystal_walen', 'peacoquettedesigns', 'ninola-design', 'neonborealis', 'anniemontgomerydesign', 'elladorine', 'hipkiddesigns', 'katerhees', 'endemic', 'andrea_zuill', 'at_the_cottage', 'magentarosedesigns', 'bamokreativ', 'crystal_walen', 'katerhees', 'nouveau_bohemian', 'indybloomdesign', 'irishvikingdesigns', 'fernlesliestudio', 'ottomanbrim', 'crystal_walen', 'lavish_season', 'mousehaus', 'crystal_walen', 'rebecca_reck_art', 'ninola-design', 'nouveau_bohemian', 'crystal_walen', 'weavingmajor', 'boszorka', 'xoxotique', 'sarah_knight', 'peacoquettedesigns', 'ceciliamok', 'camcreative', 'peacoquettedesigns', 'shelbyallison', 'crystal_walen', 'nouveau_bohemian', 'sveta_aho', 'daily_miracles', 'erin__kendal', 'gingerlique', 'erin__kendal', 'catalinakim', 'yesterdaycollection', 'littlesmilemakers', 'sammyk', 'parisbebe', 'katerhees', 'littlearrowdesign', 'nouveau_bohemian', 'cynthia_arre', 'wren_leyland', 'crystal_walen', 'mlags', 'penguinhouse', 'fernlesliestudio', 'peacoquettedesigns', 'rebelform', 'circe_oropeza', 'daily_miracles', 'adenaj', 'kindermama', 'floraryfabrics', 'designed_by_debby', 'denesannadesign', 'littleislandcompany', 'charlottewinter', 'crystal_walen', 'patternanddesign', 'willowlanetextiles', 'spacefem', 'mimipinto', 'pattern_garden']
fabric_names: ['PETAL_SIGNATURE_COTTON', 'SATIN', 'COTTON_POPLIN_BRAVA', 'PERFORMANCE_PIQUE', 'CHIFFON', 'ORGANIC_SWEET_PEA_GAUZE', 'POLY_CREPE_DE_CHINE', 'COTTON_LAWN_APPAREL', 'LIGHTWEIGHT_COTTON_TWILL', 'MODERN_JERSEY', 'COTTON_SPANDEX_JERSEY', 'ORGANIC_COTTON_SATEEN', 'LINEN_COTTON_CANVAS', 'ORGANIC_COTTON_KNIT_PRIMA', 'FLEECE', 'MINKY', 'DOGWOOD_DENIM', 'PERFORMANCE_LINEN', 'RECYCLED_CANVAS', 'SPORT_LYCRA', 'CYPRESS_COTTON_BRAVA', 'CELOSIA_VELVET', 'PERFORMANCE_VELVET', 'BELGIAN_LINEN']
data: [('FABRIC_PETAL_SIGNATURE_COTTON', 1.75, 10.58, 18.22), ('FABRIC_SATIN', 1.75, 11.85, 19.71), ('FABRIC_COTTON_POPLIN_BRAVA', 1.75, 11.85, 19.71), ('FABRIC_PERFORMANCE_PIQUE', 1.75, 12.9, 22.65), ('FABRIC_CHIFFON', 1.75, 12.9, 22.65), ('FABRIC_ORGANIC_SWEET_PEA_GAUZE', 1.75, 12.9, 22.65), ('FABRIC_POLY_CREPE_DE_CHINE', 1.75, 12.9, 22.65), ('FABRIC_COTTON_LAWN_APPAREL', 1.75, 11.75, 21.66), ('FABRIC_LIGHTWEIGHT_COTTON_TWILL', 1.75, 12.9, 25.63), ('FABRIC_MODERN_JERSEY', 1.75, 13.29, 26.18), ('FABRIC_COTTON_SPANDEX_JERSEY', 1.75, 13.57, 26.42), ('FABRIC_ORGANIC_COTTON_SATEEN', 1.75, 13.83, 26.67), ('FABRIC_LINEN_COTTON_CANVAS', 1.75, 14.79, 27.59), ('FABRIC_ORGANIC_COTTON_KNIT_PRIMA', 1.75, 13.83, 26.67), ('FABRIC_FLEECE', 1.75, 14.79, 27.59), ('FABRIC_MINKY', 1.75, 15.29, 28.15), ('FABRIC_DOGWOOD_DENIM', 1.75, 15.83, 29.54), ('FABRIC_PERFORMANCE_LINEN', 1.75, 16.8, 31.58), ('FABRIC_RECYCLED_CANVAS', 1.75, 16.8, 31.58), ('FABRIC_SPORT_LYCRA', 1.75, 16.8, 31.58), ('FABRIC_CYPRESS_COTTON_BRAVA', 1.75, 17.23, 32.48), ('FABRIC_CELOSIA_VELVET', 1.75, 18.7, 35.46), ('FABRIC_PERFORMANCE_VELVET', 1.75, 21.5, 41.08), ('FABRIC_BELGIAN_LINEN', 1.75, 29.54, 58.17), ('FABRIC_PETAL_SIGNATURE_COTTON', 'N/A', 'N/A', 16.4), ('FABRIC_SATIN', 'N/A', 'N/A', 17.74), ('FABRIC_COTTON_POPLIN_BRAVA', 'N/A', 'N/A', 17.74), ('FABRIC_PERFORMANCE_PIQUE', 'N/A', 'N/A', 20.39), ('FABRIC_CHIFFON', 'N/A', 'N/A', 20.39), ('FABRIC_ORGANIC_SWEET_PEA_GAUZE', 'N/A', 'N/A', 20.39), ('FABRIC_POLY_CREPE_DE_CHINE', 'N/A', 'N/A', 20.39), ('FABRIC_COTTON_LAWN_APPAREL', 'N/A', 'N/A', 19.5), ('FABRIC_LIGHTWEIGHT_COTTON_TWILL', 'N/A', 'N/A', 23.07), ('FABRIC_MODERN_JERSEY', 'N/A', 'N/A', 23.57), ('FABRIC_COTTON_SPANDEX_JERSEY', 'N/A', 'N/A', 23.78), ('FABRIC_ORGANIC_COTTON_SATEEN', 'N/A', 'N/A', 24.01), ('FABRIC_LINEN_COTTON_CANVAS', 'N/A', 'N/A', 24.84), ('FABRIC_ORGANIC_COTTON_KNIT_PRIMA', 'N/A', 'N/A', 24.01), ('FABRIC_FLEECE', 'N/A', 'N/A', 24.84), ('FABRIC_MINKY', 'N/A', 'N/A', 25.34), ('FABRIC_DOGWOOD_DENIM', 'N/A', 'N/A', 26.59), ('FABRIC_PERFORMANCE_LINEN', 'N/A', 'N/A', 28.43), ('FABRIC_RECYCLED_CANVAS', 'N/A', 'N/A', 28.43), ('FABRIC_SPORT_LYCRA', 'N/A', 'N/A', 28.43), ('FABRIC_CYPRESS_COTTON_BRAVA', 'N/A', 'N/A', 29.24), ('FABRIC_CELOSIA_VELVET', 'N/A', 'N/A', 31.92), ('FABRIC_PERFORMANCE_VELVET', 'N/A', 'N/A', 36.98), ('FABRIC_BELGIAN_LINEN', 'N/A', 'N/A', 52.36)...]
I tried the code standalone you provide it is working...How can I make it work with my current code.I am looking for like this:
{
('Catching Fireflies', 'thestorysmith'): {
'fabric_name00': 'FABRIC_PETAL_SIGNATURE_COTTON',
'test_swatch_meter_00': 1.75,
'fat_quarter_meter__00': 10.58,
'meter_00': 18.22
},
('Spoonflower Color Map', 'spoonflower_help'): {
'fabric_name00': 'FABRIC_PETAL_SIGNATURE_COTTON',
'test_swatch_meter_00': 1.75,
'fat_quarter_meter__00': 11.85, 'meter_01': 19.71
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'fabric_name00': 'FABRIC_PETAL_SIGNATURE_COTTON',
'test_swatch_meter_00': 1.75,
'fat_quarter_meter__00': 11.85, 'meter_02': 19.71
}
}
{
('Catching Fireflies', 'thestorysmith'): {
'fabric_name01': 'FABRIC_SATIN',
'test_swatch_meter_01': 1.75,
'fat_quarter_meter__01': 10.58,
'meter_01': 18.22
},
('Spoonflower Color Map', 'spoonflower_help'): {
'fabric_name01': 'FABRIC_SATIN',
'test_swatch_meter_01': 1.75,
'fat_quarter_meter__01': 11.85, 'meter_01': 19.71
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'fabric_name01': 'FABRIC_SATIN',
'test_swatch_meter_01': 1.75,
'fat_quarter_meter__01': 11.85, 'meter_02': 19.71
}
}
{
('Catching Fireflies', 'thestorysmith'): {
'fabric_name02': 'FABRIC_COTTON_POPLIN_BRAVA',
'test_swatch_meter_02': 1.75,
'fat_quarter_meter__02': 10.58,
'meter_02': 18.22
},
('Spoonflower Color Map', 'spoonflower_help'): {
'fabric_name02': 'FABRIC_COTTON_POPLIN_BRAVA',
'test_swatch_meter_02': 1.75,
'fat_quarter_meter__02': 11.85, 'meter_01': 19.71
},
('Night Sky Stars Midnight Blue', 'at_the_cottage'): {
'fabric_name02': 'FABRIC_COTTON_POPLIN_BRAVA',
'test_swatch_meter_02': 1.75,
'fat_quarter_meter__02': 11.85, 'meter_02': 19.71
}
}Each designs are available in 24 different fabric and I want each designs map to each fabric in which they are available.
Posts: 11,873
Threads: 474
Joined: Sep 2016
Posts: 11,873
Threads: 474
Joined: Sep 2016
Please don't quote previous posts (unless necessary) , it makes individual posts difficult to read.
The lists are of unequal length.
I put: print(f"List lengths, desigName: {len(desigName)}, creatorName: {len(creatorName)}, fabric_names: {len(fabric_names)}, data: {len(data)}\n")
in the code after lists were defined, and here's the results:
Output: List lengths, desigName: 84, creatorName: 84, fabric_names: 24, data: 48
the first three lists are expected to be the same size.
the data list can be longer (in first example only) as the proper entry is searched for, but there must be an entry for each corresponding design name.
If second example is used, all lists are expected to be the same size (number of entries)
(as presented in post 1)
this code is dependant on that.
Posts: 13
Threads: 8
Joined: Apr 2020
I used the second example it's also giving me this error:
Error: Traceback (most recent call last):
File "rough.py", line 264, in <module>
newdict = create_new_dict()
File "rough.py", line 255, in create_new_dict
cnode = cd.add_node(dnode, fabric_names[n])
IndexError: list index out of range
Posts: 11,873
Threads: 474
Joined: Sep 2016
Please read post 7. The lists are of unequal length. They must be balanced for method 2 to work
Output: List lengths, desigName: 84, creatorName: 84, fabric_names: 24, data: 48
Posts: 13
Threads: 8
Joined: Apr 2020
Got it!But what if lists lengths are inconsistent how to work around that? Any solution for this kind of scenario?
|
