datatype check - Printable Version

datatype check - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: datatype check (/thread-36115.html)

datatype check - arkiboys - Jan-18-2022

Hello,
using synapse pipeline, I generate a .parquet file with data in columns such as HoseNo (Which contains numbers), ClientName (which contains text), etc.
In pyspark, I check for the type of the value in each column and it shows as str for every single field.
Do you know why I do not get number columns as integer datatypes?

example:

type(row["HouseNo"])

result:

Output:
<class 'str'>

p.s. I am unable to placed the above simple python code in code snipt using the button in the thread.

Thank you

RE: datatype check - ndc85430 - Jan-18-2022

It's pretty much impossible to answer without seeing your code, don't you think? Is it taking care to convert strings to numbers? Is it failing somewhere? Please don't make us guess!