Pyspark dataframe

[deanhystad]

What have you tried? It seems the solution is very straightforward, so maybe I don't understand what you are asking.

These are a couple of different ways to get a value in a dataframe

import pandas as pd

df = pd.DataFrame(
    {
        "Col1": range(1, 5),
        "Col2": list("ABCD"),
        "Col3": range(101, 105),
        "Col4": ("stuff", None, "stuff", None),
    }
)

print(df)
# Using the row index values: 0, 1, 2, 3
print('\ndf["Col3"][3]', df["Col3"][3], sep="\n")

# Reindexing to use Col2.  Get row "D", column "Col3"
temp = df.set_index("Col2")
print('\ntemp["Col3"]["D"]', temp["Col3"]["D"], sep="\n")

# As a matrix
print("\ndf.values[3][2]", df.values[3][2], sep="\n")

   Col1 Col2  Col3   Col4
0     1    A   101  stuff
1     2    B   102   None
2     3    C   103  stuff
3     4    D   104   None

df["Col3"][3]
104

temp["Col3"]["D"]
104

df.values[3][2]
104

The first two are the same. The only difference is in the second I changed the row index to use the values in "Col2". That lets ne get the value from Col3 that has "D" in Col2.

The third one uses DataFrame.values to get the values as a 2D numpy array. Then I use integer array indexing to get the value.