我正在尝试运行一些代码,但是出现了错误:
代码如下:'DataFrame'对象没有'_get_object_id'属性
items = [(1,12),(1,float('Nan')),(1,14),(1,10),(2,22),(2,20),(2,float('Nan')),(3,300),
(3,float('Nan'))]
sc = spark.sparkContext
rdd = sc.parallelize(items)
df = rdd.toDF(["id", "col1"])
import pyspark.sql.functions as func
means = df.groupby("id").agg(func.mean("col1"))
# The error is thrown at this line
df = df.withColumn("col1", func.when((df["col1"].isNull()), means.where(func.col("id")==df["id"])).otherwise(func.col("col1")))