我有两个Spark的数据框架。其中一个是使用HiveContext从Hive表中接收的:
spark_df1 = hc.sql("select * from testdb.titanic_pure_data_test")
我从.csv
文件中得到的第二个spark数据框:
lines = sc.textFile("hdfs://HDFS-1/home/testdb/1500000_Sales_Records.csv").map(lambda line: line.split(","))
spark_df_test = lines.toDF(['Region','Country','Item_Type','Sales_Channel','Order_Priority','Order_Date','Order_ID','Ship_Date','Units_Sold','Unit_Price','Unit_Cost','Total_Revenue','Total_Cost','Total_Profit'])`
我想将任何数据框保存为Hive表。
spark_df1.write.mode("overwrite").format("orc").saveAsTable("testdb.new_res5")
spark_df_test
) 时,出现了这个错误。在"/home/jup-user/testdb/scripts/caching.py"文件的第90行,spark_df_test.write.mode("overwrite").format("orc").saveAsTable("testdb.new_res5")中,pyspark.sql.utils.AnalysisException错误被抛出,错误信息为 "Specifying database name or other qualifiers are not allowed for temporary tables. If the table name has dots (.) in it, please quote the table name with backticks (`)."。