PYSPARK：如何可视化GraphFrame？

Question

假设我已经创建了以下图表。我的问题是如何将其可视化？

 # Create a Vertex DataFrame with unique ID column "id"
    v = sqlContext.createDataFrame([
      ("a", "Alice", 34),
      ("b", "Bob", 36),
      ("c", "Charlie", 30),
    ], ["id", "name", "age"])
    # Create an Edge DataFrame with "src" and "dst" columns
    e = sqlContext.createDataFrame([
      ("a", "b", "friend"),
      ("b", "c", "follow"),
      ("c", "b", "follow"),
    ], ["src", "dst", "relationship"])
    # Create a GraphFrame
    from graphframes import *
    g = GraphFrame(v, e)

- Alex

---

Alex：你是在问可视化工具、技术等方面的问题，以便可视化图形框架吗？那么你可以看到笔记本界面。请参见 On-Time Flight Performance with GraphFrames for Apache Spark。 - Ram Ghadiyaram

@RamGhadiyaram 我想使用Spark/Python库来可视化图表。如果没有这样的工具，我会寻找其他替代方案。 - Alex

Answer 1

我正在使用Python/PySpark/Jupyter，使用networkx库中的绘图功能。关键在于从grapheframe创建一个networkx图。

import networkx as nx
from graphframes import GraphFrame

def PlotGraph(edge_list):
    Gplot=nx.Graph()
    for row in edge_list.select('src','dst').take(1000):
        Gplot.add_edge(row['src'],row['dst'])

    plt.subplot(121)
    nx.draw(Gplot, with_labels=True, font_weight='bold')


spark = SparkSession \
    .builder \
    .appName("PlotAPp") \
    .getOrCreate()

sqlContext = SQLContext(spark)

vertices = sqlContext.createDataFrame([
  ("a", "Alice", 34),
  ("b", "Bob", 36),
  ("c", "Charlie", 30),
  ("d", "David", 29),
  ("e", "Esther", 32),
("e1", "Esther2", 32),
  ("f", "Fanny", 36),
  ("g", "Gabby", 60),
    ("h", "Mark", 61),
    ("i", "Gunter", 62),
    ("j", "Marit", 63)], ["id", "name", "age"])

edges = sqlContext.createDataFrame([
  ("a", "b", "friend"),
  ("b", "a", "follow"),
  ("c", "a", "follow"),
  ("c", "f", "follow"),
  ("g", "h", "follow"),
  ("h", "i", "friend"),
  ("h", "j", "friend"),
  ("j", "h", "friend"),
    ("e", "e1", "friend")
], ["src", "dst", "relationship"])

g = GraphFrame(vertices, edges)
PlotGraph(g.edges)

一些图形的绘制

- Alex Ortner

Answer 2

我找不到任何原生的GraphFrame库，可以可视化数据。

尽管如此，您可以使用DataBricks中的display()函数尝试执行此操作。您可以在这里看到一个示例。

此外，您可以尝试将GraphFrame转换为Python列表，并使用matplotlib或Pygraphviz库。

- drkostas