从特定顶点创建图形graphx spark

问题描述

我想从火车数据集中构建一个图形。这是我的代码：

val vertices = df.rdd.flatMap(row => row.getAs[Seq[Row]](3)
        .map(element => (element.getLong(0),element.getBoolean(1),element.getBoolean(2))))

val verticesTrain = vertices.filter{case(id,test,validation) => (test==false)&&(validation==false)}.map(_._1)

val edges = df.rdd.flatMap(row => row.getAs[Seq[Row]](1)
        .map(element => (element.getLong(0),element.getLong(1))))

val graph = Graph.apply(verticesTrain.map(vertex => (vertex,1.0)),edges.map{case(s,d)=>Edge(s,d,1.0)})

但是，当我计算图形的顶点时，似乎我拥有所有的顶点，而不仅仅是顶点顶点中的顶点。

graph.vertices.count()
Out: Long
56944

verticesTrain.count()
Out: Long
44906

如何仅使用顶点火车作为顶点来构建图形？

解决方法

使用有效的子图：

当您要从图形的边缘或顶点过滤掉时，应使用此功能。

这是我用于解决此特定问题的代码：

var CACHE_NAME = 'my-cache-v1';
var urlsToCache = [
    '/'
    // More to be added later
];

self.addEventListener('install',function(event) {
  // Perform install steps
  event.waitUntil(
    caches.open(CACHE_NAME)
      .then(function(cache) {
        console.log('Opened cache');
        return cache.addAll(urlsToCache);
      })
  );
});


self.addEventListener('activate',function(event) {
  event.waitUntil(
    createDB() //Use this function to create or open the database
  );
});

self.addEventListener('fetch',function(event) {
  event.respondWith(
    caches.match(event.request)
      .then(function(response) {
        // Cache hit - return response
        if (response) {
          return response;
        }

        return fetch(event.request).then(
          function(response) {
            // Check if we received a valid response
            if(!response || response.status !== 200 || response.type !== 'basic') {
              return response;
            }
            
            var responseToCache = response.clone();

            caches.open(CACHE_NAME)
              .then(function(cache) {
                cache.put(event.request,responseToCache);
              });

            return response;
          }
        );
      })
    );
});

function createDB() {
  idb.open('mydata',1,function(upgradeDB) {
    var store = upgradeDB.createObjectStore('user',{
      keyPath: 'id'
    });
    store.put({id: 1,name: 'John Doe'}); //This can be updated with an AJAX call to the database later
  });
}