如何在CoreData CloudKit中防止数据重复？

Question

我们的每一行数据都包含一个唯一的uuid列。

在采用CloudKit之前，uuid列具有唯一约束。这使我们能够防止数据重复。

现在，我们开始将CloudKit集成到现有的CoreData中。这样的唯一约束已被删除。以下用户流程将导致数据重复。

使用CloudKit导致数据重复的步骤

1. 第一次启动应用程序。
2. 由于没有数据，因此将生成具有预定义uuid的预定义数据。
3. 将预定义数据同步到iCloud。
4. 卸载应用程序。
5. 重新安装应用程序。
6. 第一次启动应用程序。
7. 由于没有数据，因此将生成具有预定义uuid的预定义数据。
8. 从步骤3中的先前旧的预定义数据同步到设备。
9. 我们现在有两个具有相同uuid的预定义数据！:(

我想知道是否有办法可以防止这种重复？

在第8步中，我们希望我们有一种方法在写入CoreData之前执行这样的逻辑。

检查CoreData中是否存在此类uuid。如果没有，则写入CoreData。 如果有，则选择具有最新更新日期的一个，然后覆盖 现有数据。

我曾经尝试将上述逻辑插入https://developer.apple.com/documentation/coredata/nsmanagedobject/1506209-willsave。为了防止保存，我正在使用self.managedObjectContext？.rollback()。但它只是崩溃了。

您有什么可靠的机制可以用来防止CoreData CloudKit中的数据重复吗？

---

其他信息：

采用CloudKit之前

我们使用以下CoreData堆栈

class CoreDataStack {
    static let INSTANCE = CoreDataStack()
    
    private init() {
    }
    
    private(set) lazy var persistentContainer: NSPersistentContainer = {
        precondition(Thread.isMainThread)
        
        let container = NSPersistentContainer(name: "xxx", managedObjectModel: NSManagedObjectModel.wenote)
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        return container
    }()

我们的 CoreData 数据架构具有：

1. 独特的约束。
2. 拒绝关联删除规则。
3. 非空字段不具备默认值。

---

采用 CloudKit 后

class CoreDataStack {
    static let INSTANCE = CoreDataStack()
    
    private init() {
    }
    
    private(set) lazy var persistentContainer: NSPersistentContainer = {
        precondition(Thread.isMainThread)
        
        let container = NSPersistentCloudKitContainer(name: "xxx", managedObjectModel: NSManagedObjectModel.wenote)
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        return container
    }()

我们将CoreData数据架构更改为：

1. 没有唯一约束条件。
2. Nullify（空值）删除关系规则。
3. 为非空字段设置默认值。

---

根据来自https://developer.apple.com/forums/thread/699634?login=true的一位开发技术支持工程师的反馈，他提到我们可以：

1. 通过消耗存储持久历史记录来检测相关的更改
2. 删除重复数据

但是，它并不完全清楚应该如何实现，因为提供的github链接已经失效。

- Cheok Yan Cheng

---

你是否正在使用 NSPersistentCloudKitContainer？ - Vadim Belyaev

是的。抱歉。请允许我通过更新我的问题来提供更多技术细节。 - Cheok Yan Cheng

Answer 1

一旦我们与CloudKit集成，就没有独特的约束条件功能。

这个限制的解决方法是：

一旦通过CloudKit插入后检测到重复数据，我们将执行重复数据删除。

这种解决方法具有挑战性，那就是我们如何在CloudKit执行插入时得知并被通知呢？

以下是逐步通知您有关CloudKit执行插入的方法：

1. 在CoreData中打开NSPersistentHistoryTrackingKey功能。
2. 在CoreData中打开NSPersistentStoreRemoteChangeNotificationPostOptionKey功能。
3. 设置viewContext.transactionAuthor = "app"。这是一个重要的步骤，以便当我们在事务历史上进行查询时，我们知道哪个DB事务是由我们的应用程序发起的，哪个DB事务是由CloudKit发起的。
4. 每当我们通过NSPersistentStoreRemoteChangeNotificationPostOptionKey功能自动收到通知时，我们将开始在事务历史记录上进行查询。该查询将基于事务作者和上次查询令牌进行过滤。请参考代码示例以获取更详细的信息。
5. 一旦我们检测到事务是插入，并且它操作我们感兴趣的实体，我们将开始根据相关实体执行重复数据删除。

---

代码示例

import CoreData

class CoreDataStack: CoreDataStackable {
    let appTransactionAuthorName = "app"
    
    /**
     The file URL for persisting the persistent history token.
    */
    private lazy var tokenFile: URL = {
        return UserDataDirectory.token.url.appendingPathComponent("token.data", isDirectory: false)
    }()
    
    /**
     Track the last history token processed for a store, and write its value to file.
     
     The historyQueue reads the token when executing operations, and updates it after processing is complete.
     */
    private var lastHistoryToken: NSPersistentHistoryToken? = nil {
        didSet {
            guard let token = lastHistoryToken,
                let data = try? NSKeyedArchiver.archivedData( withRootObject: token, requiringSecureCoding: true) else { return }
            
            if !UserDataDirectory.token.url.createCompleteDirectoryHierarchyIfDoesNotExist() {
                return
            }
            
            do {
                try data.write(to: tokenFile)
            } catch {
                error_log(error)
            }
        }
    }
    
    /**
     An operation queue for handling history processing tasks: watching changes, deduplicating tags, and triggering UI updates if needed.
     */
    private lazy var historyQueue: OperationQueue = {
        let queue = OperationQueue()
        queue.maxConcurrentOperationCount = 1
        return queue
    }()
    
    var viewContext: NSManagedObjectContext {
        persistentContainer.viewContext
    }
    
    static let INSTANCE = CoreDataStack()
    
    private init() {
        // Load the last token from the token file.
        if let tokenData = try? Data(contentsOf: tokenFile) {
            do {
                lastHistoryToken = try NSKeyedUnarchiver.unarchivedObject(ofClass: NSPersistentHistoryToken.self, from: tokenData)
            } catch {
                error_log(error)
            }
        }
    }
    
    deinit {
        deinitStoreRemoteChangeNotification()
    }
    
    private(set) lazy var persistentContainer: NSPersistentContainer = {
        precondition(Thread.isMainThread)
        
        let container = NSPersistentCloudKitContainer(name: "xxx", managedObjectModel: NSManagedObjectModel.xxx)
        
        // turn on persistent history tracking
        let description = container.persistentStoreDescriptions.first
        description?.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
        description?.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey)
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // Provide transaction author name, so that we can know whether this DB transaction is performed by our app
        // locally, or performed by CloudKit during background sync.
        container.viewContext.transactionAuthor = appTransactionAuthorName
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        // Observe Core Data remote change notifications.
        initStoreRemoteChangeNotification(container)
        
        return container
    }()
    
    private(set) lazy var backgroundContext: NSManagedObjectContext = {
        precondition(Thread.isMainThread)
        
        let backgroundContext = persistentContainer.newBackgroundContext()

        // Provide transaction author name, so that we can know whether this DB transaction is performed by our app
        // locally, or performed by CloudKit during background sync.
        backgroundContext.transactionAuthor = appTransactionAuthorName
        
        // Similar behavior as Android's Room OnConflictStrategy.REPLACE
        // Old data will be overwritten by new data if index conflicts happen.
        backgroundContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        
        // TODO: Not sure these are required...
        //backgroundContext.undoManager = nil
        
        return backgroundContext
    }()
    
    private func initStoreRemoteChangeNotification(_ container: NSPersistentContainer) {
        // Observe Core Data remote change notifications.
        NotificationCenter.default.addObserver(
            self,
            selector: #selector(storeRemoteChange(_:)),
            name: .NSPersistentStoreRemoteChange,
            object: container.persistentStoreCoordinator
        )
    }
    
    private func deinitStoreRemoteChangeNotification() {
        NotificationCenter.default.removeObserver(self)
    }
    
    @objc func storeRemoteChange(_ notification: Notification) {
        // Process persistent history to merge changes from other coordinators.
        historyQueue.addOperation {
            self.processPersistentHistory()
        }
    }
    
    /**
     Process persistent history, posting any relevant transactions to the current view.
     */
    private func processPersistentHistory() {
        backgroundContext.performAndWait {
            
            // Fetch history received from outside the app since the last token
            let historyFetchRequest = NSPersistentHistoryTransaction.fetchRequest!
            historyFetchRequest.predicate = NSPredicate(format: "author != %@", appTransactionAuthorName)
            let request = NSPersistentHistoryChangeRequest.fetchHistory(after: lastHistoryToken)
            request.fetchRequest = historyFetchRequest

            let result = (try? backgroundContext.execute(request)) as? NSPersistentHistoryResult
            guard let transactions = result?.result as? [NSPersistentHistoryTransaction] else { return }

            if transactions.isEmpty {
                return
            }
            
            for transaction in transactions {
                if let changes = transaction.changes {
                    for change in changes {
                        let entity = change.changedObjectID.entity.name
                        let changeType = change.changeType
                        let objectID = change.changedObjectID
                        
                        if entity == "NSTabInfo" && changeType == .insert {
                            deduplicateNSTabInfo(objectID)
                        }
                    }
                }
            }
            
            // Update the history token using the last transaction.
            lastHistoryToken = transactions.last!.token
        }
    }
    
    private func deduplicateNSTabInfo(_ objectID: NSManagedObjectID) {
        do {
            guard let nsTabInfo = try backgroundContext.existingObject(with: objectID) as? NSTabInfo else { return }
            
            let uuid = nsTabInfo.uuid
            
            guard let nsTabInfos = NSTabInfoRepository.INSTANCE.getNSTabInfosInBackground(uuid) else { return }
            
            if nsTabInfos.isEmpty {
                return
            }
            
            var bestNSTabInfo: NSTabInfo? = nil
            
            for nsTabInfo in nsTabInfos {
                if let _bestNSTabInfo = bestNSTabInfo {
                    if nsTabInfo.syncedTimestamp > _bestNSTabInfo.syncedTimestamp {
                        bestNSTabInfo = nsTabInfo
                    }
                } else {
                    bestNSTabInfo = nsTabInfo
                }
            }
            
            for nsTabInfo in nsTabInfos {
                if nsTabInfo === bestNSTabInfo {
                    continue
                }
                
                // Remove old duplicated data!
                backgroundContext.delete(nsTabInfo)
            }
            
            RepositoryUtils.saveContextIfPossible(backgroundContext)
        } catch {
            error_log(error)
        }
    }
}

---

参考资料

1. https://developer.apple.com/documentation/coredata/synchronizing_a_local_store_to_the_cloud - 该示例代码中的CoreDataStack.swift文件演示了如何在云端同步后移除重复数据的类似例子。
2. https://developer.apple.com/documentation/coredata/consuming_relevant_store_changes - 关于事务历史记录的信息。
3. 使用NSPersistentCloudKitContainer预填充Core Data存储时的最佳方法是什么？ - 一个类似的问题。

- Cheok Yan Cheng