Add Sync Behavior #72

New Issue

adam · 2025-12-29T15:23:35+01:00

adam commented

2025-12-29 15:23:35 +01:00

Originally created by @jannon on GitHub (Aug 6, 2016).

Hey, so one of the things I liked about the Sync Project was it's ability to fully sync a list of JSON objects with your Core Data store include insert, update, and delete.

There were a lot of other things I like about CoreStore more, so I decided on using it, but I've come to the point in my integration now that I need the delete functionality. For now, I've just handled it myself after getting the results from importUniqueObjects, but it would be awesome if CoreStore could provide the functionality.

I was thinking it could either be an additional parameter on importUniqueObjects to specify the operations to perform (.Insert, .Update, .Delete) or a separate method (e.g. syncUniqueObjects) that also performs the deletes.

I started to think about adding it and submitting a PR when I realized that I just have absolutely no time to do it right, with tests and docs, and figuring out how to make sure Objective-C works (because it's the worst). Below is something like what I had in mind.
DISCLAIMER: I don't know if the below code even compiles. It's mainly to illustrate what I was thinking in terms of interface and functionality:

public func syncUniqueObjects<T, S: SequenceType where T: NSManagedObject, T: ImportableUniqueObject, S.Generator.Element == T.ImportSource>(
        into: Into<T>,
        sourceArray: S,
        _ fetchClauses: FetchClause...,
        @noescape preProcess: (mapping: [T.UniqueIDType: T.ImportSource]) throws -> [T.UniqueIDType: T.ImportSource] = { $0 }) throws -> (imported: [T], deleted: [T.UniqueIDType]) {
        return try self.syncUniqueObjects(into, sourceArray: sourceArray, fetchClauses, preProcess: preProcess)
    }

public func syncUniqueObjects<T, S: SequenceType where T: NSManagedObject, T: ImportableUniqueObject, S.Generator.Element == T.ImportSource>(
        into: Into<T>,
        sourceArray: S,
        _ fetchClauses: [FetchClause],
        @noescape preProcess: (mapping: [T.UniqueIDType: T.ImportSource]) throws -> [T.UniqueIDType: T.ImportSource] = { $0 }) throws -> (imported: [T], deleted: [T.UniqueIDType]) {

        CoreStore.assert(
            self.isRunningInAllowedQueue(),
            "Attempted to fetch from a \(cs_typeName(self)) outside its designated queue."
        )
        let existing = self.context.fetchAll(From(T), fetchClauses) ?? []
        return try self.syncUniqueObjects(into, sourceArray: sourceArray, existingIDs: existing.map({ $0.uniqueIDValue }), preProcess: preProcess)
    }

public func syncUniqueObjects<T, S: SequenceType where T: NSManagedObject, T: ImportableUniqueObject, S.Generator.Element == T.ImportSource>(
        into: Into<T>,
        sourceArray: S,
        existingIDs: [T.UniqueIDType],
        @noescape preProcess: (mapping: [T.UniqueIDType: T.ImportSource]) throws -> [T.UniqueIDType: T.ImportSource] = { $0 }) throws -> (imported: [T], deleted: [T.UniqueIDType]) {

        let imported = try self.importUniqueObjects(into, sourceArray: sourceArray, preProcess: preProcess)
        let missingIDSet = Set(existingIDs).subtract(imported.map({ $0.uniqueIDValue }))
        self.context.deleteAll(From(T), Where("%K in %@", T.uniqueIDKeyPath, missingIDSet))

        return (imported, Array(missingIDSet))
    }

Thanks!

Originally created by @jannon on GitHub (Aug 6, 2016). Hey, so one of the things I liked about the [Sync Project](https://github.com/hyperoslo/Sync) was it's ability to fully sync a list of JSON objects with your Core Data store include insert, update, and delete. There were a lot of other things I like about CoreStore more, so I decided on using it, but I've come to the point in my integration now that I need the delete functionality. For now, I've just handled it myself after getting the results from `importUniqueObjects`, but it would be awesome if CoreStore could provide the functionality. I was thinking it could either be an additional parameter on `importUniqueObjects` to specify the operations to perform (.Insert, .Update, .Delete) or a separate method (e.g. `syncUniqueObjects`) that also performs the deletes. I started to think about adding it and submitting a PR when I realized that I just have absolutely no time to do it right, with tests and docs, and figuring out how to make sure Objective-C works (because it's the worst). Below is something like what I had in mind. **DISCLAIMER:** I don't know if the below code even compiles. It's mainly to illustrate what I was thinking in terms of interface and functionality: ``` swift public func syncUniqueObjects<T, S: SequenceType where T: NSManagedObject, T: ImportableUniqueObject, S.Generator.Element == T.ImportSource>( into: Into<T>, sourceArray: S, _ fetchClauses: FetchClause..., @noescape preProcess: (mapping: [T.UniqueIDType: T.ImportSource]) throws -> [T.UniqueIDType: T.ImportSource] = { $0 }) throws -> (imported: [T], deleted: [T.UniqueIDType]) { return try self.syncUniqueObjects(into, sourceArray: sourceArray, fetchClauses, preProcess: preProcess) } public func syncUniqueObjects<T, S: SequenceType where T: NSManagedObject, T: ImportableUniqueObject, S.Generator.Element == T.ImportSource>( into: Into<T>, sourceArray: S, _ fetchClauses: [FetchClause], @noescape preProcess: (mapping: [T.UniqueIDType: T.ImportSource]) throws -> [T.UniqueIDType: T.ImportSource] = { $0 }) throws -> (imported: [T], deleted: [T.UniqueIDType]) { CoreStore.assert( self.isRunningInAllowedQueue(), "Attempted to fetch from a \(cs_typeName(self)) outside its designated queue." ) let existing = self.context.fetchAll(From(T), fetchClauses) ?? [] return try self.syncUniqueObjects(into, sourceArray: sourceArray, existingIDs: existing.map({ $0.uniqueIDValue }), preProcess: preProcess) } public func syncUniqueObjects<T, S: SequenceType where T: NSManagedObject, T: ImportableUniqueObject, S.Generator.Element == T.ImportSource>( into: Into<T>, sourceArray: S, existingIDs: [T.UniqueIDType], @noescape preProcess: (mapping: [T.UniqueIDType: T.ImportSource]) throws -> [T.UniqueIDType: T.ImportSource] = { $0 }) throws -> (imported: [T], deleted: [T.UniqueIDType]) { let imported = try self.importUniqueObjects(into, sourceArray: sourceArray, preProcess: preProcess) let missingIDSet = Set(existingIDs).subtract(imported.map({ $0.uniqueIDValue })) self.context.deleteAll(From(T), Where("%K in %@", T.uniqueIDKeyPath, missingIDSet)) return (imported, Array(missingIDSet)) } ``` Thanks!

adam closed this issue

2025-12-29 15:23:36 +01:00

adam commented

2025-12-29 15:23:36 +01:00

@JohnEstropia commented on GitHub (Aug 7, 2016):

Thanks for the idea!

I also like Sync's features and for some time now have been considering implementing a small subset of their features. The remote and local keys mapping in particular looks really attractive. But for it's implementation CoreStore needs a big overhaul of the current importing architecture and I haven't had the time to grind on it yet.

As for your sample implementation, It looks to me like you are just deleting objects that are not in the source array. Is this correct? If so, this can be done similarly with just one line after importUniqueObjects():

let imported = try transaction.importUniqueObjects(Into<MyEntity>(), sourceArray: sourceArray, preProcess: preProcess)
transaction.deleteAll(From<MyEntity>(), Where("NOT (SELF IN %@)", imported))

I've considered adding an auto-deletion mechanism before as well, but it's just too usecase-specific:

In some places it might be ok to delete the non-existent objects.
Sometimes there are entities that need to exist after deletion. Usually they have a local flag, something like "isDeletedFromServer", that gets set instead of being deleted.
In my experience, most entities that require heavy importing also implement a paging mechanism. In that case, you wouldn't want to delete/flag anything at all.

So I let "ImportableUniqueObject" API just do the importing, and leave everything else to the app. I recommend putting your syncUniqueObjects methods in an extension.

In your case though it looks like you need to return the unique IDs of the deleted objects as well. If so, maybe we can add a separate delete method for ImportableUniqueObjects:

// extension BaseDataTransaction
func deleteAllUniqueObjects<T: NSManagedObject where T: ImportableUniqueObject>(from: From<T>, _ deleteClauses: [DeleteClause]) -> [T. UniqueIDType]?

What do you think?

@JohnEstropia commented on GitHub (Aug 7, 2016): Thanks for the idea! I also like Sync's features and for some time now have been considering implementing a small subset of their features. The remote and local keys mapping in particular looks really attractive. But for it's implementation CoreStore needs a big overhaul of the current importing architecture and I haven't had the time to grind on it yet. As for your sample implementation, It looks to me like you are just deleting objects that are not in the source array. Is this correct? If so, this can be done similarly with just one line after `importUniqueObjects()`: ``` let imported = try transaction.importUniqueObjects(Into<MyEntity>(), sourceArray: sourceArray, preProcess: preProcess) transaction.deleteAll(From<MyEntity>(), Where("NOT (SELF IN %@)", imported)) ``` I've considered adding an auto-deletion mechanism before as well, but it's just too usecase-specific: - In some places it might be ok to delete the non-existent objects. - Sometimes there are entities that need to exist after deletion. Usually they have a local flag, something like "isDeletedFromServer", that gets set instead of being deleted. - In my experience, most entities that require heavy importing also implement a paging mechanism. In that case, you wouldn't want to delete/flag anything at all. So I let "ImportableUniqueObject" API just do the importing, and leave everything else to the app. I recommend putting your `syncUniqueObjects` methods in an extension. In your case though it looks like you need to return the unique IDs of the deleted objects as well. If so, maybe we can add a separate delete method for `ImportableUniqueObject`s: ``` // extension BaseDataTransaction func deleteAllUniqueObjects<T: NSManagedObject where T: ImportableUniqueObject>(from: From<T>, _ deleteClauses: [DeleteClause]) -> [T. UniqueIDType]? ``` What do you think?

adam commented

2025-12-29 15:23:36 +01:00

@JohnEstropia commented on GitHub (Aug 24, 2016):

Hi, I'm closing this issue for now. If you have any suggestions or pull requests please don't hesitate :)

@JohnEstropia commented on GitHub (Aug 24, 2016): Hi, I'm closing this issue for now. If you have any suggestions or pull requests please don't hesitate :)

Sign in to join this conversation.

Branches Tags

develop

master

prototype/concurrency

minIOS13

prototype/containers

transactionSourceTracking

prototype/swiftui

minIOS11

swift5_3

prototype/propertyWrapperFields

prototype/propertyWrappers

prototype/propertyWrapper

Swift5

throwables

prototype/Swift_4_1

prototype/Swift_4_0

prototype/Swift_3_2

prototype/queryBuilders

temp/develop

corestore4_develop

prototype/mainContextToPSC

master_ios_7_to_9

swift_2_3

gh-pages

_corespotlight-test

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/CoreStore#72