CoreStore with differenceKit #241

New Issue

Closed

opened 2025-12-29 15:27:19 +01:00 by adam · 10 comments

adam commented 2025-12-29 15:27:19 +01:00

Owner

Copy Link

Copy Source

Originally created by @eraydiler on GitHub (Nov 16, 2018).

Hello,

I'm trying to use DifferenceKit library for updating UI triggered by relevant CoreStore methods.
I simply need to have two different collection (before-update, after-update) to use these library.

However, it looks like the dataProvider.objects, which holds elements represented in UI, is being automatically updated, so I am unable to make a comparison between previous and updated data.

Is there a way to access the previous state of these objects so that I can make a comparison with the updated objects?

extension SimpleListViewController {
    func listMonitor(
        _ monitor: ListMonitor<LocalTask>,
        didUpdateObject object: LocalTask,
        atIndexPath indexPath: IndexPath) {

        guard
            let oldObjects = dataProvider.objects as? [LocalTask],
            var newObjects = dataProvider.objects as? [LocalTask]

            else {
                return
        }

        newObjects[indexPath.item] = object // These line is useless now, because newObjects is already updated

        let changeSet = StagedChangeset(
            source: oldObjects,
            target: newObjects
        )

        listView.reload(using: changeSet) { data in
            update(objects: data)
        }
    }
}

Originally created by @eraydiler on GitHub (Nov 16, 2018). Hello, I'm trying to use DifferenceKit library for updating UI triggered by relevant CoreStore methods. I simply need to have two different collection (before-update, after-update) to use these library. However, it looks like the `dataProvider.objects`, which holds elements represented in UI, is being automatically updated, so I am unable to make a comparison between previous and updated data. Is there a way to access the previous state of these objects so that I can make a comparison with the updated objects? ```swift extension SimpleListViewController { func listMonitor( _ monitor: ListMonitor<LocalTask>, didUpdateObject object: LocalTask, atIndexPath indexPath: IndexPath) { guard let oldObjects = dataProvider.objects as? [LocalTask], var newObjects = dataProvider.objects as? [LocalTask] else { return } newObjects[indexPath.item] = object // These line is useless now, because newObjects is already updated let changeSet = StagedChangeset( source: oldObjects, target: newObjects ) listView.reload(using: changeSet) { data in update(objects: data) } } } ````

adam closed this issue 2025-12-29 15:27:19 +01:00

adam commented 2025-12-29 15:27:20 +01:00

Author

Owner

Copy Link

Copy Source

@JohnEstropia commented on GitHub (Nov 19, 2018):

ListMonitor (or strictly speaking, NSFetchedResultsController) manages these changes for you so they are not really a good fit for DifferenceKit. If you run DifferenceKit on top of ListMonitors you will be unnecessarily duplicating a lot of heavy work.

Diff'ing libraries like DifferenceKit were designed for objects that don't manage states, meaning you have an "old version" and a "new version". With ORMs like Core Data, objects are live and update themselves in real time.

@JohnEstropia commented on GitHub (Nov 19, 2018): `ListMonitor` (or strictly speaking, `NSFetchedResultsController`) manages these changes for you so they are not really a good fit for DifferenceKit. If you run DifferenceKit on top of `ListMonitor`s you will be unnecessarily duplicating a lot of heavy work. Diff'ing libraries like `DifferenceKit` were designed for objects that don't manage states, meaning you have an "old version" and a "new version". With ORMs like Core Data, objects are *live* and update themselves in real time.

adam commented 2025-12-29 15:27:21 +01:00

Author

Owner

Copy Link

Copy Source

@tosbaha commented on GitHub (Nov 22, 2018):

I have coded something before maybe it can help you too. If you are downloading data from a web server and want to compare the values, you may use a DifferenceKit. I personally use DeepDiff

So let's say you have new Data coming from a web server Which is called Post and each Post have some PostDetail 1:M Relationship

Storage.stack.rx.perform(asynchronous: { transaction in
// Import uniqueObjects
let imported = try transaction.importUniqueObjects(Into(PostCD.self), sourceArray: posts)
// Optional Remove non existent data from the local database
transaction.deleteAll(From<PostCD>(), Where<PostCD>("NOT (SELF IN %@)", imported))
})

In order for this code to work, your NSManagedObject classes must conform to ImportableObject protocol.

import CoreStore
import DeepDiff

extension PostDetailCD:ImportableObject {
    typealias ImportSource = PostDetail

    func didInsert(from source: PostDetail, in transaction: BaseDataTransaction) throws {
        self.message = source.message
        self.date = source.date
    }
}


extension PostCD:ImportableUniqueObject {
    typealias ImportSource = Post // Your Model from the Server
    // MARK: ImportableUniqueObject
    typealias UniqueIDType = String
    static var uniqueIDKeyPath: String {
        return #keyPath(PostCD.trackingID)
    }
    
    // MARK: ImportableObject
    static func shouldInsert(from source: ImportSource, in transaction: BaseDataTransaction) -> Bool {
        return true
    }
    
    static func shouldUpdate(from source: ImportSource, in transaction: BaseDataTransaction) -> Bool {
        return true
    }
    
    static func uniqueID(from source: Post, in transaction: BaseDataTransaction) throws -> String? {
        return source.trackingID
    }
    
    
    func update(from source: Post, in transaction: BaseDataTransaction) throws {
        self.name = source.name
        self.trackingID = source.trackingID

        
        if let oldDetails = self.detail.array as? [PostDetailCD] {
            var oldArray = [PostDetail]()
            for detail in oldDetails {
                oldArray.append(PostDetail.init(message: detail.message, date: detail.date))
            }
                        
            let changes = diff(old: oldArray, new: source.details)
//            if changes.count > 0 {
//                print("Difference: \(changes)")
//            }
            
            for change in changes {
                switch change {
                case .delete(let detail):
                    transaction.delete(self.detail.object(at: detail.index) as? PostDetailCD)
                case .insert(let detail):
                    if let newDetail = try transaction.importObject(Into(PostDetailCD.self), source: detail.item) {
                        newDetail.parent = self
                        self.detail.insert(newDetail, at: detail.index)
                    }
                case .replace(let detail):
                    if let oldDetail = self.detail.object(at: detail.index) as? PostDetailCD {
                        oldDetail.message = detail.newItem.message
                        oldDetail.date = detail.newItem.date

                    } else {
                        print("This shouldn't happen!!")
                    }
                case .move(let detail):
                    self.detail.moveObjects(at: IndexSet(integer: detail.fromIndex), to: detail.toIndex)
                }
            }
            
        }
    }
}

@tosbaha commented on GitHub (Nov 22, 2018): I have coded something before maybe it can help you too. If you are downloading data from a web server and want to compare the values, you may use a DifferenceKit. I personally use `DeepDiff` So let's say you have new Data coming from a web server Which is called `Post` and each `Post` have some `PostDetail` 1:M Relationship ```swift Storage.stack.rx.perform(asynchronous: { transaction in // Import uniqueObjects let imported = try transaction.importUniqueObjects(Into(PostCD.self), sourceArray: posts) // Optional Remove non existent data from the local database transaction.deleteAll(From<PostCD>(), Where<PostCD>("NOT (SELF IN %@)", imported)) }) ``` In order for this code to work, your NSManagedObject classes must conform to `ImportableObject` protocol. ```swift import CoreStore import DeepDiff extension PostDetailCD:ImportableObject { typealias ImportSource = PostDetail func didInsert(from source: PostDetail, in transaction: BaseDataTransaction) throws { self.message = source.message self.date = source.date } } extension PostCD:ImportableUniqueObject { typealias ImportSource = Post // Your Model from the Server // MARK: ImportableUniqueObject typealias UniqueIDType = String static var uniqueIDKeyPath: String { return #keyPath(PostCD.trackingID) } // MARK: ImportableObject static func shouldInsert(from source: ImportSource, in transaction: BaseDataTransaction) -> Bool { return true } static func shouldUpdate(from source: ImportSource, in transaction: BaseDataTransaction) -> Bool { return true } static func uniqueID(from source: Post, in transaction: BaseDataTransaction) throws -> String? { return source.trackingID } func update(from source: Post, in transaction: BaseDataTransaction) throws { self.name = source.name self.trackingID = source.trackingID if let oldDetails = self.detail.array as? [PostDetailCD] { var oldArray = [PostDetail]() for detail in oldDetails { oldArray.append(PostDetail.init(message: detail.message, date: detail.date)) } let changes = diff(old: oldArray, new: source.details) // if changes.count > 0 { // print("Difference: \(changes)") // } for change in changes { switch change { case .delete(let detail): transaction.delete(self.detail.object(at: detail.index) as? PostDetailCD) case .insert(let detail): if let newDetail = try transaction.importObject(Into(PostDetailCD.self), source: detail.item) { newDetail.parent = self self.detail.insert(newDetail, at: detail.index) } case .replace(let detail): if let oldDetail = self.detail.object(at: detail.index) as? PostDetailCD { oldDetail.message = detail.newItem.message oldDetail.date = detail.newItem.date } else { print("This shouldn't happen!!") } case .move(let detail): self.detail.moveObjects(at: IndexSet(integer: detail.fromIndex), to: detail.toIndex) } } } } } ```

adam commented 2025-12-29 15:27:21 +01:00

Author

Owner

Copy Link

Copy Source

@JohnEstropia commented on GitHub (Dec 5, 2018):

@tosbaha I may be missing something your implementation, but will .replace be called if there are just updates to the PostDetail? The way I see it is by the time you call PosDetail.init, the detail instance would be the updated instance. (Unless you are managing the update order as well)

Also, since you are doing this in the context of the transaction, you would need extra handling if there was an error further on.

@JohnEstropia commented on GitHub (Dec 5, 2018): @tosbaha I may be missing something your implementation, but will `.replace` be called if there are just updates to the `PostDetail`? The way I see it is by the time you call `PosDetail.init`, the `detail` instance would be the updated instance. (Unless you are managing the update order as well) Also, since you are doing this in the context of the transaction, you would need extra handling if there was an error further on.

adam commented 2025-12-29 15:27:25 +01:00

Author

Owner

Copy Link

Copy Source

@tosbaha commented on GitHub (Dec 5, 2018):

I haven't tested all possible cases but here is how it works. DeepDiff compares both Post and array PostDetail according to DeepDiff report it either deletes,insert or replace the item from the array. So if PostDetail array has 100 items and only one is added later on, I only insert once.Likewise, if 99 details are same but last detail is changed, I only replace the last item. Therefore I prevent heavy delete insert operations. As I said, I could be missing something but so far it works faster then deleting/replacing all details when the data on server changes. I will appreciate if you tell me if there is any mistake in my logic or implementation. Thanks.

@tosbaha commented on GitHub (Dec 5, 2018): I haven't tested all possible cases but here is how it works. `DeepDiff` compares both Post and array `PostDetail` according to `DeepDiff` report it either deletes,insert or replace the item from the array. So if `PostDetail` array has 100 items and only one is added later on, I only insert once.Likewise, if 99 details are same but last detail is changed, I only replace the last item. Therefore I prevent heavy delete insert operations. As I said, I could be missing something but so far it works faster then deleting/replacing all details when the data on server changes. I will appreciate if you tell me if there is any mistake in my logic or implementation. Thanks.

adam commented 2025-12-29 15:27:26 +01:00

Author

Owner

Copy Link

Copy Source

@tosbaha commented on GitHub (Dec 16, 2018):

I spoke too soon :( My code only works if last detail is changed since my insert code is buggy. It is not possible to insert at given index.

    self.detail.insert(newDetail, at: detail.index)

Above code doesn't seem to have any effect. It always inserts at the last when I use try transaction.importObject
How can I solve this issue?

Only hacky way I have found is this

Since object is inserted at the end of Array, I move the object from end to whatever the index it is.
self.detail.moveObjects(at: IndexSet(integer: self.detail.count - 1 ), to: index)

@tosbaha commented on GitHub (Dec 16, 2018): I spoke too soon :( My code only works if last detail is changed since my insert code is buggy. It is not possible to insert at given index. ```swift self.detail.insert(newDetail, at: detail.index) ``` Above code doesn't seem to have any effect. It always inserts at the last when I use ` try transaction.importObject` How can I solve this issue? Only hacky way I have found is this Since object is inserted at the end of Array, I move the object from end to whatever the index it is. `self.detail.moveObjects(at: IndexSet(integer: self.detail.count - 1 ), to: index)`

adam commented 2025-12-29 15:27:27 +01:00

Author

Owner

Copy Link

Copy Source

@JohnEstropia commented on GitHub (Dec 18, 2018):

Assuming self.detail is a to-many relationship, and PostCD an NSManagedObject subclass, you cannot directly mutate the NSOrderedSet or NSSet property. See https://stackoverflow.com/a/27888302/809614

My suggestion for your case (if I understood the intent correctly):

1. Create your var oldArray: [PostDetail] from your existing message objects before you do any imports
2. Load the changes by calling diff(old: oldArray, new: source.details) (still before importing)
3. Copy self.details into a temporary var temp: [PostDetailCD] = []
4. Handle all changes and work with the temp array, not self.details
5. When you're done set self.details = NSOrderedSet(array: temp)

It is a lot faster to set a relationship all at once than mutating the internal collection for each iteration.

@JohnEstropia commented on GitHub (Dec 18, 2018): Assuming `self.detail` is a to-many relationship, and `PostCD` an `NSManagedObject` subclass, you cannot directly mutate the `NSOrderedSet` or `NSSet` property. See https://stackoverflow.com/a/27888302/809614 My suggestion for your case (if I understood the intent correctly): 1. Create your `var oldArray: [PostDetail]` from your existing message objects before you do any imports 2. Load the changes by calling `diff(old: oldArray, new: source.details)` (still before importing) 3. Copy `self.details` into a temporary `var temp: [PostDetailCD] = []` 4. Handle all `changes` and work with the `temp` array, not `self.details` 5. When you're done set `self.details = NSOrderedSet(array: temp)` It is a lot faster to set a relationship all at once than mutating the internal collection for each iteration.

adam commented 2025-12-29 15:27:27 +01:00

Author

Owner

Copy Link

Copy Source

@JohnEstropia commented on GitHub (Dec 18, 2018):

Just a note here, that @eraydiler and @tosbaha 's cases are totally different.
@eraydiler is trying to use Core Data objects directly with within diff(), while @tosbaha 's is diff'ing only the ImportSource arrays. The latter may be necessary depending on the use case, but the former is something I would discourage.

@JohnEstropia commented on GitHub (Dec 18, 2018): Just a note here, that @eraydiler and @tosbaha 's cases are totally different. @eraydiler is trying to use Core Data objects directly with within `diff()`, while @tosbaha 's is diff'ing only the `ImportSource` arrays. The latter may be necessary depending on the use case, but the former is something I would discourage.

adam commented 2025-12-29 15:27:27 +01:00

Author

Owner

Copy Link

Copy Source

@tosbaha commented on GitHub (Dec 18, 2018):

Thanks a lot @JohnEstropia I decided to just check if there is difference or not. If different set all at once. Thanks once again for taking your time for explaining in detail.

@tosbaha commented on GitHub (Dec 18, 2018): Thanks a lot @JohnEstropia I decided to just check if there is difference or not. If different set all at once. Thanks once again for taking your time for explaining in detail.

adam commented 2025-12-29 15:27:27 +01:00

Author

Owner

Copy Link

Copy Source

@eraydiler commented on GitHub (Dec 21, 2018):

Thanks for the answers @JohnEstropia, @tosbaha. After having some time I decided to go without using differenceKit.

@eraydiler commented on GitHub (Dec 21, 2018): Thanks for the answers @JohnEstropia, @tosbaha. After having some time I decided to go without using differenceKit.

adam commented 2025-12-29 15:27:28 +01:00

Author

Owner

Copy Link

Copy Source

@JohnEstropia commented on GitHub (Jan 8, 2020):

This is quite old but to anyone following the thread, CoreStore now has ListPublishers which work in tandem with DiffableDataSources. This uses the same algorithm that DifferenceKit uses, so check the README for more info. Closing this thread now

@JohnEstropia commented on GitHub (Jan 8, 2020): This is quite old but to anyone following the thread, CoreStore now has `ListPublisher`s which work in tandem with `DiffableDataSource`s. This uses the same algorithm that DifferenceKit uses, so check the README for more info. Closing this thread now

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

develop

prototype/swift6

master

prototype/concurrency

minIOS13

prototype/containers

transactionSourceTracking

prototype/swiftui

minIOS11

swift5_3

prototype/propertyWrapperFields

prototype/propertyWrappers

prototype/propertyWrapper

Swift5

throwables

prototype/Swift_4_1

prototype/Swift_4_0

prototype/Swift_3_2

prototype/queryBuilders

temp/develop

corestore4_develop

prototype/mainContextToPSC

master_ios_7_to_9

swift_2_3

gh-pages

_corespotlight-test

9.3.0

9.2.0

9.1.0

9.0.0

8.1.0

8.0.1

8.0.0

7.3.1

7.3.0

7.2.0

7.1.0

7.0.4

7.0.3

7.0.2

7.0.1

7.0.0

6.3.2

6.3.1

6.3.0

6.2.1

6.2.0

6.1.0

6.0.0

5.3.1

5.3.0

5.2.0

5.1.1

5.1.0

5.0.0

4.2.4

4.2.3

4.2.2

4.2.1

4.2.0

4.1.4

4.1.3

4.1.1

4.1.0

4.0.5

4.0.4

4.0.3

4.0.2

4.0.1

4.0.0

4.0.0-beta3

4.0.0-beta2

4.0.0-beta

3.1.1

3.1.0

3.0.3

3.0.2

3.0.1

3.0.0

2.1.3

2.1.2

2.1.1

2.1.0

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.6.10

1.6.9

1.6.8

1.6.7

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.1

1.2.0

1.0.2

1.0.1

1.0.0

0.2.1

0.2.0

0.1.2

0.1.1

0.1.0

Labels

Clear labels

abandoned

corestore bug

discussion

duplicate

enhancement

fixed

fixed

help wanted

in progress

investigating

ios/compiler bug

irreproducible

looking for workarounds

not a bug

pending

pending docs update

pull-request

Mirrored from GitHub Pull Request

question

regression

user bug

wontfix

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

adam (Adam Melkus)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/CoreStore#241