Feature: Importable synchronizable objects #67

New Issue

adam · 2025-12-29T15:23:29+01:00

adam commented

2025-12-29 15:23:29 +01:00

Originally created by @ruslanskorb on GitHub (Jun 29, 2016).

Hi @JohnEstropia!

To synchronize the data created when the application was offline, we need to use an additional unique ID generated by the client. This ID can be included in the import source to update locally created objects.

Since I needed it I added a new protocol ImportableSynchronizableObject and related methods to the BaseDataTransaction class.

If you think that it might be useful for you or other users of CoreStore just let me know and I will create a pull request.

Originally created by @ruslanskorb on GitHub (Jun 29, 2016). Hi @JohnEstropia! To synchronize the data created when the application was offline, we need to use an additional unique ID generated by the client. This ID can be included in the import source to update locally created objects. Since I needed it I added [a new protocol `ImportableSynchronizableObject` and related methods to the `BaseDataTransaction` class](https://github.com/JohnEstropia/CoreStore/compare/master...ruslanskorb:importable-synchronizable-object). If you think that it might be useful for you or other users of **CoreStore** just let me know and I will create a pull request.

adam closed this issue

2025-12-29 15:23:29 +01:00

adam commented

2025-12-29 15:23:29 +01:00

@JohnEstropia commented on GitHub (Jun 30, 2016):

Hmm... I may have misunderstood the requirement, but in this case shouldn't the local ID be the unique ID?
Also I'm not sure if allowing two independent unique IDs to exist is good (or at least flexible) design.

If the serverObjectIDValue is optional, how do we guarantee that we will never hit a case where two objects with different localObjectIDValues have the same serverObjectIDValue? (the synchronization becomes tightly-coupled to the client-server implementation)
The array returned by importSynchronizableObjects(...) seems to prioritize local objects. What if other apps need it the other way around?
Come to think of it, if the app is offline where do you import from in the first place? It seems roundabout to create dictionaries just to parse and import them again.

On the other hand, I appreciate the very clean code (and full documentation!). It's just that I think the current design/implementation is a bit app-specific.

Of course you are free to keep your fork and share your extension as you wish :)

@JohnEstropia commented on GitHub (Jun 30, 2016): Hmm... I may have misunderstood the requirement, but in this case shouldn't the local ID be the unique ID? Also I'm not sure if allowing two independent unique IDs to exist is good (or at least flexible) design. - If the `serverObjectIDValue` is optional, how do we guarantee that we will never hit a case where two objects with different `localObjectIDValue`s have the same `serverObjectIDValue`? (the synchronization becomes tightly-coupled to the client-server implementation) - The array returned by `importSynchronizableObjects(...)` seems to prioritize local objects. What if other apps need it the other way around? - Come to think of it, if the app is offline where do you import from in the first place? It seems roundabout to create dictionaries just to parse and import them again. On the other hand, I appreciate the very clean code (and full documentation!). It's just that I think the current design/implementation is a bit app-specific. Of course you are free to keep your fork and share your extension as you wish :)

adam commented

2025-12-29 15:23:29 +01:00

@JohnEstropia commented on GitHub (Jul 1, 2016):

Thanks again for the feature suggestion! I'll close this for now, but feel free to keep sharing ideas! :)

@JohnEstropia commented on GitHub (Jul 1, 2016): Thanks again for the feature suggestion! I'll close this for now, but feel free to keep sharing ideas! :)

adam commented

2025-12-29 15:23:30 +01:00

@ruslanskorb commented on GitHub (Jul 1, 2016):

Thanks for the feedback!

You're right about the priority of local objects in importSynchronizableObjects(...). I updated it and force pushed to my repo. Now all the objects are arranged according to their position in the source array.

To answer the rest of your questions I need to describe the synchronization process.

The local object ID is the unique ID that is generated and used by the client to find and update the object in the local store.
The server object ID is also the unique ID that is generated by the server and used by the client to communicate with the server.

After creating a new object by the user (e.g., adding an entry in the notebook) the client generates and assigns a new local object ID to it.

Then:

if the app is online the client
- sends the local object ID to the server
- receives the server object ID and the local object ID in the response
- imports this data using the method importSynchronizableObject(…); in this case the local object ID will be used to find the local object and update its server object ID
- sends the server object ID along with the user-added data to the server
- receives the server object ID (the object was successfully created on the server)
If the app is offline the client
- will send this object along with the other changes to the server when the app will have access to the Internet

When the user receives a brand new object from the server (without the local object ID, because only the client stored it) the client imports it using the method importSynchronizableObject(…). In this case the server object ID will be used to find and update the local object. If such an object does not exist the client will create a new one and generate and assign a new local object ID to it.

Eventually we have only two cases when we assign the server object ID to the local object and not one of them does not allow to assign one and the same serverObjectIDValue to local objects with different localObjectIDValue.

I don't describe here all the possible cases but this description should be enough to understand the role of ImportableSynchronizableObject in the synchronization process and to answer your questions.

In any case, with time, I plan to describe the whole process of synchronization and share it with the developer community.

Thank you for reading this! :-]

@ruslanskorb commented on GitHub (Jul 1, 2016): Thanks for the feedback! You're right about the priority of local objects in `importSynchronizableObjects(...)`. I updated it and force pushed to my repo. Now all the objects are arranged according to their position in the source array. To answer the rest of your questions I need to describe the synchronization process. The local object ID is the unique ID that is generated and used by the client to find and update the object in the local store. The server object ID is also the unique ID that is generated by the server and used by the client to communicate with the server. After creating a new object by the user (e.g., adding an entry in the notebook) the client generates and assigns a new local object ID to it. Then: - if the app is online the client - sends the local object ID to the server - receives the server object ID and the local object ID in the response - imports this data using the method `importSynchronizableObject(…)`; in this case the local object ID will be used to find the local object and update its server object ID - sends the server object ID along with the user-added data to the server - receives the server object ID (the object was successfully created on the server) - If the app is offline the client - will send this object along with the other changes to the server when the app will have access to the Internet When the user receives a brand new object from the server (without the local object ID, because only the client stored it) the client imports it using the method `importSynchronizableObject(…)`. In this case the server object ID will be used to find and update the local object. If such an object does not exist the client will create a new one and generate and assign a new local object ID to it. Eventually we have only two cases when we assign the server object ID to the local object and not one of them does not allow to assign one and the same `serverObjectIDValue` to local objects with different `localObjectIDValue`. I don't describe here all the possible cases but this description should be enough to understand the role of `ImportableSynchronizableObject` in the synchronization process and to answer your questions. In any case, with time, I plan to describe the whole process of synchronization and share it with the developer community. Thank you for reading this! :-]

adam commented

2025-12-29 15:23:30 +01:00

@JohnEstropia commented on GitHub (Jul 2, 2016):

Thanks, I understand your synchronization process now. But I still think this design is a bit too app-specific for CoreStore's purposes.

Since you mention that the local ID is not stored in the server, you may want to check for a case where you might get duplicated objects.

An object is created with local ID in the client side.
The object is sent to the server
The server stores the object, assigns the server ID, and sends it back to the client together with the local ID requested.
Due to network failure or if the app was terminated, the client never receives the server ID for the local object.
On the next app execution, the client receives the previous object from the server, but because the server already forgot the local ID, the client cannot associate the server object and the local object. The server object is thus treated as a new entry.
Since to the client the previous local object was still considered "unsent", it re-sends it to the server and if the request goes through, the client will now have two objects created for that single instance.

You may be already aware of this, or I might just have misunderstood something else. If so, just disregard this comment :)

@JohnEstropia commented on GitHub (Jul 2, 2016): Thanks, I understand your synchronization process now. But I still think this design is a bit too app-specific for CoreStore's purposes. Since you mention that the local ID is not stored in the server, you may want to check for a case where you might get duplicated objects. 1. An object is created with local ID in the client side. 2. The object is sent to the server 3. The server stores the object, assigns the server ID, and sends it back to the client together with the local ID requested. 4. Due to network failure or if the app was terminated, the client never receives the server ID for the local object. 5. On the next app execution, the client receives the previous object from the server, but because the server already forgot the local ID, the client cannot associate the server object and the local object. The server object is thus treated as a new entry. 6. Since to the client the previous local object was still considered "unsent", it re-sends it to the server and if the request goes through, the client will now have two objects created for that single instance. You may be already aware of this, or I might just have misunderstood something else. If so, just disregard this comment :)

adam commented

2025-12-29 15:23:31 +01:00

@ruslanskorb commented on GitHub (Jul 2, 2016):

Between step 1 and step 2 the client sends the local object ID to the server and receives and saves the server object ID. The server does not create the object at this moment, it is only generates and returns the server object ID that will be sent by the client to the server in step 2. In step 3 the server stores the object and assigns the server object ID (from step 1.5) to it.

So, on the next app execution, the client receives the previous object from the server and updates it using the server object ID.

In the case where the client re-sends "unsent" local object to the server the server object ID will be used to find and merge the server object. If such an object does not exist the server will create a new one and assign this server object ID to it.

So, we will never get duplicated objects :-]

@ruslanskorb commented on GitHub (Jul 2, 2016): Between step 1 and step 2 the client sends the local object ID to the server and receives and saves the server object ID. The server does not create the object at this moment, it is only generates and returns the server object ID that will be sent by the client to the server in step 2. In step 3 the server stores the object and assigns the server object ID (from step 1.5) to it. So, on the next app execution, the client receives the previous object from the server and updates it using the server object ID. In the case where the client re-sends "unsent" local object to the server the server object ID will be used to find and merge the server object. If such an object does not exist the server will create a new one and assign this server object ID to it. So, we will never get duplicated objects :-]

adam commented

2025-12-29 15:23:31 +01:00

@JohnEstropia commented on GitHub (Jul 2, 2016):

I see, so the server just pre-assigns the server ID but does not store it yet. That makes more sense now, thanks :) Sounds like a reliable method 👍

Between use cases like these and the need for compound unique keys, I might consider providing a more flexible way to import objects in the future. Thanks again for sharing your ideas!

@JohnEstropia commented on GitHub (Jul 2, 2016): I see, so the server just pre-assigns the server ID but does not store it yet. That makes more sense now, thanks :) Sounds like a reliable method 👍 Between use cases like these and the need for compound unique keys, I might consider providing a more flexible way to import objects in the future. Thanks again for sharing your ideas!

adam commented

2025-12-29 15:23:32 +01:00

@ruslanskorb commented on GitHub (Jul 2, 2016):

Glad to hear it! 👍

I also will think about other more flexible ways to import objects and share my ideas with you.

Thanks for all your comments!

@ruslanskorb commented on GitHub (Jul 2, 2016): Glad to hear it! :thumbsup: I also will think about other more flexible ways to import objects and share my ideas with you. Thanks for all your comments!

Sign in to join this conversation.

Branches Tags

develop

master

prototype/concurrency

minIOS13

prototype/containers

transactionSourceTracking

prototype/swiftui

minIOS11

swift5_3

prototype/propertyWrapperFields

prototype/propertyWrappers

prototype/propertyWrapper

Swift5

throwables

prototype/Swift_4_1

prototype/Swift_4_0

prototype/Swift_3_2

prototype/queryBuilders

temp/develop

corestore4_develop

prototype/mainContextToPSC

master_ios_7_to_9

swift_2_3

gh-pages

_corespotlight-test

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/CoreStore#67