starred/CoreStore
1
0
Fork 0
mirror of https://github.com/JohnEstropia/CoreStore.git synced 2026-01-11 20:00:30 +01:00
Code Issues 95 Packages Projects Releases 10 Wiki Activity

Question: How to create a new entity relationship during CoreStoreObject migration? #233

New Issue
Closed
opened 2025-12-29 15:27:06 +01:00 by adam · 3 comments
adam commented 2025-12-29 15:27:06 +01:00
Owner
Copy Link

Originally created by @scogeo on GitHub (Oct 11, 2018).

I'm looking at CoreStore for a future project and really like what I see so far, but I have a question about more complex migrations when using CoreStoreObjects rather than model files.

I created a quick test project that has three schema versions. The first version, V1, is pretty straightforward, and the second version, V2, just adds an additional field so migration is automatic.

However in the 3rd version, the data model is refactored to take the image attribute in the V2 data model and move it into a new child entity. This is straight forward to do in Core Data with a mapping file. However, I can't seem to find a way to do it with CoreStore.

I'm looking at what it would take to support this in CoreStore, but before I dig in further I wanted to make sure I wasn't missing something obvious.

I've coded up below what I would like to happen in the attachment_v2_to_v3_mapping. Essentially, I want to migrate the Notes over and delete the image attribute using the CustomMapping.inferredTransformation. Then run through all the Notes again and this time migrate them to Attachments if they have an image in the source object. Again, this is possible with a mapping model file. With CoreStore, this produces an assertion error:

[CoreStore: Assertion Failure] CustomSchemaMappingProvider.swift:56 init(from:to:entityMappings:)
↪︎ Duplicate source/destination entities found in provided "entityMappings" argument.

Any advice?

typealias Note = V3.Note
typealias Attachment = V3.Attachment

enum V1 {
  class Note: CoreStoreObject {
    let title = Value.Required<String>("title", initial: "")
    let body = Value.Required<String>("body", initial: "")
    let dateCreated = Value.Required<Date>("dateCreated", initial: Date())
    let displayIndex = Value.Required<Int64>("displayIndex", initial: 0)
    
  }
  
  static let schema = CoreStoreSchema(
    modelVersion: "V1",
    entities: [
      Entity<Note>("Note")
    ],
    versionLock: [
      "Note": [0x621f63992e6f40a0, 0x2dc8a66cc0960cc6, 0x961f9c422bd578d7, 0xe52ab5762210e51]
    ]
  )
}

enum V2 {
  class Note: CoreStoreObject {
    let title = Value.Required<String>("title", initial: "")
    let body = Value.Required<String>("body", initial: "")
    let dateCreated = Value.Required<Date>("dateCreated", initial: Date())
    let displayIndex = Value.Required<Int64>("displayIndex", initial: 0)
    
    // Added image
    let image = Transformable.Optional<UIImage>("image")
  }
  
  static let schema = CoreStoreSchema(
    modelVersion: "V2",
    entities: [
      Entity<Note>("Note")
    ],
    versionLock: [
      "Note": [0xb785f1bcd7e018b5, 0xdc36121b91024133, 0xf2383ccafb52c8e1, 0x7c17a7ed12412b17]
    ]
  )
}

enum V3 {
  class Note: CoreStoreObject {
    let title = Value.Required<String>("title", initial: "")
    let body = Value.Required<String>("body", initial: "")
    let dateCreated = Value.Required<Date>("dateCreated", initial: Date())
    let displayIndex = Value.Required<Int64>("displayIndex", initial: 0)

    // Replaced image with attachment relationship
    let attachments = Relationship.ToManyUnordered<Attachment>("attachments", inverse: { $0.note })
  }
  
  class Attachment : CoreStoreObject {
    let dateCreated = Value.Required<Date>("dateCreated", initial: Date())
    let image = Transformable.Optional<UIImage>("image")
    
    let note = Relationship.ToOne<Note>("note")
  }
  
  static let schema = CoreStoreSchema(
    modelVersion: "V3",
    entities: [
      Entity<Note>("Note"),
      Entity<Attachment>("Attachment")
    ],
    versionLock: [
      "Attachment": [0x65acc9405d37a9ba, 0x219ce4458ff8a171, 0x26b9d4dd80d4128f, 0xd2af23109c72729c],
      "Note": [0xef20204dbdeae54f, 0x2739ea7390afc9d2, 0xe182b346a06d5988, 0xe1688b713aa5d8d2]
    ]
  )
  
  static let attachment_v2_to_v3_mapping = CustomSchemaMappingProvider(
    from: "V2",
    to: "V3",
    entityMappings: [
      .transformEntity(
        sourceEntity: "Note",
        destinationEntity: "Note",
        transformer: CustomSchemaMappingProvider.CustomMapping.inferredTransformation
        ),
      .transformEntity(
        sourceEntity: "Note",
        destinationEntity: "Attachment",
        transformer: { (sourceObject: CustomSchemaMappingProvider.UnsafeSourceObject, createDestinationObject: () -> CustomSchemaMappingProvider.UnsafeDestinationObject) in
          if (sourceObject["image"] == nil) {
            return
          }
          let destinationObject = createDestinationObject()
          
          // how to link up the relationship? something like this would be nice
          //destinationObject["note"] = migratedSourceObject
          
          destinationObject.enumerateAttributes { (attribute, sourceAttribute) in
            if let sourceAttribute = sourceAttribute {
              destinationObject[attribute] = sourceObject[sourceAttribute]
            }
          }
      }
      )
     
    ]
  )
}

let noteDataStack = DataStack(
  V1.schema,
  V2.schema,
  V3.schema,
  migrationChain: [ "V1", "V2", "V3" ]
)
Originally created by @scogeo on GitHub (Oct 11, 2018). I'm looking at CoreStore for a future project and really like what I see so far, but I have a question about more complex migrations when using CoreStoreObjects rather than model files. I created a quick test project that has three schema versions. The first version, V1, is pretty straightforward, and the second version, V2, just adds an additional field so migration is automatic. However in the 3rd version, the data model is refactored to take the image attribute in the V2 data model and move it into a new child entity. This is straight forward to do in Core Data with a mapping file. However, I can't seem to find a way to do it with CoreStore. I'm looking at what it would take to support this in CoreStore, but before I dig in further I wanted to make sure I wasn't missing something obvious. I've coded up below what I would like to happen in the `attachment_v2_to_v3_mapping`. Essentially, I want to migrate the Notes over and delete the `image` attribute using the `CustomMapping.inferredTransformation`. Then run through all the Notes again and this time migrate them to Attachments if they have an image in the source object. Again, this is possible with a mapping model file. With CoreStore, this produces an assertion error: >[CoreStore: Assertion Failure] CustomSchemaMappingProvider.swift:56 init(from:to:entityMappings:) > ↪︎ Duplicate source/destination entities found in provided "entityMappings" argument. Any advice? ```swift typealias Note = V3.Note typealias Attachment = V3.Attachment enum V1 { class Note: CoreStoreObject { let title = Value.Required<String>("title", initial: "") let body = Value.Required<String>("body", initial: "") let dateCreated = Value.Required<Date>("dateCreated", initial: Date()) let displayIndex = Value.Required<Int64>("displayIndex", initial: 0) } static let schema = CoreStoreSchema( modelVersion: "V1", entities: [ Entity<Note>("Note") ], versionLock: [ "Note": [0x621f63992e6f40a0, 0x2dc8a66cc0960cc6, 0x961f9c422bd578d7, 0xe52ab5762210e51] ] ) } enum V2 { class Note: CoreStoreObject { let title = Value.Required<String>("title", initial: "") let body = Value.Required<String>("body", initial: "") let dateCreated = Value.Required<Date>("dateCreated", initial: Date()) let displayIndex = Value.Required<Int64>("displayIndex", initial: 0) // Added image let image = Transformable.Optional<UIImage>("image") } static let schema = CoreStoreSchema( modelVersion: "V2", entities: [ Entity<Note>("Note") ], versionLock: [ "Note": [0xb785f1bcd7e018b5, 0xdc36121b91024133, 0xf2383ccafb52c8e1, 0x7c17a7ed12412b17] ] ) } enum V3 { class Note: CoreStoreObject { let title = Value.Required<String>("title", initial: "") let body = Value.Required<String>("body", initial: "") let dateCreated = Value.Required<Date>("dateCreated", initial: Date()) let displayIndex = Value.Required<Int64>("displayIndex", initial: 0) // Replaced image with attachment relationship let attachments = Relationship.ToManyUnordered<Attachment>("attachments", inverse: { $0.note }) } class Attachment : CoreStoreObject { let dateCreated = Value.Required<Date>("dateCreated", initial: Date()) let image = Transformable.Optional<UIImage>("image") let note = Relationship.ToOne<Note>("note") } static let schema = CoreStoreSchema( modelVersion: "V3", entities: [ Entity<Note>("Note"), Entity<Attachment>("Attachment") ], versionLock: [ "Attachment": [0x65acc9405d37a9ba, 0x219ce4458ff8a171, 0x26b9d4dd80d4128f, 0xd2af23109c72729c], "Note": [0xef20204dbdeae54f, 0x2739ea7390afc9d2, 0xe182b346a06d5988, 0xe1688b713aa5d8d2] ] ) static let attachment_v2_to_v3_mapping = CustomSchemaMappingProvider( from: "V2", to: "V3", entityMappings: [ .transformEntity( sourceEntity: "Note", destinationEntity: "Note", transformer: CustomSchemaMappingProvider.CustomMapping.inferredTransformation ), .transformEntity( sourceEntity: "Note", destinationEntity: "Attachment", transformer: { (sourceObject: CustomSchemaMappingProvider.UnsafeSourceObject, createDestinationObject: () -> CustomSchemaMappingProvider.UnsafeDestinationObject) in if (sourceObject["image"] == nil) { return } let destinationObject = createDestinationObject() // how to link up the relationship? something like this would be nice //destinationObject["note"] = migratedSourceObject destinationObject.enumerateAttributes { (attribute, sourceAttribute) in if let sourceAttribute = sourceAttribute { destinationObject[attribute] = sourceObject[sourceAttribute] } } } ) ] ) } let noteDataStack = DataStack( V1.schema, V2.schema, V3.schema, migrationChain: [ "V1", "V2", "V3" ] ) ```
adam added the question label 2025-12-29 15:27:06 +01:00
adam closed this issue 2025-12-29 15:27:06 +01:00
adam commented 2025-12-29 15:27:06 +01:00
Author
Owner
Copy Link

@JohnEstropia commented on GitHub (Oct 12, 2018):

Ah, right now the custom mappings declaration is designed so that we don't need to write inferrable mappings. This is contrary to .xcmappingmodel where all migrations are declared, including inferred transforms.

I'm not sure this would be easy to support in the migrator's current form. Multiple destinations for a single source may be possible by removing the asserts and commenting out a few lines of code, but setting the relationship would be particularly difficult. For example,

//destinationObject["note"] = migratedSourceObject

We'll need a second pass for relationships where we can fetch created objects during the first pass.

If you really need to do this now, I'm afraid the only option is to process new objects manually right after CoreStore's migration completes. You can also opt to keep using .xcmappingmodel using XcodeDataModelSchema and XcodeSchemaMappingProvider, albeit you'll be using NSManagedObjects instead of CoreStoreObjects.

Also, just a heads up,

let dateCreated = Value.Required<Date>("dateCreated", initial: Date())

the initial: argument here is static. Meaning the Date() instance will be used for all new instance of the object and will not be the date during the object's creation time.

@JohnEstropia commented on GitHub (Oct 12, 2018): Ah, right now the custom mappings declaration is designed so that we don't need to write inferrable mappings. This is contrary to `.xcmappingmodel` where all migrations are declared, including inferred transforms. I'm not sure this would be easy to support in the migrator's current form. Multiple destinations for a single source may be possible by removing the asserts and commenting out a few lines of code, but setting the relationship would be particularly difficult. For example, ``` //destinationObject["note"] = migratedSourceObject ``` We'll need a second pass for relationships where we can fetch created objects during the first pass. If you really need to do this now, I'm afraid the only option is to process new objects manually right after CoreStore's migration completes. You can also opt to keep using `.xcmappingmodel` using `XcodeDataModelSchema` and `XcodeSchemaMappingProvider`, albeit you'll be using `NSManagedObject`s instead of `CoreStoreObject`s. Also, just a heads up, ```swift let dateCreated = Value.Required<Date>("dateCreated", initial: Date()) ``` the `initial:` argument here is static. Meaning the `Date()` instance will be used for all new instance of the object and will not be the date during the object's creation time.
adam commented 2025-12-29 15:27:06 +01:00
Author
Owner
Copy Link

@scogeo commented on GitHub (Oct 12, 2018):

Thanks for the response. I really like the simplicity of using CoreStoreObjects, so I think I will still pursue that path and go with a multi-pass migration if something like this comes up as the project evolves.

I suppose in this example, I could create a V2.5 schema which contains the new entity and relationship but doesn't delete any attributes in 'Note', and allow the migration to proceed automatically. I could run my own code to hook up the relationships on the V2.5 database, then finally migrate it over to V3.

I'll code it up in my prototype and post a snippet here for reference in case anyone else has a similar issue.

Also thanks for pointing out the problem with Date() in the initial:. This is just a toy prototype for playing with CoreStore, but good to know for my real project.

@scogeo commented on GitHub (Oct 12, 2018): Thanks for the response. I really like the simplicity of using `CoreStoreObject`s, so I think I will still pursue that path and go with a multi-pass migration if something like this comes up as the project evolves. I suppose in this example, I could create a `V2.5` schema which contains the new entity and relationship but doesn't delete any attributes in 'Note', and allow the migration to proceed automatically. I could run my own code to hook up the relationships on the `V2.5` database, then finally migrate it over to `V3`. I'll code it up in my prototype and post a snippet here for reference in case anyone else has a similar issue. Also thanks for pointing out the problem with `Date()` in the `initial:`. This is just a toy prototype for playing with CoreStore, but good to know for my real project.
adam commented 2025-12-29 15:27:07 +01:00
Author
Owner
Copy Link

@scogeo commented on GitHub (Oct 16, 2018):

I prototyped a more complex migration example using two DataStacks and an additional migration schema. The basic idea is to migrate the first stack to the migration schema and add it to aDataStack perform some DB operations to link up the new relationships. Then close that stack and load a new stack which migrates to the final schema.

I've included a few snippets below in case anyone else finds them useful. Otherwise, this issue can probably be closed.

Here's a migration schema that contains the additions from the V3 schema without any deletions.

enum V2_3_migrate {

  class Note: CoreStoreObject {
    let title = Value.Required<String>("title", initial: "")
    let body = Value.Required<String>("body", initial: "")
    let dateCreated = Value.Required<Date>("dateCreated", initial: Date.distantPast)
    let displayIndex = Value.Required<Int64>("displayIndex", initial: 0)
    let imageData = Transformable.Optional<UIImage>("image")

    let attachments = Relationship.ToManyUnordered<Attachment>("attachments", inverse: { $0.note })
  }
  
  class Attachment : CoreStoreObject {
    let dateCreated = Value.Required<Date>("dateCreated", initial: Date.distantPast)
    let image = Transformable.Optional<UIImage>("image")
    
    let note = Relationship.ToOne<Note>("note")
  }
  
  static let schema = CoreStoreSchema(
    modelVersion: "V2_3_migrate",
    entities: [
      Entity<Note>("Note"),
      Entity<Attachment>("Attachment")
    ],
    versionLock: [
      "Attachment": [0x65acc9405d37a9ba, 0x219ce4458ff8a171, 0x26b9d4dd80d4128f, 0xd2af23109c72729c],
      "Note": [0xa446268f413fe436, 0xf2738ccee76c5542, 0xe800cc7394a0d6d1, 0x414ef30e7756cf6a]
    ]
  )
}

let noteDataStackV3 = DataStack(
  V2_3_migrate.schema,
  V3.schema,
  migrationChain: [ V2_3_migrate.schema.modelVersion , V3.schema.modelVersion ]
)

Then the code to perform the two-step migration:

  let storage = ...
  do {
      let fileURL = storage.fileURL
      let metadata = try NSPersistentStoreCoordinator.metadataForPersistentStore(
        ofType: type(of: storage).storeType,
        at: fileURL as URL,
        options: storage.storeOptions
      )
      
      // Is the current storage version prior to V3?
      var needsMigration = false
      for schema in [V1.schema, V2.schema, V2_3_migrate.schema] {
        if schema.rawModel().isConfiguration(withName: storage.configuration, compatibleWithStoreMetadata: metadata) {
          needsMigration = true
          break
        }
      }
      
      let finalCompletionHandler: () -> Void = {
        CoreStore.defaultStack = noteDataStackV3
        NotificationCenter.default.post(
          name: Notification.Name.didChangeDataStackNotification,
          object: self,
          userInfo: [ "stack" : noteDataStackV3 ]
        )
      }
      
      if needsMigration {
        self.performMigration(storage) {
          self.loadAndMigrateStack(dataStack:noteDataStackV3, storage: storage) {
            finalCompletionHandler()
          }
        }
      }
      else {
        self.loadAndMigrateStack(dataStack:noteDataStackV3, storage: storage) {
          finalCompletionHandler()
        }
      }
      
    }
    catch {
      print("error reading metadata \(error)")
      fatalError()
    }

And a few helper functions:

func loadAndMigrateStack(dataStack: DataStack, storage: SQLiteStore, completionHandler: @escaping () -> Void) {
    let migrationProgress: Progress? = dataStack.addStorage(
      storage,
      completion: { (result) -> Void in
        switch result {
        case .success(let storage):
          completionHandler()
        case .failure(let error):
          print("Failed adding sqlite store with error: \(error)")
          fatalError()
        }
    }
    )
    
    migrationProgress?.setProgressHandler { [weak self] (progress) -> Void in
      print(progress.localizedDescription)
    }
  }

  func performMigration(_ storage: SQLiteStore, completionHandler: @escaping () -> Void) {
      // Advance to the end of this data stack.
    let noteDataStackV1 = DataStack(
      V1.schema,
      V2.schema,
      V2_3_migrate.schema,
      migrationChain: [
        V1.schema.modelVersion,
        V2.schema.modelVersion,
        V2_3_migrate.schema.modelVersion ]
    )
    
    loadAndMigrateStack(dataStack: noteDataStackV1, storage: storage) {
      // Do The image conversion
      if let notesWithImages : [V2_3_migrate.Note] = noteDataStackV1.fetchAll(
        From<V2_3_migrate.Note>(),
        Where<V2_3_migrate.Note>("image != nil")
        ) {
        
        for note in notesWithImages {
          do {
            try noteDataStackV1.perform(
              synchronous: { (transaction) -> Void in
                let image = note.imageData.value
                let note = transaction.edit(note)!                
                let attachment = transaction.create(Into<V2_3_migrate.Attachment>())
                
                attachment.dateCreated .= note.dateCreated
                attachment.image .= image
                attachment.note .= note
                
                note.imageData .= nil                
            })
          }
          catch {
            print("error migrating notes")
          }
        }
        
        completionHandler()
      }
    }   
  }
@scogeo commented on GitHub (Oct 16, 2018): I prototyped a more complex migration example using two `DataStack`s and an additional migration schema. The basic idea is to migrate the first stack to the migration schema and add it to a`DataStack` perform some DB operations to link up the new relationships. Then close that stack and load a new stack which migrates to the final schema. I've included a few snippets below in case anyone else finds them useful. Otherwise, this issue can probably be closed. Here's a migration schema that contains the additions from the `V3` schema without any deletions. ```swift enum V2_3_migrate { class Note: CoreStoreObject { let title = Value.Required<String>("title", initial: "") let body = Value.Required<String>("body", initial: "") let dateCreated = Value.Required<Date>("dateCreated", initial: Date.distantPast) let displayIndex = Value.Required<Int64>("displayIndex", initial: 0) let imageData = Transformable.Optional<UIImage>("image") let attachments = Relationship.ToManyUnordered<Attachment>("attachments", inverse: { $0.note }) } class Attachment : CoreStoreObject { let dateCreated = Value.Required<Date>("dateCreated", initial: Date.distantPast) let image = Transformable.Optional<UIImage>("image") let note = Relationship.ToOne<Note>("note") } static let schema = CoreStoreSchema( modelVersion: "V2_3_migrate", entities: [ Entity<Note>("Note"), Entity<Attachment>("Attachment") ], versionLock: [ "Attachment": [0x65acc9405d37a9ba, 0x219ce4458ff8a171, 0x26b9d4dd80d4128f, 0xd2af23109c72729c], "Note": [0xa446268f413fe436, 0xf2738ccee76c5542, 0xe800cc7394a0d6d1, 0x414ef30e7756cf6a] ] ) } let noteDataStackV3 = DataStack( V2_3_migrate.schema, V3.schema, migrationChain: [ V2_3_migrate.schema.modelVersion , V3.schema.modelVersion ] ) ``` Then the code to perform the two-step migration: ```swift let storage = ... do { let fileURL = storage.fileURL let metadata = try NSPersistentStoreCoordinator.metadataForPersistentStore( ofType: type(of: storage).storeType, at: fileURL as URL, options: storage.storeOptions ) // Is the current storage version prior to V3? var needsMigration = false for schema in [V1.schema, V2.schema, V2_3_migrate.schema] { if schema.rawModel().isConfiguration(withName: storage.configuration, compatibleWithStoreMetadata: metadata) { needsMigration = true break } } let finalCompletionHandler: () -> Void = { CoreStore.defaultStack = noteDataStackV3 NotificationCenter.default.post( name: Notification.Name.didChangeDataStackNotification, object: self, userInfo: [ "stack" : noteDataStackV3 ] ) } if needsMigration { self.performMigration(storage) { self.loadAndMigrateStack(dataStack:noteDataStackV3, storage: storage) { finalCompletionHandler() } } } else { self.loadAndMigrateStack(dataStack:noteDataStackV3, storage: storage) { finalCompletionHandler() } } } catch { print("error reading metadata \(error)") fatalError() } ``` And a few helper functions: ```swift func loadAndMigrateStack(dataStack: DataStack, storage: SQLiteStore, completionHandler: @escaping () -> Void) { let migrationProgress: Progress? = dataStack.addStorage( storage, completion: { (result) -> Void in switch result { case .success(let storage): completionHandler() case .failure(let error): print("Failed adding sqlite store with error: \(error)") fatalError() } } ) migrationProgress?.setProgressHandler { [weak self] (progress) -> Void in print(progress.localizedDescription) } } func performMigration(_ storage: SQLiteStore, completionHandler: @escaping () -> Void) { // Advance to the end of this data stack. let noteDataStackV1 = DataStack( V1.schema, V2.schema, V2_3_migrate.schema, migrationChain: [ V1.schema.modelVersion, V2.schema.modelVersion, V2_3_migrate.schema.modelVersion ] ) loadAndMigrateStack(dataStack: noteDataStackV1, storage: storage) { // Do The image conversion if let notesWithImages : [V2_3_migrate.Note] = noteDataStackV1.fetchAll( From<V2_3_migrate.Note>(), Where<V2_3_migrate.Note>("image != nil") ) { for note in notesWithImages { do { try noteDataStackV1.perform( synchronous: { (transaction) -> Void in let image = note.imageData.value let note = transaction.edit(note)! let attachment = transaction.create(Into<V2_3_migrate.Attachment>()) attachment.dateCreated .= note.dateCreated attachment.image .= image attachment.note .= note note.imageData .= nil }) } catch { print("error migrating notes") } } completionHandler() } } } ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
abandoned
corestore bug
discussion
duplicate
enhancement
fixed
fixed
help wanted
in progress
investigating
ios/compiler bug
irreproducible
looking for workarounds
not a bug
pending
pending docs update
pull-request
Mirrored from GitHub Pull Request
 question
regression
user bug
wontfix
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/CoreStore#233
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?