ListPublisher publishList with fetchOffset returns incomplete results #442

New Issue

adam · 2025-12-29T15:32:00+01:00

adam commented

2025-12-29 15:32:00 +01:00

Originally created by @KodaKoder on GitHub (Sep 30, 2025).

Description

When using publishList() with a query that includes fetchOffset via .tweak { fr in fr.fetchOffset = N }, the resulting snapshot contains fewer items than expected. The fetchOffset parameter is being applied twice: once by Core Data's NSFetchedResultsController and again during snapshot construction, causing items to be incorrectly skipped.

Environment

CoreStore Version: 9.3.0
Platform: iOS/macOS
Swift Version: 5.x
Xcode Version: 15.x

Steps to Reproduce

let dataStack: DataStack = // ... initialized DataStack

// Create query with fetchOffset
let query = From<MyEntity>()
    .where(\.someField == someValue)
    .orderBy(.descending(\.date))
    .tweak { fetchRequest in
        fetchRequest.fetchOffset = 5
        fetchRequest.fetchLimit = 10
    }

// Create publisher
let publisher = dataStack.publishList(query)
let snapshot = publisher.snapshot

// Check results
print("Snapshot count: \(snapshot.count)")  // Expected: 10, Actual: 5

Expected Behavior

When fetchOffset = 5 and fetchLimit = 10, the publisher should return items 5-14 (10 items total).

Actual Behavior

The publisher returns only items 10-14 (5 items total). The first 5 items from the Core Data fetch are skipped during snapshot creation.

Debug Evidence

Using LLDB breakpoint in controllerDidChangeContent:

controllerDidChangeContent snapshot.itemIdentifiers: 5       ❌ WRONG
controllerDidChangeContent fetchedObjects.count: 10          ✓ CORRECT
controllerDidChangeContent fetchOffset: 5 fetchLimit: 10

This proves:

NSFetchedResultsController correctly fetched 10 objects (items 5-14)
The snapshot incorrectly contains only 5 items (items 10-14)

Comparison with `fetchAll`

The bug does not occur with fetchAll():

let results = try dataStack.fetchAll(
    From<MyEntity>(),
    Where<MyEntity>(predicate),
    OrderBy<MyEntity>(sortDescriptors),
    Tweak { fetchRequest in
        fetchRequest.fetchOffset = 5
        fetchRequest.fetchLimit = 10
    }
)
print("Results count: \(results.count)")  // Correctly returns 10 items

Root Cause

In Internals.FetchedDiffableDataSourceSnapshotDelegate.swift, the controllerDidChangeContent method passes fetchOffset to the snapshot initializer:

var snapshot = Internals.DiffableDataSourceSnapshot(
    sections: controller.sections ?? [],
    sectionIndexTransformer: self.handler?.sectionIndexTransformer ?? { _ in nil },
    fetchOffset: controller.fetchRequest.fetchOffset,  // ❌ DOUBLE APPLICATION
    fetchLimit: controller.fetchRequest.fetchLimit
)

The problem: controller.sections already contains pre-filtered objects (items 5-14) because NSFetchedResultsController applied fetchOffset. The snapshot initializer then applies fetchOffset again to these already-offset results, causing items 5-9 to be skipped.

Flow of the bug:

fetchRequest.fetchOffset = 5, fetchLimit = 10
NSFetchedResultsController fetches items 5-14 from Core Data (10 items) ✓
controller.sections contains objects: [item5, item6, ..., item14]
Snapshot init receives fetchOffset: 5 and skips another 5 items
Result: snapshot contains [item10, ..., item14] (only 5 items) ❌

Proposed Fix

Change to always pass fetchOffset: 0:

// FIXED CODE
var snapshot = Internals.DiffableDataSourceSnapshot(
    sections: controller.sections ?? [],
    sectionIndexTransformer: self.handler?.sectionIndexTransformer ?? { _ in nil },
    fetchOffset: 0,  // ✓ Already applied by NSFetchedResultsController
    fetchLimit: controller.fetchRequest.fetchLimit
)

The sections passed from NSFetchedResultsController are already offset, so the snapshot initializer should not apply any additional offset.

Test Case

To verify the fix works:

// Query 1: offset=0, limit=10
let query1 = From<MyEntity>()
    .orderBy(.descending(\.date))
    .tweak { fr in
        fr.fetchOffset = 0
        fr.fetchLimit = 10
    }
let pub1 = dataStack.publishList(query1)
let items1 = pub1.snapshot.itemIdentifiers  // [item0...item9]

// Query 2: offset=5, limit=10
let query2 = From<MyEntity>()
    .orderBy(.descending(\.date))
    .tweak { fr in
        fr.fetchOffset = 5
        fr.fetchLimit = 10
    }
let pub2 = dataStack.publishList(query2)
let items2 = pub2.snapshot.itemIdentifiers  // Should be [item5...item14]

// Verify overlap: items1[5...9] should equal items2[0...4]
XCTAssertEqual(items1.suffix(5), items2.prefix(5))
XCTAssertEqual(items2.count, 10)  // Currently fails: returns 5

Impact

This bug breaks pagination implementations that rely on fetchOffset with publishList(). Developers using sliding window pagination or infinite scroll will see:

Missing items in the list
Incorrect item counts
Items appearing at wrong positions

Workaround

Until fixed, avoid using fetchOffset with publishList():

Use fetchAll() instead of publishList() for paginated queries
Or fetch all items without offset and slice in memory

Additional Notes

The BackingStructure.init method (lines 402-452 in Internals.DiffableDataSourceSnapshot.swift) correctly handles fetchOffset for its intended use case (un-offset sections). The bug is specifically in how controllerDidChangeContent calls this initializer with already-offset sections.

Originally created by @KodaKoder on GitHub (Sep 30, 2025). ## Description When using `publishList()` with a query that includes `fetchOffset` via `.tweak { fr in fr.fetchOffset = N }`, the resulting snapshot contains fewer items than expected. The `fetchOffset` parameter is being applied twice: once by Core Data's `NSFetchedResultsController` and again during snapshot construction, causing items to be incorrectly skipped. ## Environment - **CoreStore Version**: 9.3.0 - **Platform**: iOS/macOS - **Swift Version**: 5.x - **Xcode Version**: 15.x ## Steps to Reproduce ```swift let dataStack: DataStack = // ... initialized DataStack // Create query with fetchOffset let query = From<MyEntity>() .where(\.someField == someValue) .orderBy(.descending(\.date)) .tweak { fetchRequest in fetchRequest.fetchOffset = 5 fetchRequest.fetchLimit = 10 } // Create publisher let publisher = dataStack.publishList(query) let snapshot = publisher.snapshot // Check results print("Snapshot count: \(snapshot.count)") // Expected: 10, Actual: 5 ``` ## Expected Behavior When `fetchOffset = 5` and `fetchLimit = 10`, the publisher should return items 5-14 (10 items total). ## Actual Behavior The publisher returns only items 10-14 (5 items total). The first 5 items from the Core Data fetch are skipped during snapshot creation. ## Debug Evidence Using LLDB breakpoint in `controllerDidChangeContent`: ``` controllerDidChangeContent snapshot.itemIdentifiers: 5 ❌ WRONG controllerDidChangeContent fetchedObjects.count: 10 ✓ CORRECT controllerDidChangeContent fetchOffset: 5 fetchLimit: 10 ``` This proves: 1. `NSFetchedResultsController` correctly fetched 10 objects (items 5-14) 2. The snapshot incorrectly contains only 5 items (items 10-14) ## Comparison with `fetchAll` The bug does **not** occur with `fetchAll()`: ```swift let results = try dataStack.fetchAll( From<MyEntity>(), Where<MyEntity>(predicate), OrderBy<MyEntity>(sortDescriptors), Tweak { fetchRequest in fetchRequest.fetchOffset = 5 fetchRequest.fetchLimit = 10 } ) print("Results count: \(results.count)") // Correctly returns 10 items ``` ## Root Cause In `Internals.FetchedDiffableDataSourceSnapshotDelegate.swift`, the `controllerDidChangeContent` method passes `fetchOffset` to the snapshot initializer: ```swift var snapshot = Internals.DiffableDataSourceSnapshot( sections: controller.sections ?? [], sectionIndexTransformer: self.handler?.sectionIndexTransformer ?? { _ in nil }, fetchOffset: controller.fetchRequest.fetchOffset, // ❌ DOUBLE APPLICATION fetchLimit: controller.fetchRequest.fetchLimit ) ``` The problem: `controller.sections` already contains **pre-filtered** objects (items 5-14) because `NSFetchedResultsController` applied `fetchOffset`. The snapshot initializer then applies `fetchOffset` **again** to these already-offset results, causing items 5-9 to be skipped. **Flow of the bug:** 1. `fetchRequest.fetchOffset = 5, fetchLimit = 10` 2. `NSFetchedResultsController` fetches items 5-14 from Core Data (10 items) ✓ 3. `controller.sections` contains objects: `[item5, item6, ..., item14]` 4. Snapshot init receives `fetchOffset: 5` and skips **another** 5 items 5. Result: snapshot contains `[item10, ..., item14]` (only 5 items) ❌ ## Proposed Fix Change to always pass `fetchOffset: 0`: ```swift // FIXED CODE var snapshot = Internals.DiffableDataSourceSnapshot( sections: controller.sections ?? [], sectionIndexTransformer: self.handler?.sectionIndexTransformer ?? { _ in nil }, fetchOffset: 0, // ✓ Already applied by NSFetchedResultsController fetchLimit: controller.fetchRequest.fetchLimit ) ``` The sections passed from `NSFetchedResultsController` are already offset, so the snapshot initializer should not apply any additional offset. ## Test Case To verify the fix works: ```swift // Query 1: offset=0, limit=10 let query1 = From<MyEntity>() .orderBy(.descending(\.date)) .tweak { fr in fr.fetchOffset = 0 fr.fetchLimit = 10 } let pub1 = dataStack.publishList(query1) let items1 = pub1.snapshot.itemIdentifiers // [item0...item9] // Query 2: offset=5, limit=10 let query2 = From<MyEntity>() .orderBy(.descending(\.date)) .tweak { fr in fr.fetchOffset = 5 fr.fetchLimit = 10 } let pub2 = dataStack.publishList(query2) let items2 = pub2.snapshot.itemIdentifiers // Should be [item5...item14] // Verify overlap: items1[5...9] should equal items2[0...4] XCTAssertEqual(items1.suffix(5), items2.prefix(5)) XCTAssertEqual(items2.count, 10) // Currently fails: returns 5 ``` ## Impact This bug breaks pagination implementations that rely on `fetchOffset` with `publishList()`. Developers using sliding window pagination or infinite scroll will see: - Missing items in the list - Incorrect item counts - Items appearing at wrong positions ## Workaround Until fixed, avoid using `fetchOffset` with `publishList()`: - Use `fetchAll()` instead of `publishList()` for paginated queries - Or fetch all items without offset and slice in memory ## Additional Notes The `BackingStructure.init` method (lines 402-452 in `Internals.DiffableDataSourceSnapshot.swift`) correctly handles `fetchOffset` for its intended use case (un-offset sections). The bug is specifically in how `controllerDidChangeContent` calls this initializer with already-offset sections.

Sign in to join this conversation.

Branches Tags

develop

master

prototype/concurrency

minIOS13

prototype/containers

transactionSourceTracking

prototype/swiftui

minIOS11

swift5_3

prototype/propertyWrapperFields

prototype/propertyWrappers

prototype/propertyWrapper

Swift5

throwables

prototype/Swift_4_1

prototype/Swift_4_0

prototype/Swift_3_2

prototype/queryBuilders

temp/develop

corestore4_develop

prototype/mainContextToPSC

master_ios_7_to_9

swift_2_3

gh-pages

_corespotlight-test

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/CoreStore#442

ListPublisher publishList with fetchOffset returns incomplete results #442