When developing apps, we almost always end up having to handle some sort of pagination or another.

Often, it involves keeping some page number around. Sometimes, it requires some logic involving the current page number and the total number of pages (ie most REST APIs, including Algolia's). Occasionaly, you may end up handling some cursor and passing it around (Firestore and DynamoDB come to mind). And I'm pretty sure there are many other ways this is done out there.

When you only have to deal with one or the other, it's a straightforward no-brainer. But every once in a while, your rare stamps marketplace app ends up having to display both simple ordered lists and search results, using both Firestore and Algolia as datasources, to display the exact same thing, using very different pagination behaviors.

Fun fact: unlike most collectibles, stamps apparently have to have been used to be of any value - Photo by Sonia Kardash / Unsplash

Or perhaps you'll be alternating between a local source (such as CoreData or Realm) and a remote source, depending on the connection state. Or maybe, like it happened to me last March with Spotr, you'll have to switch backends, from a service that is shutting down in two weeks (Stamplay) to another that may be shut down at any given time without any warning (Firebase is, after all, owned by Google and part of GCP).

In any case, you shouldn't recode your collections for each separate datasource. Yes, you could handle this is in your ViewModels or Presenter and expose it transparently to your views. But is it their problem to begin with?

Your ViewModel, crying for help - Photo by Priscilla Du Preez / Unsplash

Not for me. My ViewModels' work is to decide when to get the next batch of results, eventually transform the data for my view, and pass it along. That's it. Not figure out whether my current page number is less than or equal to the total number of pages - 1, or whether or not last call returned the exact number of items I asked for. And certainly not to try and keep track of different pagination values that may or may not be optional.

We shouldn't have to spread multiple pagination patterns handling all over our app. It should be contained. It should be DRY. It should offer a unique, consistent API whatever the source is. And it should be totally invisible to the rest of our code.

Overview

How do we do that? First, we define a custom 'Result Type', with two properties: the fetched result, wether it's a single item or a list, and some optional function returning the next batch of results. Next, we make it so that every single one of our API calls wrap their return value with our new 'Result Type'.

Ideally, this would look like this

typealias CustomResultType = (value: ItemType, next: (() -> CustomResultType)?)

private func somePrivatePaginatedApiFunction(param: ParamType, page: Int = 0) -> ItemType {
    ... your code here
    return (result: result, currentPage: page, totalPages: totalPages)
}

public func someExposedApiFunction = (param: ParamType) -> CustomResultType {
    const result = somePrivatePaginatedApiFunction(param, 0) 
    const next = result.currentPage == result.totalPages - 1 ? nil : () -> somePrivatePaginatedApiFunction(params, result.currentPage + 1)
    return (value: result.result, next: next) 
}

callbacks voluntarily omitted for clarity

That's the main idea. Sadly, it just doesn't work this way. If you try it, XCode will just scream at you, saying Type alias 'CustomResultType' references itself, because our type alias does reference itself. Also, our type alias's hardcoded 'ItemType' definitely isn't anywhere near universal. Let's overcome these issues.

How? Generics and, as you might have already noticed, a healthy dose of functional programming. For those of you who are unfamiliar with those and want to know more about them, keep reading. Everyone else, skip to the implementation.

The tools

Generics

Aside from being intrinsically awesomely awesome, generic types are the go-to tool whenever you want to make a type... uh... well, generic.

My explanation, just as clear as this road - Photo by Markus Spiske / Unsplash

Kidding aside, generics are what you turn to when you want to make classes, methods, structures or functions that can take different type of parameters. Such as a function that can swap two items' positions in an array, no matter the array's type. Some function that could add either Ints, Doubles or CGFloats, depending on what you give it at runtime. Or, say... a result enum who's values can be of any type, without losing said type! (using that dreadful Any would lose the values' types).

If you're interested, or you'd like to get a better grasp of the concept, go have a look at the official Swift documentation, and this nice Raywenderlich tutorial. They should make it crystal clear.

Functional Programming

The other awesome tool we'll be using is the functional programming paradigm. Well, a very small subset of it; nothing really fancy: the abilities to return a function, and to pass functions as parameters. That's possible because functions in Swift are said to be first class (see Sundell's great article on first class functions), meaning we can use them just like any other values.

If you've ever used map, flatMap, filter or reduce, then you've already encountered functional programming, because these functions take your mapping, filtering or reducing functions as arguments, making them what we call higher order functions (here's a nice post on that).

Also, a very nice introduction tutorial for functional programming in Swift, by Ray Wenderlich. These guys make lots of tutorials.

So, basically, we'll be making a higher order function that returns a function, which will return both our next page and, if applicable the function returning the page after that, as well as... Yes, this is recursive. And awesome!

This is totally appropriate. Because snowflakes are fractals. And fractals are great recursivity examples. Also, it'll soon be Christmas, no matter what xkcd says - Photo by Damian McCoig / Unsplash

Implementation

The generic, custom, Result type

The workaround for the self-referencing typealias is to use an enum (shoutout to Nibr), with the current result(s) and the function to get the next batch of results as optional associated values.

enum CustomResult {
    indirect case node(ItemType?, (() -> CustomResult)?)
}

AND. IT. BUILDS! Nice! Next, make it generic!

enum GenericCustomResult<T> {
    indirect case node(T?, (() -> GenericCustomResult<T>)?)
}

Great! We have our generic result type. Now, we simplify it's usage. Accessing an enum's cases' associated values is a bit of a pain, we'll be using this specific one all over the place, and since there's only one case, we don't want the rest of our app to have to deal with it each and every time we use it.

enum GenericCustomResult<T> {
    indirect case node(T?, (() -> GenericCustomResult<T>)?)
    
    var value: T? {
        if case let GenericCustomResult.node(value, _) = self {
            return value
        }
        return nil
    }
    
    var next: (() -> GenericCustomResult<T>)? {
        if case let GenericCustomResult.node(_, next) = self {
            return next
        }
        return nil
    }
}

Finally, the completion block. Most if not all of these calls will be asynchronous, so our 'next' needs to take a completion function, in order to return the results.

enum GenericCustomResult<T> {
    indirect case node(T?, ((@escaping (GenericCustomResult<T>, Error?) -> Void) -> ())?)
    
    var value: T? {
        if case let GenericCustomResult.node(value, _) = self {
            return value
        }
        return nil
    }
    
    var next: ((@escaping (GenericCustomResult<T>, Error?) -> Void) -> ())? {
        if case let GenericCustomResult.node(_, next) = self {
            return next
        }
        return nil
    }
}

Yes, it's not as nice as it was in the beginning. But it'll work nicely, you'll see

The returning method

We have a return type. What would an API call returning this new type look like? It depends, based on the API your app consummes (wether it uses pages or cursors; wether it does or doesn't return the total number of pages,...), and the tools you use to do so. But first, let's define our data's model.

struct RandomDataStructure: Codable {
    let firstProperty: Int
    let secondProperty: String
}

Nothing fancy

URLSession

URLSession is the original, native, iOS framework for networking.

func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping ([RandomDataStructure]?, Error?) -> Void) {
    let session = URLSession(configuration: .default)
    
    guard var urlComponents = URLComponents(string: "https://yourdomain.com/yourRoute") else {
        completion(nil, someError)
        return
    }
    urlComponents.query = "term=\(searchTerm)&page=\(page)"
    
    guard let url = urlComponents.url else {
        return
    }
    
    let dataTask = session.dataTask(with: url) { data, response, error in
        
        if let error = error {
            return completion(nil, error)
        } else if let data = data,
            let response = response as? HTTPURLResponse,
            response.statusCode == 200 {
            do {
                let results = try JSONDecoder().decode([RandomDataStructure].self, from: data)
                return completion(results, nil)
            } catch (let error) {
                return completion(nil, error)
            }
        }
    }
    
    dataTask.resume()
}

Initial code fetching some random data from your API

func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
    let session = URLSession(configuration: .default)
    
    guard var urlComponents = URLComponents(string: "https://yourdomain.com/yourRoute") else {
        completion(nil, someError)
        return
    }
    urlComponents.query = "term=\(searchTerm)&page=\(page)"
    
    guard let url = urlComponents.url else {
        completion(nil, someOtherError)
        return
    }
    let dataTask = session.dataTask(with: url) { data, response, error in
        if let error = error {
            return completion(GenericCustomResult.node(nil, nil), error)
        } else if let data = data,
            let response = response as? HTTPURLResponse,
            response.statusCode == 200 {
            do {
                let results = try JSONDecoder().decode([RandomDataStructure].self, from: data)
                let next = { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
                    getSearchResults(searchTerm: searchTerm, page: page + 1, completion: completion)
                }
                return completion(GenericCustomResult.node(results, next), nil)
            } catch (let error) {
                return completion(GenericCustomResult.node(nil, nil), error)
            }
        }
    }
    dataTask.resume()
}

Same function, using our new, custom, result type. Not much of a difference, is there?

Alamofire

Alamofire is an open-source networking library for iOS, written in Swift, that takes much of the pain away compared to URLSession. See for yourself

func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void)  {
    guard let url = URL(string: "https://yourdomain.com/yourRoute") else {
        completion(nil, someError)
        return
    }
    Alamofire.request(url, method: .get, parameters: ["searchTerm": searchTerm, "page": page])
        .validate()
        .responseDecodable(of: [RandomDataStructure].self) { response in
            switch response.result {
            case .success(let value):
                let next = { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
                    alamofireGetSearchResults(searchTerm: searchTerm, page: page + 1, completion: completion)
                }
                completion(GenericCustomResult.node(value, next), nil)
            case .failure(let error):
                completion(nil, error)
            }
            return
    }
}

damn, this is much shorter than the URLSession way. Alamofire is awesome - Initial Code

func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void)  {
    guard let url = URL(string: "https://yourdomain.com/yourRoute") else {
        completion(nil, someError)
        return
    }
    Alamofire.request(url, method: .get, parameters: ["searchTerm": searchTerm, "page": page])
        .validate()
        .responseDecodable(of: [RandomDataStructure].self) { response in
            switch response.result {
            case .success(let value):
                let next = { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
                    alamofireGetSearchResults(searchTerm: searchTerm, page: page + 1, completion: completion)
                }
                completion(GenericCustomResult.node(value, next), nil)
            case .failure(let error):
                completion(nil, error)
            }
            return
    }
}

updated with our new type. Still much, much shorter

Algolia

Algolia is a Search SaaS, with an excellent documentation, impressive performances and great capabilities, that takes care of most, if not all, of the heavy-lifting involved in implementing a search feature for your app (mobile or web).

func getSearchResults(searchTerm: String, page: Int = 0, perPage: Int = 20, completion: @escaping ([RandomDataStructure]?, Error?) -> Void)  {
    
    // Initialize the client. Generally injected into or initialized in your class instead of the function itself
    let client: Client = Client(appID: "yourAlgoliaAppId", apiKey: "yourAlgoliaApiKey")
    let index = self.client.index(withName: "YourIndex")
    
    // Build your query, using the actual query string, the number of items per page and the page number
    let query = Query()
    query.query = searchTerm
    query.hitsPerPage = perPage
    query.page = page
    
    index.search(query, completionHandler: { (content, error) -> Void in
        
        // Check if you actually got any content or if there's an error
        guard let content = content && error == nil else {
            completion(nil, error)
            return
        }
        
        // More checking
        guard let hits = content["hits"] as? [Any], let page = content["page"] as? UInt, let nbPages = content["nbPages"] as? UInt else {
            completion(nil, someCustomError)
            return
        }
        
        // Check if the are any actual results for the query
        if hits.count == 0 {
            completion(nil, someCustomError or nil)
            return
        }
        
        // Map the results
        let results = hits.compactMap({ (item: Any) -> [RandomDataStructure]? in
            do {
                return try JSONDecoder().decode([RandomDataStructure].self, withJSONObject: item)
            } catch let error {
                return nil
            }
        })
        
        // Return it all !
        completion(results, nil)
        return
    })
}

Everybody loves Algolia. Also, this a great opportunity to show how we handle the last page case

func getSearchResults(searchTerm: String, page: Int = 0, perPage: Int = 20, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void)  {
    
    // Initialize the client. Generally injected into or initialized in your class instead of the function itself
    let client: Client = Client(appID: "yourAlgoliaAppId", apiKey: "yourAlgoliaApiKey")
    let index = self.client.index(withName: "YourIndex")
    
    // Build your query, using the actual query string, the number of items per page and the page number
    let query = Query()
    query.query = searchTerm
    query.hitsPerPage = perPage
    query.page = page
    
    index.search(query, completionHandler: { (content, error) -> Void in
        
        // Check if you actually got any content or if there's an error
        guard let content = content && error == nil else {
            completion(nil, error)
            return
        }
        
        // More checking
        guard let hits = content["hits"] as? [Any], let page = content["page"] as? UInt, let nbPages = content["nbPages"] as? UInt else {
            completion(nil, someCustomError)
            return
        }
        
        // Check if the are any actual results for the query
        if hits.count == 0 {
            completion(GenericCustomResult.node(nil, nil), someCustomError or nil)
            return
        }
        
        // Map the results
        let results = hits.compactMap({ (item: Any) -> [RandomDataStructure]? in
            do {
                return try JSONDecoder().decode([RandomDataStructure].self, withJSONObject: item)
            } catch let error {
                return nil
            }
        })
        
        // Create the function that will return the next page, if there is a next page to fetch
        let next = page >= nbPages - 1
            ? nil
            : { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
                getSearchResults(searchTerm: searchTerm, page: page + 1, perPage: perPage, completion: completion)
            }
        
        // Return it all!
        completion(GenericCustomResult.node(results, next), nil)
        return
    })
}

Tadaaa!

Firestore

Last one, Firestore. It uses an interesting pagination pattern: cursors. Basically, you tell it at which or after which item matching your item it should start looking.

func firestoreGetSearchResults(searchTerm: String, limit: Int = 10, startAfter document: DocumentSnapshot? = nil, completion: @escaping ([RandomDataStructure]?, lastSnapshot: DocumentSnapshot?, Error?) -> Void) {

    // Define the collection we'll query
    let collection = db.collection("cities")
    
    // Build our query using the query cursor and limit
    let query = startAfter == nil ? collection.order(by: "populations").limit(to: limit) : collection.order(by: "populations").limit(to: limit).start(after: startAfter)
    query.getDocuments() { (querySnapshot, err) in
    
    	// Check wether or not the query returned anything, or failed
        guard let documents = querySnapshot.documents && err == nil else {
            completion(nil, nil, err)
            return
        }
        
        // Get the last item in our results, as it will be our next cursor
        let lastSnapshot = querySnapshot.documents.last
        let results = ... // map your firestore dictionary to your own data structure

	// Return everything!
	completion(results, lastSnapshot, nil)
	return
    }
}

And here comes the query cursor

func getSearchResults(searchTerm: String, limit: Int = 10, startAfter document: DocumentSnapshot? = nil, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
	
    // Define the collection we'll query
    let collection = db.collection("cities")
    
    // Build our query using the query cursor and limit
    let query = startAfter == nil ? collection.order(by: "populations").limit(to: limit) : collection.order(by: "populations").limit(to: limit).start(after: startAfter)
    query.getDocuments() { (querySnapshot, err) in
    
    	// Check wether or not the query returned anything, or failed
        guard let documents = querySnapshot.documents && err == nil else {
            completion(nil, err)
            return
        }
        
        // Get the last item in our results, as it will be our next cursor
        let lastSnapshot = querySnapshot.documents.last
        let results = ... // map your firestore dictionary to your own data structure
        
        // Create the function that will get the next 'page' if there is any next 'page' to fetch
        let next = querySnapshot.documents.length < limit
            ? nil
            : { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
                firestoreGetSearchResults(searchTerm: searchTerm, limit: limit, startAfter: lastSnapshot, completion: completion)
            }
        
        // Return everything!
        completion(GenericCustomResult.node(results, next), nil)
        return
    }
}

And it's gone. Also, see how we used the query limit to figure out wether or not there is a next page?

Using the returned value

Ok. Now, what do we do with it all? How do we actual use all of it? That's the whole point, right? Simplifying our API consumption, and making sure the rest of our app doesn't have to bother with pagination.

Say you are using an MVC, architecture. Your (stripped down and much simplified) ViewController would look like this

class ViewController: UIViewController {
    var items: [RandomDataStructure] = []
    var next: ((@escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) -> ())? = nil
    
    func newItemsHandling(result: GenericCustomResult<[RandomDataStructure]>, error: Error?) {
        guard let newItems = result.value, let next = result.next else {
            return
        }
        self.next = next
        self.items = self.items + newItems
    }
    
    func setup() {
        getSearchResults(searchTerm: "fizz") { [weak self] result, error in
            self?.newItemsHandling(result: result, error: error)
        }
    }
    
    func onNext() {
        self.next?() { [weak self] result, error in
            self?.newItemsHandling(result: result, error: error)
        }
    }
}

Pretty sweet, right?

Have an MVVM architecture? Don't fret; we've got you covered!

class ViewModel {
    var items: [RandomDataStructure] = []
    var next: ((@escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) -> ())? = nil
    
    func newItemsHandling(result: GenericCustomResult<[RandomDataStructure]>, error: Error?) {
        guard let newItems = result.value, let next = result.next else {
            return
        }
        self.next = next
        self.items = self.items + newItems
    }
    
    func setup() {
        getSearchResults(searchTerm: "fizz") { [weak self] result, error in
            self?.newItemsHandling(result: result, error: error)
        }
    }
    
    func onNext() {
        self.next?() { [weak self] result, error in
            self?.newItemsHandling(result: result, error: error)
        }
    }
}

Yes; it's the same thing

Using firestore and have to handle query cursors? It's the exact same thing!

class ViewModel {
    var items: [RandomDataStructure] = []
    var next: ((@escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) -> ())? = nil
    
    func newItemsHandling(result: GenericCustomResult<[RandomDataStructure]>, error: Error?) {
        guard let newItems = result.value, let next = result.next else {
            return
        }
        self.next = next
        self.items = self.items + newItems
    }
    
    func setup() {
        getSearchResults(searchTerm: "fizz") { [weak self] result, error in
            self?.newItemsHandling(result: result, error: error)
        }
    }
    
    func onNext() {
        self.next?() { [weak self] result, error in
            self?.newItemsHandling(result: result, error: error)
        }
    }
}

Yes. The exact same thing. Again. Isn't it beautiful?

Epilogue

Yes, that's it. That really is all there is to it.

If you liked this, or it helped you out, you'll be pleased to know there will soon be a second part to this current post, exploring how we can further improve our enum, using the Iterator pattern, Swift 5's new Result type and reactive programming! You'll probably also enjoy my previous post on cell selection handling in Swift.

Always handle your table & collection view cell selection properly, using functions

When writing iOS, you will always end up using UITableView or UICollectionView. With them come cells, delegates, datasources, cellForRow(at indexPath:), didSelect,... I’ve often seen it become (and inherited) an ugly mess. Not anymore.

The (not so) Weekly FitzThomas Léger

it too involves a very light touch of functional programming

Finally, this post is directly inspired from the issue that lead me to asking this question on StackOverflow a while ago.

How to make a type alias that references itself?

I made a few gateways / providers to integrate with an API in my API, using RX Swift and I’m trying to handle the pagination in what seems to me like a clean and simple way. Basically, the function

Stack OverflowElFitz

If you want to get into blogging, or have a hard time keeping your existing blog afloat, I wanted to let know you that that challenges you've encountered, had a hard time finding solutions to, and ultimately solved, probably are some of the best source materials!

Thank you for reading this far, and have a great day!