Swiftly deal with pagination in Swift
Not satisfied with how you're handling pagination in your Swift iOS app? Pretty sure there has to be a better way to paginate your UICollectionView or UITableView? So did I, and here's how I did it!
When developing apps, we almost always end up having to handle some sort of pagination or another.
Often, it involves keeping some page number around. Sometimes, it requires some logic involving the current page number and the total number of pages (ie most REST APIs, including Algolia's). Occasionaly, you may end up handling some cursor and passing it around (Firestore and DynamoDB come to mind). And I'm pretty sure there are many other ways this is done out there.
When you only have to deal with one or the other, it's a straightforward no-brainer. But every once in a while, your rare stamps marketplace app ends up having to display both simple ordered lists and search results, using both Firestore and Algolia as datasources, to display the exact same thing, using very different pagination behaviors.
Or perhaps you'll be alternating between a local source (such as CoreData or Realm) and a remote source, depending on the connection state. Or maybe, like it happened to me last March with Spotr, you'll have to switch backends, from a service that is shutting down in two weeks (Stamplay) to another that may be shut down at any given time without any warning (Firebase is, after all, owned by Google and part of GCP).
In any case, you shouldn't recode your collections for each separate datasource. Yes, you could handle this is in your ViewModels or Presenter and expose it transparently to your views. But is it their problem to begin with?
Not for me. My ViewModels' work is to decide when to get the next batch of results, eventually transform the data for my view, and pass it along. That's it. Not figure out whether my current page number is less than or equal to the total number of pages - 1, or whether or not last call returned the exact number of items I asked for. And certainly not to try and keep track of different pagination values that may or may not be optional.
We shouldn't have to spread multiple pagination patterns handling all over our app. It should be contained. It should be DRY. It should offer a unique, consistent API whatever the source is. And it should be totally invisible to the rest of our code.
Overview
How do we do that? First, we define a custom 'Result Type', with two properties: the fetched result, wether it's a single item or a list, and some optional function returning the next batch of results. Next, we make it so that every single one of our API calls wrap their return value with our new 'Result Type'.
Ideally, this would look like this
typealias CustomResultType = (value: ItemType, next: (() -> CustomResultType)?)
private func somePrivatePaginatedApiFunction(param: ParamType, page: Int = 0) -> ItemType {
... your code here
return (result: result, currentPage: page, totalPages: totalPages)
}
public func someExposedApiFunction = (param: ParamType) -> CustomResultType {
const result = somePrivatePaginatedApiFunction(param, 0)
const next = result.currentPage == result.totalPages - 1 ? nil : () -> somePrivatePaginatedApiFunction(params, result.currentPage + 1)
return (value: result.result, next: next)
}That's the main idea. Sadly, it just doesn't work this way. If you try it, XCode will just scream at you, saying Type alias 'CustomResultType' references itself, because our type alias does reference itself. Also, our type alias's hardcoded 'ItemType' definitely isn't anywhere near universal. Let's overcome these issues.
How? Generics and, as you might have already noticed, a healthy dose of functional programming. For those of you who are unfamiliar with those and want to know more about them, keep reading. Everyone else, skip to the implementation.
The tools
Generics
Aside from being intrinsically awesomely awesome, generic types are the go-to tool whenever you want to make a type... uh... well, generic.
Kidding aside, generics are what you turn to when you want to make classes, methods, structures or functions that can take different type of parameters. Such as a function that can swap two items' positions in an array, no matter the array's type. Some function that could add either Ints, Doubles or CGFloats, depending on what you give it at runtime. Or, say... a result enum who's values can be of any type, without losing said type! (using that dreadful Any would lose the values' types).
If you're interested, or you'd like to get a better grasp of the concept, go have a look at the official Swift documentation, and this nice Raywenderlich tutorial. They should make it crystal clear.
Functional Programming
The other awesome tool we'll be using is the functional programming paradigm. Well, a very small subset of it; nothing really fancy: the abilities to return a function, and to pass functions as parameters. That's possible because functions in Swift are said to be first class (see Sundell's great article on first class functions), meaning we can use them just like any other values.
If you've ever used map, flatMap, filter or reduce, then you've already encountered functional programming, because these functions take your mapping, filtering or reducing functions as arguments, making them what we call higher order functions (here's a nice post on that).
Also, a very nice introduction tutorial for functional programming in Swift, by Ray Wenderlich. These guys make lots of tutorials.
So, basically, we'll be making a higher order function that returns a function, which will return both our next page and, if applicable the function returning the page after that, as well as... Yes, this is recursive. And awesome!
Implementation
The generic, custom, Result type
The workaround for the self-referencing typealias is to use an enum (shoutout to Nibr), with the current result(s) and the function to get the next batch of results as optional associated values.
enum CustomResult {
indirect case node(ItemType?, (() -> CustomResult)?)
}AND. IT. BUILDS! Nice! Next, make it generic!
enum GenericCustomResult<T> {
indirect case node(T?, (() -> GenericCustomResult<T>)?)
}Great! We have our generic result type. Now, we simplify it's usage. Accessing an enum's cases' associated values is a bit of a pain, we'll be using this specific one all over the place, and since there's only one case, we don't want the rest of our app to have to deal with it each and every time we use it.
enum GenericCustomResult<T> {
indirect case node(T?, (() -> GenericCustomResult<T>)?)
var value: T? {
if case let GenericCustomResult.node(value, _) = self {
return value
}
return nil
}
var next: (() -> GenericCustomResult<T>)? {
if case let GenericCustomResult.node(_, next) = self {
return next
}
return nil
}
}Finally, the completion block. Most if not all of these calls will be asynchronous, so our 'next' needs to take a completion function, in order to return the results.
enum GenericCustomResult<T> {
indirect case node(T?, ((@escaping (GenericCustomResult<T>, Error?) -> Void) -> ())?)
var value: T? {
if case let GenericCustomResult.node(value, _) = self {
return value
}
return nil
}
var next: ((@escaping (GenericCustomResult<T>, Error?) -> Void) -> ())? {
if case let GenericCustomResult.node(_, next) = self {
return next
}
return nil
}
}The returning method
We have a return type. What would an API call returning this new type look like? It depends, based on the API your app consummes (wether it uses pages or cursors; wether it does or doesn't return the total number of pages,...), and the tools you use to do so. But first, let's define our data's model.
struct RandomDataStructure: Codable {
let firstProperty: Int
let secondProperty: String
}URLSession
URLSession is the original, native, iOS framework for networking.
func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping ([RandomDataStructure]?, Error?) -> Void) {
let session = URLSession(configuration: .default)
guard var urlComponents = URLComponents(string: "https://yourdomain.com/yourRoute") else {
completion(nil, someError)
return
}
urlComponents.query = "term=\(searchTerm)&page=\(page)"
guard let url = urlComponents.url else {
return
}
let dataTask = session.dataTask(with: url) { data, response, error in
if let error = error {
return completion(nil, error)
} else if let data = data,
let response = response as? HTTPURLResponse,
response.statusCode == 200 {
do {
let results = try JSONDecoder().decode([RandomDataStructure].self, from: data)
return completion(results, nil)
} catch (let error) {
return completion(nil, error)
}
}
}
dataTask.resume()
}func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
let session = URLSession(configuration: .default)
guard var urlComponents = URLComponents(string: "https://yourdomain.com/yourRoute") else {
completion(nil, someError)
return
}
urlComponents.query = "term=\(searchTerm)&page=\(page)"
guard let url = urlComponents.url else {
completion(nil, someOtherError)
return
}
let dataTask = session.dataTask(with: url) { data, response, error in
if let error = error {
return completion(GenericCustomResult.node(nil, nil), error)
} else if let data = data,
let response = response as? HTTPURLResponse,
response.statusCode == 200 {
do {
let results = try JSONDecoder().decode([RandomDataStructure].self, from: data)
let next = { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
getSearchResults(searchTerm: searchTerm, page: page + 1, completion: completion)
}
return completion(GenericCustomResult.node(results, next), nil)
} catch (let error) {
return completion(GenericCustomResult.node(nil, nil), error)
}
}
}
dataTask.resume()
}Alamofire
Alamofire is an open-source networking library for iOS, written in Swift, that takes much of the pain away compared to URLSession. See for yourself
func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
guard let url = URL(string: "https://yourdomain.com/yourRoute") else {
completion(nil, someError)
return
}
Alamofire.request(url, method: .get, parameters: ["searchTerm": searchTerm, "page": page])
.validate()
.responseDecodable(of: [RandomDataStructure].self) { response in
switch response.result {
case .success(let value):
let next = { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
alamofireGetSearchResults(searchTerm: searchTerm, page: page + 1, completion: completion)
}
completion(GenericCustomResult.node(value, next), nil)
case .failure(let error):
completion(nil, error)
}
return
}
}func getSearchResults(searchTerm: String, page: Int = 0, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
guard let url = URL(string: "https://yourdomain.com/yourRoute") else {
completion(nil, someError)
return
}
Alamofire.request(url, method: .get, parameters: ["searchTerm": searchTerm, "page": page])
.validate()
.responseDecodable(of: [RandomDataStructure].self) { response in
switch response.result {
case .success(let value):
let next = { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
alamofireGetSearchResults(searchTerm: searchTerm, page: page + 1, completion: completion)
}
completion(GenericCustomResult.node(value, next), nil)
case .failure(let error):
completion(nil, error)
}
return
}
}Algolia
Algolia is a Search SaaS, with an excellent documentation, impressive performances and great capabilities, that takes care of most, if not all, of the heavy-lifting involved in implementing a search feature for your app (mobile or web).
func getSearchResults(searchTerm: String, page: Int = 0, perPage: Int = 20, completion: @escaping ([RandomDataStructure]?, Error?) -> Void) {
// Initialize the client. Generally injected into or initialized in your class instead of the function itself
let client: Client = Client(appID: "yourAlgoliaAppId", apiKey: "yourAlgoliaApiKey")
let index = self.client.index(withName: "YourIndex")
// Build your query, using the actual query string, the number of items per page and the page number
let query = Query()
query.query = searchTerm
query.hitsPerPage = perPage
query.page = page
index.search(query, completionHandler: { (content, error) -> Void in
// Check if you actually got any content or if there's an error
guard let content = content && error == nil else {
completion(nil, error)
return
}
// More checking
guard let hits = content["hits"] as? [Any], let page = content["page"] as? UInt, let nbPages = content["nbPages"] as? UInt else {
completion(nil, someCustomError)
return
}
// Check if the are any actual results for the query
if hits.count == 0 {
completion(nil, someCustomError or nil)
return
}
// Map the results
let results = hits.compactMap({ (item: Any) -> [RandomDataStructure]? in
do {
return try JSONDecoder().decode([RandomDataStructure].self, withJSONObject: item)
} catch let error {
return nil
}
})
// Return it all !
completion(results, nil)
return
})
}func getSearchResults(searchTerm: String, page: Int = 0, perPage: Int = 20, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
// Initialize the client. Generally injected into or initialized in your class instead of the function itself
let client: Client = Client(appID: "yourAlgoliaAppId", apiKey: "yourAlgoliaApiKey")
let index = self.client.index(withName: "YourIndex")
// Build your query, using the actual query string, the number of items per page and the page number
let query = Query()
query.query = searchTerm
query.hitsPerPage = perPage
query.page = page
index.search(query, completionHandler: { (content, error) -> Void in
// Check if you actually got any content or if there's an error
guard let content = content && error == nil else {
completion(nil, error)
return
}
// More checking
guard let hits = content["hits"] as? [Any], let page = content["page"] as? UInt, let nbPages = content["nbPages"] as? UInt else {
completion(nil, someCustomError)
return
}
// Check if the are any actual results for the query
if hits.count == 0 {
completion(GenericCustomResult.node(nil, nil), someCustomError or nil)
return
}
// Map the results
let results = hits.compactMap({ (item: Any) -> [RandomDataStructure]? in
do {
return try JSONDecoder().decode([RandomDataStructure].self, withJSONObject: item)
} catch let error {
return nil
}
})
// Create the function that will return the next page, if there is a next page to fetch
let next = page >= nbPages - 1
? nil
: { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
getSearchResults(searchTerm: searchTerm, page: page + 1, perPage: perPage, completion: completion)
}
// Return it all!
completion(GenericCustomResult.node(results, next), nil)
return
})
}Firestore
Last one, Firestore. It uses an interesting pagination pattern: cursors. Basically, you tell it at which or after which item matching your item it should start looking.
func firestoreGetSearchResults(searchTerm: String, limit: Int = 10, startAfter document: DocumentSnapshot? = nil, completion: @escaping ([RandomDataStructure]?, lastSnapshot: DocumentSnapshot?, Error?) -> Void) {
// Define the collection we'll query
let collection = db.collection("cities")
// Build our query using the query cursor and limit
let query = startAfter == nil ? collection.order(by: "populations").limit(to: limit) : collection.order(by: "populations").limit(to: limit).start(after: startAfter)
query.getDocuments() { (querySnapshot, err) in
// Check wether or not the query returned anything, or failed
guard let documents = querySnapshot.documents && err == nil else {
completion(nil, nil, err)
return
}
// Get the last item in our results, as it will be our next cursor
let lastSnapshot = querySnapshot.documents.last
let results = ... // map your firestore dictionary to your own data structure
// Return everything!
completion(results, lastSnapshot, nil)
return
}
}func getSearchResults(searchTerm: String, limit: Int = 10, startAfter document: DocumentSnapshot? = nil, completion: @escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) {
// Define the collection we'll query
let collection = db.collection("cities")
// Build our query using the query cursor and limit
let query = startAfter == nil ? collection.order(by: "populations").limit(to: limit) : collection.order(by: "populations").limit(to: limit).start(after: startAfter)
query.getDocuments() { (querySnapshot, err) in
// Check wether or not the query returned anything, or failed
guard let documents = querySnapshot.documents && err == nil else {
completion(nil, err)
return
}
// Get the last item in our results, as it will be our next cursor
let lastSnapshot = querySnapshot.documents.last
let results = ... // map your firestore dictionary to your own data structure
// Create the function that will get the next 'page' if there is any next 'page' to fetch
let next = querySnapshot.documents.length < limit
? nil
: { (completion: @escaping(GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) in
firestoreGetSearchResults(searchTerm: searchTerm, limit: limit, startAfter: lastSnapshot, completion: completion)
}
// Return everything!
completion(GenericCustomResult.node(results, next), nil)
return
}
}Using the returned value
Ok. Now, what do we do with it all? How do we actual use all of it? That's the whole point, right? Simplifying our API consumption, and making sure the rest of our app doesn't have to bother with pagination.
Say you are using an MVC, architecture. Your (stripped down and much simplified) ViewController would look like this
class ViewController: UIViewController {
var items: [RandomDataStructure] = []
var next: ((@escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) -> ())? = nil
func newItemsHandling(result: GenericCustomResult<[RandomDataStructure]>, error: Error?) {
guard let newItems = result.value, let next = result.next else {
return
}
self.next = next
self.items = self.items + newItems
}
func setup() {
getSearchResults(searchTerm: "fizz") { [weak self] result, error in
self?.newItemsHandling(result: result, error: error)
}
}
func onNext() {
self.next?() { [weak self] result, error in
self?.newItemsHandling(result: result, error: error)
}
}
}
Have an MVVM architecture? Don't fret; we've got you covered!
class ViewModel {
var items: [RandomDataStructure] = []
var next: ((@escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) -> ())? = nil
func newItemsHandling(result: GenericCustomResult<[RandomDataStructure]>, error: Error?) {
guard let newItems = result.value, let next = result.next else {
return
}
self.next = next
self.items = self.items + newItems
}
func setup() {
getSearchResults(searchTerm: "fizz") { [weak self] result, error in
self?.newItemsHandling(result: result, error: error)
}
}
func onNext() {
self.next?() { [weak self] result, error in
self?.newItemsHandling(result: result, error: error)
}
}
}Using firestore and have to handle query cursors? It's the exact same thing!
class ViewModel {
var items: [RandomDataStructure] = []
var next: ((@escaping (GenericCustomResult<[RandomDataStructure]>, Error?) -> Void) -> ())? = nil
func newItemsHandling(result: GenericCustomResult<[RandomDataStructure]>, error: Error?) {
guard let newItems = result.value, let next = result.next else {
return
}
self.next = next
self.items = self.items + newItems
}
func setup() {
getSearchResults(searchTerm: "fizz") { [weak self] result, error in
self?.newItemsHandling(result: result, error: error)
}
}
func onNext() {
self.next?() { [weak self] result, error in
self?.newItemsHandling(result: result, error: error)
}
}
}Epilogue
Yes, that's it. That really is all there is to it.
If you liked this, or it helped you out, you'll be pleased to know there will soon be a second part to this current post, exploring how we can further improve our enum, using the Iterator pattern, Swift 5's new Result type and reactive programming! You'll probably also enjoy my previous post on cell selection handling in Swift.
Finally, this post is directly inspired from the issue that lead me to asking this question on StackOverflow a while ago.
ElFitz
If you want to get into blogging, or have a hard time keeping your existing blog afloat, I wanted to let know you that that challenges you've encountered, had a hard time finding solutions to, and ultimately solved, probably are some of the best source materials!
Thank you for reading this far, and have a great day!
Comments ()