Here is what SwiftData will be (…and what it could have been)
Twitter: @drewmccormack
WWDC is just around the corner, and that means it is time to get in your predictions. For a few years now, there has been discussion of a Swift native data modeling framework, which will do for data what SwiftUI is doing for the user interface. This might be the year. The timing feels right, with SwiftUI reaching a usable level. SwiftData may finally make its appearance. But what will SwiftData look like?
Before I answer that, I should state first that I have no inside information at all. I don’t know if SwiftData even exists. But I do have a long history with data on Apple’s devices. I’ve written books and articles dealing with Core Data, and develop the only third party commercial Core Data synchronization framework currently available. I use Core Data daily, and know it well. And lately, I have started to think about what it would mean for Apple’s data frameworks to follow the same route as SwiftUI, and — perhaps more interestingly — what some blue sky thinking could bring further into the future.
Core Data is not going anywhere
If you’re hoping for something completely new with SwiftData, I don’t think you are going to get it. Apple have invested many years into Core Data, and have a strong team working on it. The framework today is very stable, and much more powerful than most developers realize. You don’t really understand just how powerful Core Data is, until you start developing your own model layer from scratch. Then you start to hit the problems:
- How do I make sure that there is only one copy of a particular object in memory?
- How do I access enormous datasets without pulling everything into memory?
- How do I make transactional updates to a set of objects?
- How do I keep the data in memory up-to-date with what is on disk?
- How can I make long running updates in the background, while my UI continues to use the same data store?
- How do I merge in changes from other devices?
You may not realize it when you first start using Core Data, but you get all of this for free, and lots, lots more.
So Apple is not going to throw out the baby with the bathwater. Expect Core Data to be at the core of this thing.
Learning from SwiftUI
If Core Data is not going anywhere, what place is there for SwiftData? Well, the same place as for SwiftUI.
Many developers think of SwiftUI as something completely new, but it is actually built atop the work of Apple UI framework engineers going back to the days of NeXT and the Jobs’ takeover. SwiftUI is not as radically different as it first seems. What Apple did is replace the declarative XIB files, which contain a tree structure of XML describing the UI and how it is connected, with Swift source code…containing a tree structure describing the UI and how it is connected. This is oversimplifying a bit, but the shift is fundamentally from a data declaration of the UI to a code-al declaration of the UI.
If you thought that the list you see on screen in a SwiftUI app is something completely different to what you see in a UIKit app, think again. It’s the same control, with the same autolayout constraining it on the screen. The scrolling works the same, the buttons are the same, the text labels are the same — it is exactly the same. What is different is how the developer told the system where it wanted those controls, and how they should behave. In SwiftUI, you declare these things in code, where previously you might have used an XIB or Storyboard file.
SwiftData at WWDC2021
This is all well and good, but wasn’t this post about SwiftData? How did we get onto SwiftUI?
SwiftData will be designed in the same way. Apple want to leverage years of Core Data development, so the data storage will still use the existing framework, including mirroring via CloudKit. What will change is how you declare your model, and how you fetch and manipulate data in your app.
To begin with declaring the model, currently in Core Data you use a proprietary XML file via the data modeler in Xcode to declare your data model. Sound familiar? It should, because the data model file is analogous to an XIB file. We used to edit the XIB using Interface Builder (later Xcode), and our app would load the XML and layout the interface. When you use Core Data now, you edit your model using a the Core Data Modeler, a tool similar to Interface Builder, which spits out an XML file, and that is what your app loads to setup the data store.
Once you see this, it becomes much clearer where SwiftData will be positioned. You will be able to declare your data model entirely in code, using the same techniques you use in SwiftUI. The code will build up a tree of value types defining the entities and properties in the model, and even the migrations between model versions. This can all be done on top of Core Data, without requiring major changes to the framework.
The other piece of the puzzle is fetching and manipulating data from the store. My guess is that it will be possible to do this entirely with value types (structs). Presumably bindings will be available to integrate your data directly into SwiftUI, so that changes made to the store will propagate to the view, and changes made to the data values will be updated in Core Data.
Again, this sounds like a big change, but I don’t think you will have to dig very deep to find the old Core Data. You may be provided with struct to work with in your app, but internally, Apple will probably bridge that struct to an old-fashioned Core Data NSManagedObject. The bindings will just be syntactic sugar on top of Key-Value Observation and Key-Value Coding, and the NSManagedObjectContext will be just under the surface, keeping track of everything so it knows what you have changed when it comes time to save. Just like SwiftUI, it’s a shiny new Ferrari body on a tried and true Toyota Corolla chassis.
What it could have been
It sounds good, right? SwiftData will make working with your data model nicely integrate with the new SwiftUI framework you are using to layout your UI, and should be as robust as the latest releases of Core Data. It’s the most logical step for Apple to take, and that is why I think we will see it, if not this year, probably next.
And yet, it leaves me wondering what could have been, because the world of data is in serious flux. Ever heard of blockchains? If not, get out from under that rock and take a look. All the hype of Bitcoin and cryptocurrencies aside, there are some interesting developments afoot. Data storage is going global — its becoming decentralized — and the Toyota Corolla of data modeling frameworks is starting to show its age.
I’m not suggesting for a minute that Apple adopt the Ethereum blockchain for Core Data 2.0, but there is plenty they could be doing to make their storage APIs more decentralized. And what’s more, it doesn’t have to be an exercise in altruism. Decentralized storage fits Apple’s narrative beautifully. Here’s why…
Decentralized stores will rule the earth
Remember when you got your first Mac or iPhone? If you have been around a while, you will probably remember that it was a fairly isolated device. My first Mac didn’t connect to anything, not even the web. I could write documents, but they stayed right on the hard disk (or floppy, if you are really old!). There was no watch or phone to exchange data with. In the rare case that an app could work with a peripheral, such as an iPod, it required a direct USB connection to sync up the data each day.
It was into this world that Core Data was born. A world in which everything is centralized on one device. There was no need to handle data synchronization or merge conflicting changes, because there weren’t any. And the Core Data of today is fundamentally the same as it was then: a centralized data storage framework for a single device.
Yes, now it can upload your data to CloudKit, but it won’t automatically deduplicate objects when you download the changes to another device. Why? Because it can’t. It has been designed for a single device. It has no concept of globally distributed objects. It can’t see that a Contact object on your phone is just another version of a Contact object on your Mac, separated by time and space. It sees those contacts as unrelated. Chalk and cheese.
What we need now is for someone to take decentralized storage seriously, and I don’t mean Bitcoin speculators. Blockchain and technologies like Conflict-Free Replicated Data Types (CRDTs) have shown how it can be done. In the server world, most services are already decentralized. They have to be; there is no way a single server could handle the load of a Twitter or Facebook.
Apple just needs to gather a team together, beginning with members of the Core Data team, but also including CloudKit engineers and APFS engineers. All of these people have faced — and solved — the problems of distributed data systems. Together they should design the first decentralized store for apps.
Why are we still waiting?
It’s a good question. Given the technologies needed to make a decentralized store are not really that new, why haven’t we seen someone make one yet? Why is there no easy to use decentralized store for app developers?
One reason is that it doesn’t align with the interests of many of the big players. Companies like Google and Facebook are built almost entirely on advertising, and to make their advertising effective, they need your data. The last thing in the world they are going to promote is a decentralized store, in which all the data is end-to-end encrypted, and it can only be read on the user’s own devices. They want to own the data — they have to own the data. Without it, they don’t have a business. So Google is not going to build our decentralized data store, and neither is Facebook — don’t hold your breath waiting for ReactDecentral.
But there is one player who has a different business model, one that doesn’t rely on mining user data. It’s a player who likes to talk about customer privacy, actively working to protect users from tracking, and uploading of personal data like contacts.
Apple have long touted their stance on privacy. Even if you don’t buy into their motivations, it is difficult to deny that improving privacy works in Apple’s favor from a public relations point of view. It would make sense for Apple to develop a decentralized data storage framework with full end-to-end encryption for developers to use in their apps, so that users are no longer mandated to share data with a central service.
WWDC2023
This is the potential. This is what SwiftData could have been. It’s almost certainly not what it will be, but perhaps it can be the next step. Gather the top engineers from the Core Data, CloudKit, and APFS teams, and start now, Apple. And at WWDC2023 we can all be astonished at how easy it now is to develop a cross device app, with complete user privacy guaranteed. We can use the framework to more easily develop apps that run on all of Apple’s devices, and Apple can sell it to the public as a great step forward in taking control of their personal data. Win, win.
