What's new in PDFKit

Conrad: I'm Conrad Carlen, and today, I'll be talking about what's new in PDFKit. Here's our agenda. First a quick review of PDFKit, and then, look at what's new, including Live text and forms, a new way to make PDF pages from images, and, finally, overlay views. Let's start with a quick refresher on how PDFKit works.

PDFKit is a full-featured framework that helps your app view, edit, and write PDF files. It's available on iOS, macOS, and Mac Catalyst, and it can also be used in SwiftUI by way of UIViewRepresentable, a wrapper that lets you integrate UI views into your app. PDFKit consists of 4 core classes that cover most of the functionality you'll need in your app. PDFView is the widget that you include in your layout using SwiftUI or Interface Builder. It displays the content of a PDF document and lets people navigate, set zoom level, and copy text to the Pasteboard.

PDFDocument represents a PDF file. It's not common to subclass PDFDocument, but you will always use one. It's the root of the PDF object graph, or the trunk of the tree. You can't have a tree without one of these.

Each document contains one or more PDFPage objects. Pages render content and store resources like fonts and images that are unique to that page.

At the leaves of our object graph are PDFAnnotations. These are optional. Whereas the content of a PDFPage is not intended to be edited, annotations are interactive by nature and often editable.

Each of these objects will play a role in what I'll cover today. To learn more about the fundamentals of PDFKit, check out the great presentation "Introducing PDFKit" in the link below.

Now, let's talk about new features introduced in iOS 16 and macOS Ventura.

PDFKit now supports Live Text. It's different than in Photos, where the text is often a small amount that you can tap to copy. Unlike with photos, with a PDF, if you see text, it generally is text, and people expect it to behave as such without doing anything special. Now, with Live Text, you can select and search text in a PDF document like this one. It's just a scanned bitmap, with no text at all.

Of course, PDFs can have many pages. You wouldn't want to grind through OCR on all the pages of a PDF document when you open it, so PDFKit does it on demand, as you interact with each page, if you interact with it. OCR is done in place, so there's no need to make a copy of the document.

And, if you choose to save text for the whole document, there's an option to do so when saving.

In addition to live text, PDFKit has improved form handling. Documents that contain form fields are automatically recognized, even if they don't contain built-in text fields. You can tab through these text fields and enter text, just like you would expect.

Next, let's talk about a new API for creating PDF pages from images.

In iOS 16 and macOS Ventura, there's a new, flexible API that lets your app create PDF pages using images as inputs.

Your app supplies images using CGImageRef. PDFKit takes the CGImageRef that you provide and compresses it using high-quality JPEG encoding.

Because CGImageRef is a native data type in CoreGraphics, additional conversions are unnecessary.

There are several options that help you handle the most common cases.

MediaBox specifies the size of the page. You can choose to fit the image exactly, or choose a paper size, like Letter.

Rotation lets you specify portrait or landscape orientation.

UpscaleIfSmaller. By default, if the image is larger than the MediaBox, the image is downscaled to fit. If UpscaleIfSmaller is specified, that still applies, but, if the image is small, it will be upscaled to fill the page.

And now, to answer a question that many of you have asked-- "How can I draw on PDF pages using PencilKit?"-- the answer is to use an overlay view.

In the past, the only way to do additional drawing on PDFs was to subclass PDFPage and override the drawing method, or by using custom PDF annotations. But starting in iOS 16 and macOS Ventura, it's now possible to overlay your own view on top of each PDF page. This allows your app to create live, fully interactive views that appear on top of PDF pages.

Here are the 3 things you need to know about overlaying views.

First, you'll use a new protocol to install your overlay view on a PDF page.

When it comes time to save, you'll need to incorporate your content into the PDF.

And, speaking of saving, we'll cover some best practices when saving a PDF document.

Installing overlay views on PDF Pages is straightforward. Because PDFs can contain hundreds, if not thousands, of pages, there's no way you want to create views for all of those pages when opening a PDF. And what if the user scrolls back and forth rapidly? How do you know when to create your views? Fortunately, PDFKit is already designed to intelligently prepare content before people scroll pages into view. So it knows best when to ask for an overlay view. Your app just needs to respond to its requests that are made through the new protocol.

PDFPageOverlayViewProvider is the new protocol. By the way, PDFKitPlatformView is just a define of UIView or NSView, depending on the platform. The most important method you need to implement is overlayViewForPage.

Simply provide an instance of your view, and PDFKit will size it by applying the appropriate constraints. It will also rotate it if the page has a non-zero rotation.

The next 2 methods are optional. willDisplayOverlayView can be used to install your own gesture handlers, or to set up failure relationships with those of PDFKit.

willEndDisplayingOverlayView is called when PDFKit is done with your view, likely because the page scrolled out of view. You can release your view here, but there is another important use for this method. Assuming your view has some data to represent what it's drawing, you can use this method to get that data and set it aside. We'll do that in our example with PencilKit, but, if your view data lives somewhere else, you won't need to implement this.

In the example, this is the class we're using as the provider. It implements the PDFPageOverlayViewProvider protocol. This is iOS, so the PDFKitPlatformView is a UIView. It uses a map to go from a PDFPage to a UIView.

Here are the placeholder protocol methods. Next, lets look at the implementations. overlayViewForPage checks its pageToView map to see if it's already made a view for the given page. If not, it creates a new view. In either case, we get the drawing from the page and set that on the canvas view. In these examples, I'm using a subclass of PDFPage. All it does is to add a "drawing" property.

And now, let's focus on the next method: WillEndDisplayingOverlay.

willEndDisplayingOverlayView is simple. It gets the drawing from the view and stores it on our custom page class. Now that we've done that, let's see it in action.

Normally, around this time off the year, I would be in Maine, fishing, but instead, I'm here at WWDC, so another person is taking my place on the trip, and I'm going to show him some of my favorite spots. I'll be doing that with this app, which uses PencilKit in an overlay view.

This app consists of the code we just saw and little else. The entirety of the code to get overlay views onscreen is about 30 lines! So, Grand Lake Stream. This is the dam pool. There are always lots of fish in there. It's where most of the action is. You can get to the dam pool by taking this trail down through the woods and then fishing. You can fish all of that stuff, or you can take this road, go over the dam, and down around through here. From there, you can fish up into here, go around the island, down into here, but whatever you do, don't go past here. The water gets deep and fast. Avoid that and come down here, to the hatchery. Walk down beside the hatchery and enter this pool here. You can cast all around through here This is a great spot. I always catch fish here.

All right, now that we have some marks on the page, let's exercise zooming and scrolling.

See how responsive it is? And there we have it. Overlay views in PDFKit. So, now that you have these sketches, how do you save them? We'll use the PDFAnnotation class to do this. There are 2 things we want to achieve when saving: We want to match the onscreen appearance with high fidelity, and we want to do round-trip editing. PDF annotations have some features that will facilitate this. PDF annotations can have an "appearance stream," which is a stream of PDF drawing commands. Nearly anything that you can draw using Quartz2D can be recorded in an appearance stream. Anything else can be rendered into an image, and that can be recorded. That's how we do it if we're using Metal.

And, since it's recorded as PDF drawing, it will look identical in Adobe Reader, Chrome, etc.

PDF annotations are stored as dictionaries in a PDF document. That means we can also store custom data in private key/value pairs.

So let's see what the code looks like. Start by creating a subclass of PDFAnnotation. We do this in order to override the draw() method.

PDFKit will call this method when it saves the appearance stream that I mentioned on the last slide.

To save our document, we override UIDocument's contents(). Here is an overview of the function, for context later. We loop through all pages of the PDFDocument. We'll flesh out the loop next.

We do the following for each page: create an annotation of our custom class; encode our drawing into data; add the data to our annotation. Next time, when we open this document, we can use value:forAnnotationKey to read back the stored drawing data and put that into our overlay view.

Finally, add the annotation to the page.

Back to our contents() override. Now that we have added annotations to our pages, We use PDFDocument's dataRepresentation() and return the result.

When your content is saved as an annotation, a recipient of the document can move it, resize it, or delete it. Often, that's what you want. But sometimes, you want your annotations to be "burned in," as part of the page. There is a new PDFDocumentWriteOption in iOS 16 and macOS Ventura that makes this easy. Just add burnInAnnotationsOption = true to the save options, and that does it.

Speaking of PDF writing options, there are a few that have been made available in iOS 16 and macOS Ventura. Let's look at them. CoreGraphics has always strived to save images in PDFs with maximum fidelity, so images are saved at full resolution, with lossless compression. That's great if the PDF will be printed on a large-format printer. More likely, though, it will be displayed on screen, and all of that high-fidelity image data will result in a file that's very large. To address that, I introduce the next two options.

saveAllImagesAsJPEG does just what it says. No matter how the image was created, it will be saved with JPEG encoding in the PDF.

optimizeImagesForScreen will downsample images to a maximum of HiDPI screen resolution. These two options may be used together.

createLinearizedPDF will create a special kind of PDF that's optimized for the internet. The PDF format, as originally designed right before the Internet came along, is read from the end of the file. That means the entirety of it needs to be downloaded first before anything is displayed. A linearized PDF has everything needed to display the first page at the beginning of the file, so a web browser can show that quickly while the rest of it is loaded.

You can pass these options to PDFDocument's dataRepresentation or writeToURL methods. And there we have it. PDFKit is powerful, yet easy to use, used by many of your apps today on iOS and macOS, and now with new features for iOS 16 and macOS Ventura. I'm stoked to see what you do with them! To learn more, check out the sessions below for additional information. Thank you for watching!