Swift iOS: Camera Output

Apps that can integrate smart camera solutions are often more useful than competitors. Airlines provide mobile software for flyers to import documentation. Some apps require the user to manually import their passport information prior to flight. But I believe the more elegant solution is having the device camera examine the physical passport. United does this pretty well, their camera is accompanied by labels and outlines that assist the user in working correctly with the app. This feature has been around since 2014,

It is not too hard for developers to implement their own iOS smart-camera experience; it starts with collecting camera output. In iOS we have to first declare in our plist a privacy statement for camera permission. This shows the user a reason behind camera use. And using parts of AVFoundations without it, you'll crash.

Now this is often a little fun for me, if you create an AVFoundation discovery session and modify the device returned; the system will show your privacy statement next to a prompt for camera permission to the user. We do not crash altering a camera device, however if we start the capture session without acceptance there's a crash. It is usually better to instead handle permission prompt prior to creating camera objects. Inside AVFoundations exists a handy class method for prompting the user to see our privacy statement on top of a system handled button. If response comes back false, the user has clicked no. In this case the app will transition to a new screen that explains why we need the camera to continue. This screen has shortcuts for enabling system camera access for the app.

This call is generally all you need to work with the camera properly. But if you want to handle denial as a state, it's good to wrap the request inside the existing authorization status. In the example below, if the user has previously denied camera access; the app does not attempt to request access again. Also if the user has previously accepted camera access, the app does not request access again, instead it creates capture output. This is a very useful snippet for viewDidAppear.

There are two methods we'll need to implement to conform to being the sample buffer delegate. One will be what we want the app to do when the buffer drops a frame. In a lot of apps the best thing to do is often clear some app state or cache of states. But in many apps an empty implementation is fine.

The other required method occurs after the buffer outputs a frame. This is our place for creating an image from the buffer, and changing app state. The image size is set before the first frame is sent, on our capture session we declare a preset.

It is really important to be brief inside didOutput. Bogging up this pipeline will result in more dropped frames. The other problem is that we often want the most recent frame to update the UI, if we poorly handle this pipeline (too slow to update UI or UI never reaches the desired state) we can create a poor user experience.

One of the easier ways to enforce that we're working with the most recent frame is with a semaphore. This handy object will signal after we exit the current scope, inside the scope we read the frame from the buffer and perform app operations. Reading the buffer is easy. But it's also important that the buffer is ready or we could have a strange image. When the buffer isn't ready, have the semaphore signal that it's ready to try a new frame.

The app in the screenshots takes images from the buffer and places them in an imageView that can be zoomed. It is an app you can get on the store.