CueCam Presenter User Guide

How-To Guides

Tutorials

Explanation

Reference

Frequently Asked Questions

Explore

Blog

How can one person build this?

I’m a solo developer working across all the necessary technical, creative and business disciplines required to produce an app that people will actually use.

How have I created an ecosystem of interconnected cross-platform software with all this native code while maintaining a relatively approachable user interface?

The short answer is Apple.

Apple’s SDKs and Frameworks make it possible (if not always easy) to put together very ambitious native applications as a solo developer.

With

SwiftUI⁠

Apple embraced the declarative UI model pioneered in web development by frameworks like React. Before SwiftUI came out, I was on the verge of building something very similar of my own, having spent years of frustration dealing with NIB files and storyboards or cumbersome, imperative techniques for building UI in code which remained verbose, repetitive and extremely aggravating even after the move from Objective-C to Swift. Once something was working, I was reluctant to touch it again. Now I can not only write concise, elegant code to build some pretty complex UIs, this code is often portable between iOS and macOS (switching between AppKit and UIKit had been a very different story).

You’d think that dealing with realtime video would be a whole profession in itself. I’ve watched Linux and Android developers trying to write basic video recording and playback software as they are forced to drop into ffpmeg command line shims or deal with the infinite possibilities of different hardware capabilities and device drivers. This isn’t how things feel on Apple platforms. They can wrap just about everything up into a few

AVKit⁠

calls and let Apple deal with all the hardware interfacing behind the scenes. All I really have to think about is my application, instead of worrying about the implementation details of webcams and video codecs.

With

Metal⁠

I’m able to take advantage of Apple’s graphics hardware to spice things up visually without tanking performance. Sometimes I do my own compositing directly in Metal, sometimes I jump up a level into

SceneKit⁠

so I have a visual editor and some affordances like shader modifiers that let me avoid writing too much boilerplate, and sometimes I use

SpriteKit⁠

to throw together a cute particle effect before bringing everything back together in Metal (using SCNRenderer and SKRenderer to “translate”).

With

Network.framework⁠

I can create my own inter-device communication protocol on top of TCP to send video and control messages between devices in a reliable and extensible way. I open-sourced some of this code in

ShootKit⁠

in the hope that somebody will build an OBS plugin to use Video Pencil or Shoot without all the usual steps, but I’m not even sure if this would be portable to Windows since it’s an Apple-specific library that remains mysterious to me even after I’ve been walked through it in great detail by an Apple engineer!

Audio wasn’t easy. I had three runs at the audio stack before I finally got something that worked perfectly. I tried my usual approach of leaning on Apple’s higher level frameworks but AVAudioEngine on macOS couldn’t cut it. I tried making my own AudioUnit plugin and that had problems too. In the end I realised that I needed to see a lot more source code to make everything work and revisited Michael Tyson’s

The Amazing Audio Engine⁠

which had served me well a decade ago when I built my

Infinite Music Machine⁠

. Michael has retired the project for the general public but I’ve found it to be extremely useful, letting me do things that his suggested alternatives aren’t capable of doing.

This brings into focus the biggest problem of working with Apple’s frameworks.

They’re not open source.

I wish Apple frameworks were open source

For most “mainstream” use cases, Apple’s frameworks are very appealingly and beautifully documented.

But once you start getting into the more esoteric domains, things quickly fall apart.

You might think that, as a software developer, most of my time is spent building software.

But the bulk of my time is actually spent trying to figure out the magic incantations required to make Apple’s frameworks go from doing nearly what I want, to doing exactly what I want.

In open source, this is never a problem. You just jump into the source code of whatever library you’re using and read it. You can quickly see if it does what you want and if not, fork it and do it yourself.

Because it’s open source, some other kind developer will have likely already gone through this process so you don’t have to (if not on the main project, discoverable as a fork via Github’s network explorer).

This saves the hours of trial and error, tweaking one thing, building, tweaking something else, rebuilding, hoping it works, trying not to forget what you were checking each time, or trying to figure out that one WWDC talk from 2015 where the promised sample code was never published.

How is memory managed when I retrieve a CMSampleBuffer from an AVCaptureDevice and send it to the virtual webcam via the CMIOExtension framework? I guess I’ll have to figure that out from the one sample project and the one project on Github that uses the library (where all the comments are in Japanese).

Why have Xcode Previews starting launching a new instance of my entire application every time I change a single character in my SwiftUI source code? Resulting in my screen capture permissions getting corrupted and requiring me to reboot if I want to use my app in a call that starts in five minutes? I may never know.

Why does the right-click menu on a button in a list stop working when it is in a sidebar? Why am I getting autolayout assertion failures 40 frames deep in a stack trace which does not include a stack frame of my own code? How do I prevent this List component from redrawing every single cell 19 times for every pixel it scrolls? Why have half my buttons suddenly turned blue?

I’ll figure it out but it won’t be fun!

Apple love to show what their frameworks can do but they’re allergic to admitting that there’s anything that their framework can’t do. We just have to cross our fingers and hope that they’ll add it a year later, when they will announce an essential addition that everybody was asking for as if it were the invention of fire.

So I spend as much of my time feeling like a frustrated private detective as I do as a designer or engineer.

A wild Chat GPT appeared!

Early 2023 changed things up. Suddenly I had a way to ask stupid questions without burning any social capital. I didn’t have to get the veteran Apple engineer to tell me what TCP stands for. I could fill in the gaps in my knowledge with my fragile ego intact (I mean “without wasting the time of the generous experts who occasionally took some time to help me”).

I could start thinking more abstractly about the features I wanted to add - things that I knew roughly how to do but could involve a few hours of tinkering to get perfect - in the knowledge that the robot would probably get me most of the way there.

I could get (what essentially amount to) personalised Stack Overflow answers and have the machine summarise findings that would otherwise involve me trawling through forum and mailing list threads punctuated with the periodic demoralising patronising outbursts crying “rtfm” at some frustrated noob.

I could bootstrap content! I could get a best-practice landing page structure! I could get machine generated content directly in Elixir maps to plug into the website. Find me a human marketeer that can do that...

Did I save time and energy and work fewer hours? No, I expanded my ambitions. Instead of bringing in a designer friend to help me with a new website, I got something nearly as good in a few minutes.

I love having this new window onto the canon on human knowledge, but as I get deeper into certain Apple frameworks it starts to have the same issues with lack of documentation. Often I will ask it how to do something and when I see the answer exclaim “yes, it would be wonderful if it actually worked way, but I wouldn’t be asking you for help if it did!” and we go into a cycle of the machine apologising and then responding with exactly the same incorrect answer as Chat GPT achieves its ultimate form: “over-confident mediocre white man”.

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.