Newsletter #2

Published 2021-03-09
The Terminator saying I'm back"

Guess who's back.

Strange but probably true

In honor of Pi Day which is coming up, I would like to share the following statistics.

  • Pi used to be equivalent to the average number of posts on the Lens Studio forum before someone asked how to be verified on Snapchat, but the equivalence has now changed to the average number of posts before someone has an issue with Snap Camera.

  • Pi is also equivalent to the average number of comments on a post shared in the Spark AR Community Facebook group before someone asks you to buy their patches on Gumroad.

Neither of those statistics are true, but 1) I like to poke fun and 2) they raise some important issues. Snap Camera is sometimes really frustrating to use and could probably use some TLC and better documentation. And if you are an AR creator, how do you earn money? You work for brands, you sell courses, or you sell resources. I wish there was more support from the platforms we create on (like how YouTube lets creators monetize their videos), but for now we are on our own.

That one time I tried ZapWorks

I noticed that ZapWorks has a free Hobby tier so I created an account and downloaded ZapWorks Studio. As a first project I figured I'd recreate the twerking alien lens. I opened up the World Tracking example, deleted the boring coffee table, and tried importing my 3D model. The model name showed up in one of the panes, but nothing showed up in the 3D viewport. I read the documentation, saw that I could only have one material per model, made some changes, re-imported, and still nothing.

I figured there was something wrong with my model (even though it works in Lens Studo, Spark AR, and Microsoft's built-in 3D viewer). I downloaded an example 3D model from ZapWorks' own website, imported it, and still nothing. The 3D import modal didn't appear and the model didn't appear in the viewport. I closed the software, re-opened the project, and voila! I could see something! But the normals all seemed to be reversed 😢 At this point I threw in the towel. I wasn't sure if it was their demo model, ZapWorks Studio, or what was going on.

But then as I was writing this newsletter, I decided to give it one more try. Still no import modal like in their documentation, but I did discover that the scale of my 3D model was too big so I just couldn't see it. I resized it and added my material. So far so good! 🙌 I published my project, pulled out my phone, scanned the QR code, and got a message about my device not supporting world tracking 😭 I can place the twerking alien on the ground, but the tracking is super janky and the animation doesn't play.

I was excited to try ZapWorks and start venturing into the world of WebAR, but ZapWorks does not seem to be ready for primetime yet. Maybe I'm just spoiled by how great Lens Studio is. I'll probably keep playing around with ZapWorks, but it's not going to receive any serious attention from me right now.

What's new?

Spark AR released v108 to add native support for M1 Macs, so I guess that's exciting if you have one of those.

Lens Studio released v3.4 which actually has some new features, but it doesn't run natively on M1 Macs (but it does run). This release adds 3D full-body tracking, full-body segmentation, hand joints tracking, and a new asset library. Apart from the release, the Lens Studio team also created a fake liquid material that looks pretty fun to play around with.

I don't have any types of filters in particular that are trending, but the copycat wars rage on. Apparently, a Spark creator had their work copied by TikTok. The same filter also now shows up on Snapchat's trending section (at least it does for me at the time of writing this). It seems to be a popular effect, so I would say it was a strategic act of copying by TikTok. Based on the username I think the Snapchat version was created by the original creator of the Instagram version.

And while we're at it I also copied a TikTok filter and made my own Snapchat version. There'll be a full tutorial for it, so soon you'll be able to copy my copy. Just be sure to change it up enough so we don't get caught 😉

Helpful tricks

Lens Studio copyFrame for segmentation

Lens Studio has an awesome function called copyFrame that lets you copy an image texture. It must be used with care because it adds to the memory usage of the lens, but for my clone lens (the one I shamelessly copied from TikTok) it was exactly what I needed to make the clones. I ran into an interesting bug where everything worked inside Lens Studio but crashed on my phone. I did a little asking and Ben Knutson let me know that apparently you cannot call copyFrame directly on a segmentation texture. If you need to do that, you need to output the segmentation to a separate render target and then call copyFrame on that.