elkraneo

Illustration of a rectangle wrapped by a three-dimensional ribbon giving the sensation of melting in reference to the dilation process for outline drawing

RealityKit Featured

RealityKit Outline Selection

TLDR: MeshResource.contents, three passes, one PostProcessEffect 1. Rasterize the selected mesh into an offscreen R8Unorm silhouette mask 2. Dilate the mask into a pixel-wide edge ring, suppressing the interior 3. Composite the outline color over the source frame One of the most commonly requested affordances when dealing with 3d

Deconstructing Reality Composer Pro (Inspector Components ...continued)

Reality Composer Pro

Deconstructing Reality Composer Pro (Inspector Components ...continued)

This latest wave of fixture work extended the reverse-engineering effort across Reality Composer Pro’s built-in components. Now it's documented and implemented more fixture-backed components across Deconstructed and Uncannyuse, including Environment Lighting Configuration, Grounding Shadow, Image Based Light, Image Based Light Receiver, Virtual Environment Probe, Reverb, and the

An open sphere in an abstract graphic represents a dome with a stage, a cube acting as the main character, and several more cubes backstage

Reality Composer Pro

Deconstructing Reality Composer Pro (Inspector Components)

The previous inspector series parts explored attributes that map cleanly to USD: transforms, material bindings, and variant sets. Components are different. One might expect that RCP's component inspector would just show RealityKit components attached to entities—the same components you'd add in code at runtime. But

Illustration of a plane with colorful stalactites and stalagmites representing the diversity that can exist in hidden places

Reality Composer Pro Featured

Deconstructing Reality Composer Pro (Inspector Bindings, References and Variants)

The next level of functionality in the RCP inspector offers deeper control over model linking, looks, and variations Bindings In OpenUSD, the terminology "Binding" comes from the UsdShadeMaterialBindingAPI. Unlike traditional formats where a material is simply a property of a mesh, USD treats materials as independent prims that

Deconstructing Reality Composer Pro: Logbook

Reality Composer Pro

Deconstructing Reality Composer Pro: Logbook

A continually growing list of interesting things found during exploration that have not yet been featured in the "Deconstructing Reality Composer Pro" series entries. Mapping from Prims to Entities RCP maintains a bidirectional mapping between RealityKit entities and USD prim paths. While public RealityKit doesn't easily

Illustration of 3x3 boxes in perspective, similar to the boxes on the transform inspector in RCP.

Reality Composer Pro

Deconstructing Reality Composer Pro (Inspector Basics)

Dual-mode Inspector: scene layer shows global stage metadata (upAxis, metersPerUnit), while prim inspection displays transform, materials, and references. UI presents cm/degrees; USD persists meters/quaternions with rotationEulerHint for round-trip editing.

Deconstructing Reality Composer Pro (Scene Navigator)

Reality Composer Pro

Deconstructing Reality Composer Pro (Scene Navigator)

To implement the Scene navigator, we will need to handle .usd manipulation, and the best approach I have found is to use Apple's SwiftUsd project. Although it is a bit behind the standard and has some quirks with C++ interops, it remains powerful and convenient, and by including

Illustration of a blob melting in the center of a grill, representing a three-dimensional viewpoint.

Reality Composer Pro

Deconstructing Reality Composer Pro (Viewport)

Viewport is deep (modes, debug views, presets/history). We are going to implement just a working viewer + camera for this pass; chasing full parity will balloon scope. The main viewport is built with RealityKit, and we'll add a layer for gestures, a grid, and a simple Arcball implementation

Deconstructing Reality Composer Pro (Project Browser)

Reality Composer Pro

Deconstructing Reality Composer Pro (Project Browser)

Part of what makes Reality Composer Pro a composer is asset management. Importing, soft validation, organizing, and iterating on the project structure are essential features to have in creative pipelines. The project browser in RCP is similar to Finder's grid view and, because of that, filled with UX

Deconstructing Reality Composer Pro (Document Type)

Reality Composer Pro

Deconstructing Reality Composer Pro (Document Type)

An exploration of the .realitycomposerpro document type. This post breaks down its structure as an SPM-backed package, the nested bundle system for assets, and how this layout could enable Xcode and `realitytool` processing.

Illustration with 3D perspective of a phase-layered stack of cubes ending in a dark plate with fewer elements and distribution. Tool abstraction metaphor from this article.

Reality Composer Pro Featured

Deconstructing Reality Composer Pro (Intro)

Reality Composer Pro prioritizes the transition of elements over direct emphasis on creation and debugging tasks. It's a portal where you transport raw, wild formats to more manageable, controlled, static-defined files that can be represented on RealityKit.

Illustration of abstract figures (white cones) concentrating and melting with a central cube (purple)

Compiling .reality: The Technical Reality

Xcode runs a verified two-pass process to compile .reality files: first generating USD schemas from your Swift components, then compiling them into optimized bundles.

Developers should not have access to eye tracking information

...and there are strong reasons against it. The conversation could be philosophical, but it highlights the conflicting opinions on whether commercial interests should take precedence over human well-being. Also, the few potential exceptions could be handled in controlled environments using entitlements or research. What Does Your Gaze Reveal About You?

The Aesthetics of Immersion

The Aesthetics of Immersion

Synthetic environments and traditional landscape paintings

Screenshots from both apps running in Tahoe show the initial list of contents.

WWDC25 updates to Transwwdcripts & Resourwwdces

Just updated the data source to include WWDC25 contents and recompiled for Tahoe because it is nicer... Download Transwwdcripts v1.2Download Resourwwdces v1.1TranswwdcriptsIn the spirit of ASCIIwwdc, understanding that transcripts have significant benefits for accessibility and research, creates a situation in which making text versions of talks available becomes

An abstract drawing of a coliseum that shows the relative amount of spatial computing on this year's WWDC categories

Guide to the latest in Spatial Computing (WWDC25)

TL;DR * Spatial computing is in a transitional phase: the technology is robust, but the market and use cases are still catching up. * Apple’s approach is methodical, not flashy: rather than chasing hype, Apple is building the infrastructure—standards, APIs, specifications, developer tools—that will power the next wave

Light Without Boundaries

Light Without Boundaries

TL;DR, If you want virtual environment light gradually replacing your real-world shaded objects, use Virtual Environment Probe. Achieving visual realism for virtual elements involves many parts, but lighting stands out as particularly challenging, especially in real-time rendering for platforms like visionOS and RealityKit. Material surface behavior in response to

Illustration of a gaming board with a grid of deformed squares and a volcano in the center, everything melted comic style

The Framework Next Door

TL;DR, If you are interested in building a spatial game (or just about any immersive experience), you'll probably want to implement an architecture like TabletopKit. Create multiplayer spatial games on a virtual table surface and use FaceTime to invite players. TabletopKit first impressions We are creating a

A screenshot of the current version of Reality Composer presenting the subdivision scheme options

Subdivisions in RealityKit

TL;DR: SubD can currently only be activated on Reality Composer Pro and for geometry using physically based (not custom) materials. While working with 3D, especially for real-time rendering engines like RealityKit (the current engine powering visionOS), balancing geometry, material types, sizes, and render passes is critical for creating an

Abstract diagram of repeated geometric objects resembling vertebrae and their overview

Notes from "Towards Automated Accessibility Report Generation for Mobile Apps"

Towards Automated Accessibility Report Generation for Mobile AppsMany apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues…Apple Machine Learning Research One of the most commonly used tools for generating audits on Apple platforms is the Accessibility Inspector, specially

Abstract 3D illustration of a shape that looks like a shoehorn being pushed by a drop, representing force being applied

Voluntary Product Accessibility Templates

Conformance to accessibility standards reports or statements differ in format depending on region, but when referring to Information and communications technology products (ICT), they usually follow closely the Web Content Accessibility Guidelines (WCAG). The Voluntary product accessibility template (VPAT), a brief but informative review of how various WCAG criteria apply

Stylized illustration of a melting cube with axis indicators and a ring around it. It mimics a 3D viewfinder

Entity.write(to:)

Reality Composer from macOS (not Pro) was quietly sunset not too long ago. Aside from being a graphical editor that allowed for basic scene setup, it also had other great utilities, such as cross-platform preview and some popular features that allowed for creating interaction with triggers and events. Reality Composer

iPhone 15 Pro shows the controls gallery

Control Center Accessibility (iOS18)

Control Center is about to receive an update on the upcoming release of iOS 18. Controls are a new type of widget that extend app functionality into system spaces like Control Center, the Lock Screen, and the Action button.There are two types of controls: buttons and toggles. Controls use

An abstract illustration of six rounded green 3D cubes surrounding a purple inflated plain

Accessibility & Innovation (intro)

In one of my most recent talks, I was asked during the Q&A session about recommendations on which approach to use to promote-advocate for accessibility within an organization. I swiftly went through the human/moral case, the legal case, and the business case, citing Apple practices, European legislation,

Screenshot of the Shortcut implementation in macOS

Get Youtube channel RSS

I'm an avid user of RSS, and Readwise Reader is my main funneling tool given its ability to combine read-it-later features with an RSS reader while simultaneously acting as a good citizen in a pipeline of highlighting. One use case that has grown in me over the last