Deconstructing Reality Composer Pro (Document Type)

First, let's lay down and establish some sensible defaults to prepare for the absolute certain complexities to come. The base I will be using is going to be basically a macOS app with document file management, probably the file template from Xcode.

As for the stack, I'm going to be primarily using Xcode for development, with Zen Editor behind the scenes and some agents. Codex and Claude Code are my go-to tools, supported by open-source models, and I also experiment casually with other tools. I'm going to eliminate backwards compatibility to simplify my work and concentrate on understanding the transition from .usd to .reality format in production. I'm going to remove the sandbox too. This is based on previous experience with bundling CLI and how external processes are dealt with, which, while not the main focus here, is still valuable to understand. So playing (un)safe here.

I prefer using The Composable Architecture, and even if I recognize that certain (hot) paths should not be connected to reducers (particularly when working with rendering engines and similar runtimes), the benefits of modularization and control over actions and state sharing are invaluable. I have also found that when using strict frameworks, agents can handle complex features more effectively while working in isolation on a specific module that follows the rules of TCA in our case.

I already have some instructions customized for working with it, which I used in the past. Indeed, there is a connection between the output quality, the tools used, and the structured frameworks, which I value when managing intricate component interactions.

So yeah, it's primarily a macOS-only app that uses TCA architecture, focusing first on representing and displaying .usd files (basic prims) and then on manipulation. And yeah, that's roughly the initial roadmap. See how the components interact with the hierarchy's entities and how we define an entity. What is the match between a .usd hierarchy and a RealityKit one? It will be fascinating to see how we constrain those and how we keep their compatibility. In the second phase, we must decide whether to focus on the animation aspect or the shaders, while also addressing the statistics report tools and additional features that may excite us. Additionally, there is extra functionality in scene management that seems more complicated than it appears, particularly related to the resolution of paths, textures, and file references. That could be challenging. There is also a small menu in the hierarchy inspector that includes some premade components, such as acoustic and lighting options, among others. The creation aspect of this menu is intriguing.

There is a warning section, and there is also a preview feature. Additionally, directly connecting to the Apple Vision Pro and triggering a preview could provide an intriguing bonus. Finally, all of these features can be combined into a package that is compatible with both Reality Composer Pro and the Deconstructed project. This will allow us to leverage realitytool compilation on Xcode and ensure a completely valid .reality file suitable for real projects.

The type of file produced by Reality Composer Pro is the .realitycomposerpro package. The file structure consists of a bundle nested within another bundle, which may seem complex, but I believe it is a solid choice. The package is an SPM file that contains a bundle, which in turn includes the assets. The structure is an SPM package at the root level, containing additional predefined directories, so Xcode and realitytool can recognize how to do the compilation part.

Now, if you show the package contents in .realitycomposerpro, you will also find some items related to the management of the app itself. The metadata and some lookup tables are optimized to reference specific parts of the project itself. I have observed some runtime development artifacts, such as a shader cache compilation during runtime, along with other elements that we will discuss in the future. All of these elements appear to be designed to make RCP as fast as possible while allowing state restoration.

Base/                                      # SPM package (wrapper for integration)
├── Package.swift                          # SPM manifest
├── Package.realitycomposerpro/            # <- THE DOCUMENT (what we open/save)
│   ├── ProjectData/
│   │   └── main.json                      # UUID mappings (paths reference ../Sources/)
│   ├── WorkspaceData/
│   │   ├── Settings.rcprojectdata         # Editor settings (JSON)
│   │   ├── SceneMetadataList.json         # Hierarchy state
│   │   └── <username>.rcuserdata          # Per-user prefs
│   ├── Library/
│   └── PluginData/
└── Sources/                               # Assets (sibling to document)
    └── <ProjectName>/
        ├── <ProjectName>.swift            # Bundle accessor
        └── <ProjectName>.rkassets/
            └── Scene.usda                 # USD scene files

Fortunately, we have access to the Swift package init CLI, which enables us to invoke it directly for specific purposes. Unfortunately, during package creation, we are unable to create templates or make significant customizations; instead, we must make manual adjustments. This solution is somewhat fragile and naive but better than handling everything manually.

The internal package structure .realitycomposerpro is stored in JSON format. We will define the models as codable. Currently, we have limited information or understanding of the internal storage structure. That's why we'll just lay the groundwork for expanding these models. Our aim in this phase is to develop an app that can create a new valid RCP bundle upon launch, containing a valid .realitycomposerpro package inside. We should be able to open the bundle in Reality Composer Pro, make edits, save it, and then reopen it in the Deconstructed app seamlessly.

[Pitch] Improving package creation with custom templates: SwiftPM Template Initialization

Feel free to DM me and we can work out a time to have a call/chat

Swift Forums0xTim