Wes Bos described an iPhone app and modeling workflow on Oct. 22 that he says can turn a room scan into a 3D model. The app scans with LiDAR while taking hundreds of high-quality photos. His workflow then models objects and checks them against photos taken from the same positions and angles.
In his video, Bos explains why he combined the two inputs. Photos alone left the size and layout of his office for the model to infer. He then tried an iPhone LiDAR scan with Polycam, but found its images too blurry to represent some objects faithfully. His own app scans a room into a LiDAR mesh, records photos and their camera positions, and uses Apple's Room API for additional information about the space. The scan data includes depth maps associated with the photos, he says.
Bos calls the next piece Scan to 3JS, a set of Python scripts and instructions for an LLM. According to his explanation, it positions a camera in the 3D scene to match a source photo's viewpoint, compares the scene with the photo and iterates on the model. He says he sometimes has to tell it to keep going.
The video's description says Bos modeled his house by building an iPhone app to capture photo and LiDAR data, then stitching the data together with GPT-6 Astra in Codex. It labels the video #ChatGPT_Partner. Bos describes his office as “modeled to the inch,” but also says the iPhone's LiDAR mesh is jagged and missed some points. The accuracy of the finished model remains his assessment.
Comments