📱 I wanted to play with my LEGO Research Lab in augmented reality! It would be great to be able to pull a mobile device out anywhere and demo 3D physical forms in augmented reality to think through ideas visually with other people. In some cases, this would clarify visual designs much better than words or simple whiteboard sketches. Combining LEGOs and augmented reality seemed to be a good way to make a proof of concept. LEGOs themselves are made for modularity: to be taken apart and reassembled in different configurations.☁️ The first thing I tried was the Scaniverse App, which uses LiDAR. LiDAR applies lasers to determine ranges and create point clouds.The problem with this is LEGOs are generally small surfaces and they are shiny. When an object reflects light, this reduces image quality. I noted that the LEGOs looked as if they were molten. Indeed, there was a recent heat wave in California, but it was not enough to melt LEGOs! LiDAR is great for scanning larger objects, such as rooms or life-sized pieces of furniture.

🔴🟢🔵 Taking pictures with an RGB camera seemed to render the LEGO augmented reality objects truer to their form.Moving the LEGO setup on a table with wheels to another room with different lighting slightly improved the quality of the rendered augmented reality object.📷 Next, I tried using the Metascan app (free version), which used pictures from an RGB camera to create augmented reality objects.

Although some of the more reflective LEGO pieces appear a bit molten, this worked out much better. The video is the one in this post: constructed from 30 images.Making this demo was a fun fast exploration. Indeed, there are better methods of photogrammetry and methods that use more images.

Thank you for suggestions for what to look into next for scanning/imaging/processing methods, Gary Markowitz, Sterling Cooley, Mark Melnykowycz, and Mark DeLoura.

Lateral Tinkering

Background Research for the PsychedeRx Mini Series