Weekend Build Fuses Meta Ray-Bans Footage Into 4D Reconstruction

I used fable to improve my old shot counter project from last year - ended up implementing my whole future roadmap in two nights. Pretty wild. Here’s the old project which used sound and visual cues to count and classify shots:

This is IronSight - a 4D reconstruction built by fusing footage from two pairs of meta ray-bans. A google research buddy saw the prototype and joked this would've been a siggraph paper a few years ago. Today it's a weekend build with fable. Stay tuned for more mad science. Happy 4th!

@SherbyJohn Glomap and pi3 as fall back. I tried lingbot map but unfortunately it wasn’t very good for this use case

@The_Don06 That was the fun part - nice mix of classical and ml. Will do another post with a proper breakdown

@bilawalsidhu How did you get the shots and hit markers captured on the map??? Wild.

@vaibcode Queued up a bunch of experiments last night to test out my ideas and select the right primitives. Then just a few hours today. So much fun

@bilawalsidhu Holy shit, this reminds me of Brain Dances from Cyberpunk 2077

@bilawalsidhu Thanks for you sharing all of these wild explorations. You help make every day an adventure!

@bilawalsidhu how long did this take to make? I've been finding fable quite good for performance work (with steering).

@bilawalsidhu Did you use something like lingbot map for scene reconstruction?

@bilawalsidhu did you reconstruct the scene prior ?

@Jyunnie3102 Nope, all from the meta glasses

@bilawalsidhu The 2A community would love this shit

@Shiffo12 Hah yeah they use the point cloud aesthetic beautifully in that game! Basically a 4d replay of reality

@bilawalsidhu Bravooooo

@bilawalsidhu @SherbyJohn Why glomap over colmap haha

@bilawalsidhu wowee 🔥🔥

@bilawalsidhu This is amazing

@andrew_holdun Mad science experiments are so much fun

@bilawalsidhu could raise a pretty penny on this alone these days lol