Hologram Product Remixing

"Make the shoelaces red." Seconds later the sneaker on the hologram had red laces, and nothing else about it had changed.

The challenge

PROTO makes hologram displays, and ran a competition for experiences built on them. We wanted to show what generative AI could do for products: speak an idea out loud and see it on a hologram almost immediately. This was the early days of image generation, and "almost immediately" was the hard part.

The other hard part was precision. Remixing a product is only useful if it stays recognisably that product.

What I built

A pair of sneakers rotates in 3D on the display. You speak: "make the shoelaces red" changes exactly the laces, while "turn this into a car" rebuilds the sneaker as a car that keeps the product's shape and composition. You can also start from a new product entirely. A few seconds after each change, you can spin the new version around in 3D.

There's a setting for fun, too: turn the controls down and the remixes can go off the rails.

How it came together

Most of the two weeks went into setting up the image-generation workflows and exposing them as something I could trigger on demand from cloud GPUs. The React app and the experience took a few more days. We built the two in parallel, so feedback on the experience could shape the pipeline while it was still being built.

What happened

It won first place, and the prize was a desktop hologram machine for our lab. At an internal lab week, people were most surprised by how fast the turbo models were, and by the precision: one change, made exactly, without disturbing the rest of the product.

What I'd take forward

As a permanent retail install, the focus shifts to guardrails: classifiers that moderate generated brand content, evaluation sets so new models can be swapped in without the quality drifting, and a staff portal to review generations and feed their feedback into the next round of fine-tuning.