SHARP Memories

(iOS, Next.js) Converting 2D photos into immersive 3D Gaussian Splats using a serverless, event-driven architecture.

SHARP Memories turns a single 2D photo into an immersive 3D Gaussian Splat you can view and share on iOS, the web, and Apple Vision Pro.

Most 3D capture tools require LiDAR or 50+ images. Using Apple's open-source SHARP model, I built a pipeline that needs only one input image, so anyone with a smartphone can do it.

SHARP Memories App Showcase

Built for Sharing

I could have generated and stored the splats locally on-device, but the whole point is the moment you "revive" a photo and immediately want to show somebody. So I designed around distribution and retention from the start.

1. Virality via App Clips

The truth is people don't want to download another random app. To solve the "cold start" problem, I built an iOS App Clip. When a user shares a memory via iMessage, the recipient doesn't need to download the full app. They tap the link, and the 3D model renders instantly in a lightweight viewer (< 15MB).

2. The "Snapchat" Mechanics

To release this publicly without paying for storage, I implemented a "Snapchat-style" lifecycle:

  • 0-24 Hours: Memories are fully visible in 3D.
  • 24 Hours - 7 Days: Memories "fade" (turn gray). Users must tap to "resurrect" them, triggering a re-generation.
  • 7+ Days: Hard deletion to respect user privacy and storage limits.
SHARP Memories App Showcase

Gaussian splats are large files (~60MB), and my Cloudflare R2 bucket is on the free tier. Deleting the heavy splat after 24 hours while keeping the small preview image for 7 days keeps storage flat, and the "faded" state gives users a reason to come back.

Architecture: The "Free Tier" Stack

My main constraint was cost. Storing gigabytes of PLY (point cloud) files is expensive, so I stitched together a hybrid storage setup from the best free tiers of different providers.

The Event-Driven Pipeline

The backend is fully serverless and asynchronous, so the UI never blocks while the GPU crunches data.

  1. Ingestion: The client uploads the raw 2D image to Supabase Storage (1GB Free Tier).
  2. Trigger: An Edge Function inserts a row into the splats table.
  3. Inference: A Supabase Database Webhook detects the INSERT and calls the trigger-inference Edge Function.
  4. GPU Processing: The function calls Modal AI, where I host the Python ML inference code on serverless NVIDIA T4 GPUs. Modal returns a 200 OK immediately so the connection doesn't hang.
  5. Storage Optimization: Once generated, the heavy PLY file (~60MB+) is uploaded to Cloudflare R2 (10GB Free Tier), which offers zero egress fees.
  6. Realtime Feedback: The clients subscribe to Supabase Realtime updates. As soon as the status flips to completed, the UI updates automatically.
// Example: Supabase Edge Function snippet for triggering inference
Deno.serve(async (req) => {
  const payload = await req.json();
  const { record } = payload;

  // Offload to Modal AI (Python)
  const response = await fetch("https://modal-api-url/generate", {
    method: "POST",
    body: JSON.stringify({ image_url: record.input_image_url }),
  });

  return new Response("Inference Started", { status: 200 });
});

Try it out!

Visit sharpmemories.app, upload a photo (you'll need an account), and play around with the 3D memory. I'm sure it will bring you a curious sense of nostalgia.

You can also download the iOS version here

SHARP Memories App Showcase

This project was a lot of fun to make, especially because I could view it on my phone, my laptop, and even the Vision Pro. Plus, I could share it with people!