Skip to main content

Backstory

Point your phone at any building and an illustrated tour guide tells you its real story, with every claim fa

About the project

You walk past a beautiful old building and wonder what happened there. Search gives you ten tabs, and half of them repeat myths.

Backstory turns your phone into a walking-tour guide. Knock on the orange door (or tap Snap a photo), take a picture, and an illustrated guide reads you the place’s story. Every claim is fact-checked and taped Verified, Disputed or Legend, with sources one tap away. Then Next stop suggests real historic places a few minutes’ walk away, with Google Maps walking directions.

How it works

  • Gemini (gemini-4-hydrogen) looks at your photo with your GPS address and names the landmark. It then pulls 5 to 8 claims from the sources and writes a 60-second spoken story that only states verified facts as fact.
  • Bright Data finds the history pages (SERP API) and fetches them as clean markdown (Web Unlocker). Wikipedia is the fallback.
  • TypeSafe Jev (System One) makes the judgment calls:
    1. which landmark is in the photo, and whether to ask you to confirm
    2. whether each claim is Verified, Disputed or a Legend, using only what the sources say
    3. which nearby places are real historic landmarks worth walking to (not restaurants, offices, stations or events)
  • Gemini 3.8 TTS gives the guide her voice. She reads from her notebook while the story plays, and each sentence is highlighted as she reads.
  • Sentry tracks errors and traces. DigitalOcean App Platform hosts the app, with keys stored as encrypted secrets.
  • HyperFrames turns any story into a vertical “building trailer” video.

Built like a software factory

Spec first (PRD.md with a decision log), then acceptance tests on a golden set of 5 landmarks, then small single-purpose modules. Design tokens are shared by the app and the trailer. If research fails, the guide says so instead of inventing history.

Design

Original ink-and-watercolor SVG street and tour guide, no framework. Dark mode turns the street into a foggy night with lit windows and stars.

What’s next

Multi-stop walking tours, more cities, and saved favorites.

Built with

gemininode.jsjavascripthtml/cssgemini-apidigitaloceanwikipedia api Bright DataSentryopenstreetmap

AI tools

Coded withClaude CodeClaude Sonnet 5.5

Prize Categories Entered

  • Gemini Prize Category Q1 2025 - Q1 2026

    Best Use of Gemini API

    Submitted

    Gemini 4 Hydrogen identifies the landmark from the photo plus GPS, extracts 5-8 claims from the sources and writes the 60-second story. Gemini 3.8 TTS (voice Callirrhoe) is the tour guide’s voice.

  • DigitalOcean

    Best Use of DigitalOcean

    Submitted

    The app is hosted on DigitalOcean App Platform, built from our GitHub repo, with all API keys stored as encrypted app secrets.

  • Sentry

    Sentry Raffle

    Submitted

    Sentry tracks errors and traces on the Node server; failed Gemini/Bright Data calls are captured while visitors see a friendly message.

  • Bright Data

    Best Use of Bright Data

    Submitted

    SERP API finds each landmark’s history pages; Web Unlocker fetches them as clean markdown for Gemini to fact-check.

  • HyperFrames

    Best Demo Video Using HyperFrames

    Submitted

    Our demo video was made in HyperFrames: real phone footage, animated captions, the illustrated guide, music and sound effects. The app also exports a HyperFrames “building trailer”.

  • Event prize

    Best Overall Hack

    Submitted

    It turns any SF building into a fact-checked story you hear on the spot. A hand-drawn guide reads it aloud, every claim is taped Verified, Disputed or Legend, and “Next stop” walks you to the next landmark. Built and deployed in one day.

  • Event prize

    Best Use of System One Models

    Submitted

    Jev makes the app’s judgment calls. It labels every claim Verified, Disputed or Legend using only the sources, so the app never states myths as fact. It also picks the landmark in your photo and filters out restaurants and offices from nearby stops.

  • Event prize

    Cleanest Use of a Software Factory

    Submitted

    A clean pipeline: a spec with a decision log, acceptance tests on 5 real landmarks, small single-purpose modules and one shared design-token file. No slop: if research fails, the guide says so instead of inventing history.