NyandSign — Hand Sign Media Controller

Detect hand signs and control media playback

What is NyandSign? NyandSign turns your webcam into a hands-free media controller. Using Google MediaPipe's on-device hand tracking AI, it detects hand signs in real time and maps them to media playback actions — all processed locally in your browser with zero data sent to any server. Open the side panel, start your camera, and control YouTube, Spotify, Netflix, or any browser media with intuitive hand gestures. Any regular webcam works, and supported XREAL / Nreal camera devices can also be used. KEY FEATURES 🖐️ 11 Recognizable Hand Signs Fist, Peace, Three, Four, OK, Aloha (shaka), Rock, Point Right, Point Left, Thumbs Up, and Thumbs Down — each mappable to a media action of your choice. 🎬 Full Media Control ▶⏸ Play / Pause / Toggle 🔊🔉 Volume Up / Down 🔇 Mute ⏩⏪ Seek Forward / Backward (10 seconds) ⏭⏮ Next / Previous Track ⏫⏬ Playback Speed Up / Down 🔄 Reset Playback Speed (1x) ⚡ Fully Customizable Remap any gesture to any action Adjustable gesture hold time (prevents accidental triggers) Repeatable actions (hold a sign to keep adjusting volume, seek, or playback speed) Wake gesture mode — require an "activation" sign before commands are accepted Toggle gesture — enable/disable control with a two-hand sign Built-in tutorial — learn the basics in 6 guided steps 📌 PiP (Picture-in-Picture) Mode When fullscreen video makes the side panel intrusive, switch to a compact floating popup window. Hand gesture recognition continues seamlessly in the small overlay without blocking your content. 🎯 Target Tab Control Direct media commands to a specific tab — control background music while browsing other pages, and pin the target tab when needed. 🔒 100% Local & Private All hand tracking runs on-device using WebAssembly. Your camera feed never leaves your browser. No cloud processing, no data collection, no accounts required. 🌐 Bilingual Full English and Japanese (日本語) interface support. BENEFITS OF INSTALLING NyandSign Hands-free convenience — Control media while cooking, exercising, eating, or when your hands are occupied. Accessibility — Provides an alternative input method for users who have difficulty using a keyboard or mouse. Smart glasses camera support — Designed to work with supported XREAL and Nreal camera devices, enabling gesture control while wearing your glasses. Distraction-free viewing — PiP mode lets you enjoy fullscreen content without a visible side panel while maintaining gesture control. Zero learning curve — Intuitive signs make the controls easy to understand right away. No additional software — Everything runs as a lightweight Chrome extension. No desktop apps, drivers, or accounts needed. WHY THESE PERMISSIONS ARE NEEDED NyandSign requests the minimum permissions required to function. Here is exactly what each permission does and why it is necessary: activeTab Allows the extension to interact with the currently active tab when you click the extension icon. This is needed to inject the media control script into the page you are viewing, so that play/pause/volume/seek/playback-speed commands can reach the video or audio player on that page. Without this permission, the extension cannot control any media. tabs Allows the extension to read tab titles and detect which tabs contain media. This is used for the "Target Tab" feature — it lets you see a list of open tabs and choose or pin which one receives your gesture commands. Tab information is only used locally to display tab names in the side panel; it is never transmitted anywhere. scripting Allows the extension to inject a small content script into web pages. This script is the bridge between the extension and the media player on the page — it listens for gesture commands (play, pause, volume, seek, playback speed) and executes them on the page's video or audio elements. Without this permission, gestures would be detected but could not actually control any media. storage Allows the extension to save your settings (gesture mappings, volume preferences, display options) in Chrome's built-in storage. Your configuration persists across browser restarts. If Chrome Sync is enabled, Chrome itself may sync this settings data across your devices. NyandSign does not send it to its own external servers. sidePanel Allows the extension to display its interface as a Chrome side panel. The side panel is the primary UI where you see the camera feed, hand tracking visualization, gesture status, settings, and tutorial. Host permission: all URLs The content script needs to run on any page because media can be played on any website — YouTube, Spotify, Netflix, Twitch, SoundCloud, podcast sites, and countless others. Restricting to specific domains would break media control on unlisted sites. The content script only interacts with media elements and does nothing else on the page. TECHNICAL DETAILS Uses Google MediaPipe Hand Landmarker (on-device, WebAssembly) Runs at configurable FPS (default 15 fps) to balance accuracy and CPU usage Distance-based gesture recognition with palm-size normalization Temporal majority voting (200ms window) + hysteresis for stable detection All processing happens locally — camera feed is never transmitted Manifest V3 compliant WHAT'S NEW SINCE v1.0.37 v1.0.57 - Replaced the fullscreen action with playback speed reset v1.0.56 - Added playback speed up / down actions v1.0.55 - Improved PiP mode with clearer wake-state feedback v1.0.54 - Added Rock, Three, and Four signs, expanding recognition to 11 hand signs v1.0.44-v1.0.53 - Added a first-run tutorial - Added target-tab pinning - Improved stability by consolidating shared settings and constants v1.0.38-v1.0.43 - Expanded English/Japanese UI coverage - Improved multilingual smart-glasses labeling - Refined default settings and reset behavior SUPPORT If you encounter any issues or have feature requests, please visit our GitHub repository.