NyandSign — Hand Sign Control

Detect hand signs and control media or browser actions

What is NyandSign? NyandSign is a Chrome extension that detects hand signs with your webcam and lets you control media playback and browser actions hands-free. It uses Google MediaPipe's on-device AI to recognize hand signs in real time. All processing happens locally in your browser, and your camera feed is never sent to any external server. Use hand signs to control media on sites like YouTube, Spotify, and Netflix, or switch to Browser Control mode for scrolling, back/forward navigation, tab switching, and zoom. Any regular webcam works, and supported electric glasses camera devices can also be used. KEY FEATURES 🖐️ 12 Recognizable Hand Signs Fist, Peace, Three, Four, OK, Aloha (shaka), Rock, Point Right, Point Left, Point Up, Thumbs Up, and Thumbs Down — each sign can be mapped separately for each operation mode. 🎬 Media Control Mode ▶⏸ Play / Pause 🔊🔉 Volume Up / Down 🔇 Mute ⏩⏪ Seek Forward / Backward (10 seconds) ⏭⏮ Next / Previous Track ⏫⏬ Playback Speed Up / Down 1× Reset Playback Speed 🌐 Browser Control Mode ↕ Hold to Scroll — hold the assigned sign and move your hand up, down, left, or right to scroll ↟↡ Page Top / Page Bottom ↩↪ Back / Forward ⇥⇤ Next Tab / Previous Tab ⇡⇣⇠⇢ Cursor Up / Down / Left / Right ↻ Reload ＋− Zoom In / Zoom Out 100% Reset Zoom 🟡 Pointer Control Mode Display a virtual cursor inside the page, hold an assigned hand sign, and move your hand to move the cursor. Use a separate click sign to click links, buttons, and other page elements. This is implemented as an in-page virtual cursor within what a Chrome extension can control; it does not move the operating system mouse cursor itself. ⚡ Fully Customizable Switch between Media Control, Browser Control, and Pointer Control modes Customize hand sign mappings separately for each mode Adjustable gesture hold time to reduce accidental triggers Repeatable actions for continuous volume, seek, scroll, and zoom control Wake sign mode — require an activation sign before commands are accepted Wake sign options — Open Hand (either side), Open Hand (palm only), or always active Two-hand signs — toggle operation ON/OFF or switch operation modes Built-in tutorial — explains the basic flow and core operation modes 📌 PiP (Picture-in-Picture) Mode When fullscreen video makes the side panel intrusive, switch to a compact popup window. In PiP, Media Control mode uses a green border, Browser Control mode uses a blue border, and Pointer Control mode uses an amber border, so you can tell which mode is active at a glance. 🎯 Target Tab Control Send gesture actions to a specific tab. Pin a target tab to control background media or a specific page while browsing elsewhere. 🔒 100% Local & Private All hand tracking runs on-device using WebAssembly. Your camera feed never leaves your browser. No cloud processing, no data collection, no accounts required. 🌐 Bilingual English and Japanese (日本語) interface support. BENEFITS OF INSTALLING NyandSign Hands-free convenience — Control media and pages while cooking, exercising, eating, or when your hands are occupied. Accessibility — Provides an alternative input method for users who have difficulty using a keyboard or mouse. Electric glasses camera support — Designed to work with supported XREAL / Nreal camera devices, enabling gesture control while wearing your glasses. Distraction-free viewing — PiP mode lets you enjoy fullscreen content without a visible side panel while keeping hand sign control active. Flexible controls — Configure media and browser actions separately, and map your most-used actions to the signs that feel natural to you. No additional software — Everything runs as a lightweight Chrome extension. No desktop apps, drivers, or accounts needed. WHY THESE PERMISSIONS ARE NEEDED NyandSign requests the minimum permissions required to function. Here is exactly what each permission does and why it is necessary: activeTab Allows the extension to interact with the currently active tab when you click the extension icon. This is needed to inject the content script into the page you are viewing, so media actions and browser-page actions such as scrolling can reach the page. tabs Allows the extension to read tab titles and manage tab-related actions. This is used for the Target Tab feature and for actions such as Next Tab and Previous Tab. Tab information is only used locally in the side panel and is never transmitted anywhere. scripting Allows the extension to inject a small content script into web pages. This script is the bridge between NyandSign and the page: it receives gesture commands and applies them to media elements or page scrolling. Without this permission, gestures could be detected but could not control the page. storage Allows the extension to save your settings, including hand sign mappings, operation mode, volume preferences, and display options, in Chrome's built-in storage. Your configuration persists across browser restarts. If Chrome Sync is enabled, Chrome itself may sync this settings data across your devices. NyandSign does not send it to its own external servers. sidePanel Allows the extension to display its interface as a Chrome side panel. The side panel is the primary UI where you see the camera feed, hand tracking visualization, current sign, operation mode, settings, and tutorial. Host permission: all URLs The content script needs to run on any page because media and scrollable content can appear on any website — YouTube, Spotify, Netflix, Twitch, SoundCloud, news sites, documentation sites, and countless others. Restricting access to specific domains would break control on unlisted sites. The content script only handles the page interactions required by gesture commands and does not send camera data or personal information anywhere. TECHNICAL DETAILS Uses Google MediaPipe Hand Landmarker (on-device, WebAssembly) Runs at configurable FPS (default 15 fps) to balance accuracy and CPU usage Distance-based gesture recognition with palm-size normalization Temporal majority voting (200ms window) + hysteresis for stable detection All processing happens locally — camera feed is never transmitted Manifest V3 compliant WHAT'S NEW v1.1.118 - Promoted Pointer Control mode to a standard mode alongside Media Control and Browser Control - Added the Point Up hand sign, with pointer movement assigned to Point Up and click assigned to OK by default - Improved pointer movement responsiveness, reduced target-highlight load, and stabilized iframe target display and clicking - Added settings for inference resolution, idle FPS limiting, Hold to Scroll speed, and pointer movement speed - Reorganized settings into Operation, Display, and Recognition tabs, with reset confirmation and two-hand sign reset - Improved hand sign recognition rules and debug display v1.0.91 - Added experimental Pointer Control mode, which lets you move an in-page virtual cursor with hand signs - Added click support and edge scrolling while using Pointer Control mode - Improved the visual feedback while pointer movement is active - Improved cleanup so the virtual cursor and camera stream are less likely to remain after stopping the camera or closing the extension - Improved recovery when your hand briefly leaves the camera view during Hold to Scroll or pointer movement v1.0.81 - Improved stability during Hold to Scroll when hand sign recognition briefly fluctuates - Prevented another sign from triggering accidentally when ending Hold to Scroll or repeatable actions - Improved scroll target detection on Google sites and similar pages - Shows a blocked-page indicator when page actions cannot run because script injection is unavailable v1.0.75 - Added Browser Control mode - Added browser actions such as scroll, back/forward, tab switching, cursor movement, reload, and zoom - Improved compatibility for cursor movement actions - Added separate hand sign mappings for Media Control and Browser Control modes - Added two-hand sign actions for operation ON/OFF and mode switching - Updated the tutorial and PiP visual feedback for the new two-mode workflow v1.0.61 - Improved OK sign recognition, especially for angled hands and landmark estimation variance v1.0.58-v1.0.60 - Added an Open Hand (palm only) wake gesture option - Refined Open Hand detection to reduce false positives against the OK sign v1.0.57 - Replaced the fullscreen action with playback speed reset v1.0.56 - Added playback speed up / down actions v1.0.55 - Improved PiP mode with clearer wake-state feedback v1.0.54 - Added Rock, Three, and Four signs, expanding recognition to 11 hand signs v1.0.44-v1.0.53 - Added a first-run tutorial - Added target-tab pinning - Improved stability by consolidating shared settings and constants v1.0.38-v1.0.43 - Expanded English/Japanese UI coverage - Improved multilingual smart-glasses labeling - Refined default settings and reset behavior SUPPORT If you encounter any issues or have feature requests, please visit our GitHub repository.