diff --git a/.gitignore b/.gitignore index 67e450a..c92cd12 100644 --- a/.gitignore +++ b/.gitignore @@ -34,6 +34,7 @@ logs/ .cache/ .tmp/ .vercel +.claude/ # Backend runtime cache (downloaded MIDI files) server/cache/ diff --git a/claude.md b/CLAUDE.md similarity index 100% rename from claude.md rename to CLAUDE.md diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md deleted file mode 100644 index f057411..0000000 --- a/DEPLOYMENT.md +++ /dev/null @@ -1,102 +0,0 @@ -# Deployment Guide for Motif - -## iOS Fixes Applied -- ✓ Fixed auto-zoom on input focus (font-size: 16px, viewport locked) -- ✓ Prevented zoom on touch (maximum-scale=1.0, user-scalable=no) - -## Backend Deployment - -The MIDI search requires a backend server to be running. You have two options: - -### Option 1: Deploy Backend Separately (Recommended for Production) - -1. Deploy the backend to a service like Render, Railway, or Heroku: - ```bash - cd server - npm install - npm run build - npm start - ``` - -2. Set the `VITE_API_URL` environment variable in your frontend deployment to point to your backend: - ``` - VITE_API_URL=https://your-backend-url.com - ``` - -### Option 2: Run Backend Locally (Development Only) - -1. In one terminal, start the backend: - ```bash - npm run dev:backend - ``` - -2. In another terminal, start the frontend: - ```bash - npm run dev - ``` - -## Vercel Deployment - -### Deploy Frontend to Vercel: - -1. Build the project: - ```bash - npm run build - ``` - -2. Deploy to Vercel: - ```bash - vercel --prod - ``` - -3. Set environment variable in Vercel dashboard: - - Variable name: `VITE_API_URL` - - Value: Your deployed backend URL (e.g., `https://motif-backend.onrender.com`) - -### Deploy Backend to Render/Railway: - -1. Create a new Web Service -2. Connect your GitHub repository -3. Set build command: `cd server && npm install && npm run build` -4. Set start command: `cd server && npm start` -5. Add environment variable `PORT` (usually auto-set) -6. Deploy - -## Environment Variables - -Create a `.env` file in the root directory for local development: - -```bash -VITE_API_URL=http://localhost:3001 -``` - -For production, set this in your deployment platform (Vercel, Netlify, etc.): - -```bash -VITE_API_URL=https://your-backend-api-url.com -``` - -## Testing the Deployment - -1. Open the deployed URL on iOS Safari -2. Search for a song (e.g., "Hotel California") -3. You should see MIDI results load -4. Select a result and play it -5. The page should NOT zoom when touching the search input - -## Troubleshooting - -### No search results on production: -- Check that backend is running (visit `https://your-backend-url.com/health`) -- Verify `VITE_API_URL` environment variable is set correctly in Vercel -- Check browser console for CORS errors -- Rebuild frontend after setting environment variables - -### iOS zoom issue persists: -- Clear Safari cache -- Hard reload the page -- Check that the latest build is deployed - -### CORS errors: -- Backend must allow requests from your frontend domain -- Check server CORS configuration in `server/src/server.ts` diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md deleted file mode 100644 index fa4225a..0000000 --- a/IMPLEMENTATION.md +++ /dev/null @@ -1,251 +0,0 @@ -# MOTIF Implementation Status & Roadmap - -**Current Status**: MVP functional with real MIDI search, parsing, and procedural synthesis - ---- - -## ✅ What's Been Built - -### Backend API (Express + TypeScript) - -**Endpoints:** -- `GET /api/midi/search?q=song` - Multi-source MIDI search -- `GET /api/midi/fetch?u=url` - CORS proxy with validation and caching -- `GET /health` - Service health check - -**Architecture:** -``` -server/src/ -├── adapters/ # Search source implementations -│ ├── BitMidiAdapter # HTML parsing for bitmidi.com -│ └── DongraysAdapter # HTML parsing for dongrays.net -├── services/ -│ ├── MIDISearchService # Orchestrates multi-source search -│ └── MIDIFetchService # Downloads, validates, caches MIDI -└── utils/ - └── ScoreUtils # Confidence scoring & quality assessment -``` - -**Key Features:** -- **Confidence Scoring**: Token matching, quality penalties (karaoke, broken files) -- **Disk Caching**: SHA256-hashed files with JSON index -- **Validation**: MIDI header checks, file size limits (10MB max) -- **Error Handling**: Timeouts, graceful fallbacks -- **Deduplication**: Removes duplicate results across sources - -### Frontend (TypeScript + Vite + Web Audio) - -**Architecture:** -``` -src/ -├── core/ -│ ├── MotifEngine # Main orchestrator -│ └── RoleMapper # MIDI track → synthesis role assignment -├── midi/ -│ ├── MIDIProcessor # Feature extraction (tempo, density, etc.) -│ └── MIDIParser # @tonejs/midi wrapper -├── synthesis/ -│ └── SynthesisEngine # Pure Web Audio procedural synthesis -├── services/ -│ └── MIDIService # Backend API client -└── types/ - └── index # TypeScript interfaces -``` - -**Synthesis Pipeline:** -``` -Song Name → MIDI Search → Parse Events → Role Assignment → Web Audio Synthesis -``` - -### Integration Flow - -1. **User enters song name** → `MotifEngine.generateFromSong()` -2. **MIDI Search** → `MIDIService.search()` → Backend `/search` endpoint -3. **Multi-source search** → BitMidi + Dongrays adapters in parallel -4. **Result ranking** → Confidence scoring, deduplication -5. **MIDI Fetch** → `MIDIService.fetchMIDI()` → Backend `/fetch` with caching -6. **MIDI Parsing** → `@tonejs/midi` → Normalized `NoteEvent[]` array -7. **Role Assignment** → `RoleMapper` → Bass/Drone/Ostinato/Texture/Accents -8. **Web Audio Synthesis** → `SynthesisEngine` → Real-time procedural audio - ---- - -## 🔧 How It Actually Works - -### MIDI Resolution Strategy - -**Sources (MVP):** -- **BitMidi**: Regex parsing of search results, direct `.mid` links -- **Dongrays**: Similar approach, handles download endpoints -- **Synthetic Fallback**: Hash-based procedural generation if search fails - -**Scoring Heuristics:** -- Token matching between query and title -- Penalties for "karaoke", "vocal", "broken" -- Bonus for direct `.mid` links -- Source preference (BitMidi slightly favored) - -### Role-Based Synthesis - -**Role Assignment:** -```typescript -// Heuristic rules: -pitch < 48 + short notes = Bass -long duration > 2s = Drone -short + repetitive = Ostinato -high velocity = Accents -everything else = Texture -``` - -**Synthesis Per Role:** -- **Bass**: Square wave, lowpass filter, punchy envelopes -- **Drone**: Sawtooth, bandpass, sustained notes -- **Ostinato**: Triangle, highpass, rhythmic patterns -- **Texture**: Sine, bandpass, atmospheric -- **Accents**: Sine, peaking filter, sharp attacks - -**Web Audio Implementation:** -- Lookahead scheduling (100ms) -- MIDI note → Hz conversion: `440 * 2^((note-69)/12)` -- Velocity-sensitive envelopes -- Automatic looping when MIDI ends -- Per-note oscillator + gain envelope - ---- - -## 🚨 Current Limitations - -### Search Quality -- **HTML Parsing**: Fragile regex-based extraction (not DOM parsing) -- **Limited Sources**: Only 2 sources, no fallbacks if both fail -- **No Metadata**: Can't validate artist, album, year matching -- **Rate Limiting**: No request throttling or backoff - -### MIDI Processing -- **Simple Role Mapping**: Basic pitch/duration heuristics only -- **No Harmonic Analysis**: Doesn't understand chord progressions -- **Track Correlation**: Doesn't detect melody vs accompaniment intelligently -- **Tempo Handling**: Assumes constant tempo, ignores tempo changes - -### Synthesis Engine -- **Basic Timbres**: Simple oscillator types, no complex synthesis -- **No Dynamics**: Volume levels are role-based, not musically aware -- **Limited Effects**: Only basic filtering, no reverb/chorus/etc. -- **Monophonic Layers**: Each role plays one note at a time - -### Frontend UX -- **No Progress Feedback**: Search/fetch happens in black box -- **No Result Preview**: Can't see what MIDI was found before synthesis -- **No Controls**: Can't adjust synthesis parameters -- **Error Messages**: Generic error handling - ---- - -## 🎯 Next Steps (Prioritized) - -### Phase 1: Polish MVP -**Goal**: Make current system reliable and user-friendly - -1. **Better Error Handling** - - Show search progress ("Searching BitMidi...", "Parsing MIDI...") - - Display actual MIDI file found before synthesis - - Graceful degradation with informative messages - -2. **Improve Role Mapping** - - Add harmonic analysis (detect bass lines, chord patterns) - - Use track names/MIDI program changes as hints - - Smarter melody vs accompaniment detection - -3. **Synthesis Polish** - - Add polyphony within roles (chords, multiple bass notes) - - Better envelopes (ADSR with release tails) - - Basic effects (simple reverb, subtle filtering LFOs) - -### Phase 2: Search Enhancement -**Goal**: Higher success rate finding good MIDIs - -4. **Robust Parsing** - - Switch to Cheerio for proper DOM parsing - - Handle dynamic content/JavaScript-loaded results - - Add more MIDI sources (MuseScore, IMSLP public domain) - -5. **Smarter Scoring** - - Artist name matching with fuzzy string comparison - - Duration validation (reject 30-second clips, 20-minute symphonies) - - Key signature and time signature analysis - -6. **Caching & Performance** - - Cache search results (not just MIDI files) - - Add request deduplication and rate limiting - - Background refresh of popular files - -### Phase 3: Synthesis Sophistication -**Goal**: More recognizable and musical output - -7. **Advanced Synthesis** - - Multiple synthesis modes per role (subtractive, FM, additive) - - Tempo-synced effects and modulation - - Cross-role interaction (bass and drums lock together) - -8. **Musical Intelligence** - - Detect and preserve harmonic progressions - - Rhythmic pattern extraction and variation - - Dynamic arrangement (intro/verse/chorus detection) - -9. **User Controls** - - Synthesis parameter sliders (brightness, warmth, density) - - Role muting/soloing - - Tempo adjustment and time-stretching - -### Phase 4: Production Ready -**Goal**: Reliable service for real users - -10. **Infrastructure** - - Database for MIDI metadata and search caching - - CDN for popular MIDI files - - Analytics and error monitoring - -11. **Legal & Content** - - MIDI license validation - - User-uploaded MIDI support - - Integration with Creative Commons sources - ---- - -## 🔬 Technical Debt - -### Immediate -- Remove `crypto` dependency warning in backend package.json -- Add proper TypeScript strict mode compliance -- Implement proper error boundaries in frontend - -### Medium Term -- Replace regex HTML parsing with proper DOM parsing -- Add comprehensive logging/telemetry -- Write unit tests for core algorithms (role mapping, scoring) - -### Long Term -- Consider WebAssembly for intensive audio processing -- Evaluate Web Workers for MIDI parsing/analysis -- Implement WebRTC for real-time collaboration features - ---- - -## 📊 Success Metrics - -**Current State**: -- ✅ Searches return results ~70% of time -- ✅ Successfully parses most MIDI files found -- ✅ Generates audio output 100% of time (with fallback) -- ⚠️ Output recognizably similar to input ~30% of time - -**Target State**: -- 🎯 Search success rate >90% -- 🎯 Musical similarity recognition >70% -- 🎯 User "that sounds like the song" reaction >60% -- 🎯 Sub-3-second generation time 95% of requests - ---- - -**Built**: Functional end-to-end MVP with real MIDI integration -**Next**: Polish the core experience before expanding features \ No newline at end of file diff --git a/MIDI-playback-integration-plan.md b/MIDI-playback-integration-plan.md deleted file mode 100644 index 8c4aefd..0000000 --- a/MIDI-playback-integration-plan.md +++ /dev/null @@ -1,163 +0,0 @@ -## MIDI playback MVP — detailed technical integration plan - -### Product target (MVP) -- **User flow**: - 1) User types a song name (e.g. “Hotel California”) - 2) App searches MIDI sources and shows a ranked list - 3) User selects a result and can **play the MIDI “correctly”** using **General MIDI soundfonts** - 4) User can then click **Generate Motif** to synthesize a “similar-but-different” version from the same parsed MIDI - -### Current code reality (gaps to close) -- **Search is currently biased toward mock/synthetic**, not real MIDI: - - `server/src/services/MIDISearchService.ts` includes `MockAdapter` first. -- **Fetch can silently replace real URLs with synthetic MIDI**: - - `server/src/services/MIDIFetchService.ts` generates synthetic MIDI when URL contains `bitmidi.com/uploads`, preventing true BitMidi playback. -- **Frontend preview is oscillator-based**, not GM soundfont playback. - ---- - -## Architecture (what we’ll ship) - -```mermaid -flowchart LR -User -->|typesQuery| FrontendUI -FrontendUI -->|GET /api/midi/search?q=...| BackendSearch -BackendSearch -->|rankedResults| FrontendUI -FrontendUI -->|selectResult + GET /api/midi/fetch?u=...| BackendFetch -BackendFetch -->|midiBytes| FrontendParse -FrontendParse -->|NoteEvents + TrackMeta| PreviewPlayerGM -FrontendParse -->|NoteEvents| MotifEngine -MotifEngine -->|roles + chords| SynthesisEngine -PreviewPlayerGM --> AudioOut -SynthesisEngine --> AudioOut -``` - ---- - -## Phase 0 — “Real MIDI mode” defaults (backend hardening) - -### 0.1 Gate mock adapter behind env flag -- **Change**: In `server/src/services/MIDISearchService.ts`, make adapters: - - Default: `[BitMidiAdapter, DongraysAdapter]` - - Optional: prepend `MockAdapter` only if `USE_MOCK_ADAPTER=1` (or similar) -- **Acceptance**: - - Searching “Hotel California” returns **non-`synthetic:*`** results when internet is available. - - Devs can still run offline with `USE_MOCK_ADAPTER=1`. - -### 0.2 Stop auto-synth overriding real URLs in fetch -- **Change**: In `server/src/services/MIDIFetchService.ts`, only synthesize when: - - `url.startsWith('synthetic:')` (and optionally if `USE_SYNTHETIC_FETCH=1`) -- **Remove**: `url.includes('bitmidi.com/uploads')` synthetic shortcut -- **Acceptance**: - - Selecting a BitMidi result triggers a real network fetch and caches the real bytes. - - If the remote file is invalid, it fails explicitly with a clear error. - -### 0.3 Make backend behavior explicit in responses (optional but recommended) -- **Change**: Add response fields or headers indicating source: - - Example: `X-Motif-Source: real|synthetic|cache` -- **Acceptance**: - - Frontend can display “cached”/“live”/“synthetic fallback” badges (helps debugging + trust). - ---- - -## Phase 1 — GM soundfont playback (frontend “Preview” becomes correct MIDI playback) - -### 1.1 Dependency choice & asset strategy -- **Dependency**: add a browser-friendly soundfont player dependency (e.g. `soundfont-player`). -- **Soundfont hosting**: - - Prefer static hosting under `/public/soundfonts/` (versioned with the app) - - Or use a CDN, but pin versions and handle CORS -- **MVP instrument set**: - - Minimum viable: **Acoustic Grand Piano** for all melodic tracks, and a basic drum fallback - - Better: load instruments on demand per track program - -### 1.2 Implement `SoundfontMIDIPlayer` -Create `src/synthesis/SoundfontMIDIPlayer.ts` with: -- **Responsibilities** - - Load instruments (program → soundfont instrument name) - - Schedule note-on/note-off with WebAudio timing - - Provide `load(midi)` / `play()` / `stop()` / `setVolume()` APIs -- **Inputs** - - Best: use `@tonejs/midi`’s `Midi` object (tracks include `instrument.number`, `notes`, `channel`) - - Alternate: keep using `NoteEvent[]`, but you’ll lose program/channel unless you extend the event model -- **Timing correctness** - - Use seconds-based timing from `@tonejs/midi` notes (`time`, `duration` are in seconds) - - Ensure AudioContext resumes on user gesture -- **Drums** - - If channel 9/10 is detected: either map to a percussion kit if supported, or skip drums for MVP (but be explicit in UI) - -### 1.3 Wire UI to use soundfont preview -- In `src/main.ts`, replace/augment the current oscillator `MIDIPlayer` usage: - - Preview buttons should play via `SoundfontMIDIPlayer` - - Keep existing Motif buttons intact -- Keep oscillator preview only as a fallback if soundfonts fail to load (optional). - ---- - -## Phase 2 — “Play the real song correctly” UX loop - -### 2.1 Search UX requirements -- **Result list must show**: - - title, source, confidence - - parsed metadata: duration, track count, issues -- **Selection behavior**: - - Selecting a row fetches + parses once; enables Preview + Generate Motif - -### 2.2 Error handling (user-facing) -Define user-facing error classes/messages: -- **Search**: “No results”, “Backend unavailable”, “Rate limited / source blocked” -- **Fetch**: “MIDI file blocked”, “Invalid MIDI header”, “Quality rejected” -- **Parse**: “Unsupported MIDI features” / “Parse failed” -- **Preview playback**: “Soundfont failed to load” / “Audio not allowed until click” - -### 2.3 Observability (dev-facing) -- Backend: log adapter failures per source + timings -- Frontend: log selected MIDI URL, parse duration, instrument load times -- Optional: a small “Debug” accordion showing chosen URL, cache hit, parse issues, loaded instruments - ---- - -## Phase 3 — Motif generation stays step 2 (but align data model) -- Keep the current path: - - `MotifEngine.generateFromMIDI(NoteEvent[])` then `MotifEngine.play()` -- Recommended alignment work: - - Decide whether Motif should later consume richer track metadata (program/channel) to improve role mapping. - ---- - -## Integration milestones & acceptance checks - -### Milestone A — Real MIDI end-to-end -- Search returns results from BitMidi/Dongrays with mock disabled by default -- Fetch returns real bytes and caches them -- Parse endpoint `/api/midi/parse` works for metadata - -### Milestone B — “Correct” preview playback -- Preview produces recognizable instrument playback (piano at minimum) -- Stop reliably stops scheduled notes -- Works in Chrome/Safari with autoplay policies (requires click) - -### Milestone C — Motif as second step -- Generate Motif still works on the same loaded MIDI -- Preview and Motif can be A/B tested without reloading the page - ---- - -## Recommended team task breakdown - -### Backend engineer -- Implement env gating for mock/synthetic -- Tighten fetch behavior + ensure BitMidi URLs are truly fetched -- Add explicit “source = real/cache/synthetic” marker - -### Frontend engineer -- Add the chosen soundfont library -- Implement `SoundfontMIDIPlayer` -- Wire `src/main.ts` preview buttons to soundfont playback -- Add user-facing error messaging for soundfont failures - -### QA / test harness -- Maintain a short list of known-good queries (3–5 songs) and confirm: - - search results appear - - at least one MIDI fetches and plays - - Motif plays afterward diff --git a/docs/home-ux-audit.md b/docs/home-ux-audit.md deleted file mode 100644 index f1bcfb5..0000000 --- a/docs/home-ux-audit.md +++ /dev/null @@ -1,92 +0,0 @@ -# Home UX audit + improvements (arcade-first) - -This doc is a saved copy of the current home UX audit plan (no code changes). - -## Current UX issues (what’s making it feel uninspiring/confusing) -- **Unclear hierarchy**: “Search”, “Preview”, and “Synthesis Engine” compete; users don’t instantly see the main thing to do. -- **Weak step-by-step guidance**: The app is a 3-step flow (Search → Select → Run engine), but the UI doesn’t feel like a guided sequence. -- **Controls feel detached from state**: Buttons don’t always read as “locked until selection”; the reason for disabled state isn’t visible. -- **Results table is dense** on mobile: long titles wrap unpredictably, selection highlight is subtle. -- **Copy and concept drift**: “emulator/engine” and “MIDI scrape/analyse” copy varies; the story should be consistent and confident. -- **iOS audio state**: The “Enable Audio” banner is reactive but not integrated into the flow (and can appear “random”). - -## Target outcome -Arcade cabinet vibe with a clear, dramatic main CTA: -- **Primary action**: “Run the engine” (Generate & Play) -- Secondary: preview the MIDI -- Stronger “insert coin” style guidance: show step chips and a big engine panel. - -## Proposed UX changes (high impact, low risk) - -### 1) Make the flow explicit -Update `index.html`: -- Add a compact “Steps” row near the top: - - Step 1: Search - - Step 2: Pick a MIDI - - Step 3: Run the engine -- Tie each step’s visual state to app state: - - Step 2/3 show “locked” styling until available. - -### 2) Promote “Synthesis Engine” as the hero module -- Move the engine card visually above Preview (or keep order but make engine card visually dominant). -- Make engine CTA bigger and more arcade: - - Primary button: **Run the engine** - - Secondary: Stop - - Volume + progress remain but visually subordinate. -- Add a small one-line “what this does” under the button, not as a paragraph. - -### 3) Preview becomes clearly “optional” -- Rename to “Listen to the source MIDI (optional)” -- Collapse preview by default on mobile (or add a “Show preview” toggle) to reduce overwhelm. - -### 4) Results list readability + selection confidence -- Increase row tap targets and selection contrast. -- Add a right-side “Selected” chip on the selected row. -- On mobile: - - clamp long titles to 2 lines - - reduce columns (hide Source or Duration) based on width. - -### 5) iOS audio UX integrated into the engine (subtle) -- Only on iOS-like browsers and only when audio is locked. -- Show a small under-section link: “Having trouble on iOS? Enable audio” -- Clicking expands a compact CTA row (Enable Audio button + tiny state line). -- Avoid a big persistent banner that could confuse non-iOS users. - -### 6) Copy pass -Deferred for now: keep copy changes out of this pass. - -## Implementation plan (concrete) - -### Files -- Primary: `index.html` -- Minor TS adjustments (state-driven CSS classes): `src/main.ts` - -### Approach -- Add a small set of **state CSS classes** on `body` (or on `.container`): - - `state-has-results` - - `state-has-selection` - - `state-audio-locked` -- In `src/main.ts`, toggle these classes when: - - results are loaded - - a selection is made - - audio unlock state changes -- Use CSS to: - - style locked sections - - highlight the engine CTA - - improve mobile responsiveness - -## Test plan -- Desktop Chrome/Safari: - - search → select → engine run - - preview optional still works -- iOS Safari: - - first run shows “arm audio” only when needed - - no random banner when audio is already running -- Visual: - - mobile widths (320–430px) - - long titles wrapping - -## Non-goals -- No changes to the core synthesis engine behavior or backend. -- No copywriting/wording changes in this pass. - diff --git a/docs/sharing-plan.md b/docs/sharing-plan.md deleted file mode 100644 index 2e580b7..0000000 --- a/docs/sharing-plan.md +++ /dev/null @@ -1,176 +0,0 @@ -# Sharing plan: links + player landing + Motif export - -This document captures the implementation plan for **Share links** (Copy Link + Share to X), a minimal **player landing page**, and the next step: sharing **generated Motif variants** via **exported artifacts** (MIDI and/or audio). - -## Goals -- **Shareable outcome**: users can share something they heard (original MIDI preview, or a generated Motif variant). -- **One-click-to-hear**: shared links load quickly and are ready to play (but never true autoplay; iOS requires a user gesture). -- **Lightweight**: keep the landing page minimal and stable. -- **Safe**: shared URLs must not turn the backend into an open proxy. - -## MVP constraints (important) -- **iOS/Safari** blocks audio until a user gesture. The landing can preload, but playback needs a tap. -- **X/Twitter** shares are links + preview cards. No reliable in-post WebAudio playback. -- Search results can change over time, so **index-based sharing** is brittle. - -## UX spec - -### A) Main app: Share controls -Placement: near the “Selected MIDI”/result context (where users decide something is share-worthy). - -Controls: -- **Copy link**: copies a permalink to clipboard + shows a small “Copied” toast. -- **Share to X**: opens the tweet composer with prefilled text + URL. - -X intent URL format: -- `https://twitter.com/intent/tweet?text=&url=` - -Suggested default tweet text: -- `Listening to “{title}” in MOTIF — try it` - -### B) Share landing: `/play` -A minimal landing page that: -- reads parameters from the URL -- fetches the MIDI (or exported artifact) -- enables Play/Stop + volume -- shows iOS “Enable Audio” CTA if required -- includes a primary CTA: **Generate your own** → links back to the main app - -## URL design (heart of the plan) - -### v1: share the original MIDI preview (recommended MVP) -Use the MIDI URL as the stable identifier. - -Landing URL: -- `/play?u=&title=&song=` - -Notes: -- `u` is required for reproducibility. -- `title` is display-only (optional). -- `song` supports the CTA back to the main app (optional). - -Example: -- `/play?u=https%3A%2F%2Fbitmidi.com%2Fuploads%2F...mid&title=Hotel%20California&song=Hotel%20California` - -Main app CTA target: -- `/?song=` - -### Why not share “result index”? -Because search ranking shifts; index links rot. If needed, index can be a fallback only when `u` is missing. - -## `/play` landing behavior - -### Required inputs -- **Preview share**: `u` (MIDI source URL) -- **Export share**: `m` (motif midi id) or `a` (audio id) - -### State machine -- If `m` or `a` present → load exported artifact -- Else if `u` present → fetch + parse MIDI via backend proxy -- Else → show “Invalid link” + CTA to home - -### Playback -- Never autoplay. -- Enable Audio CTA shown until AudioContext is running. - -### Error cases -- Missing/invalid params → show “Invalid link” -- Backend fetch failure → show “Couldn’t load this MIDI” -- Parse failure → show “Unsupported or corrupted MIDI” - -## Export-based sharing (generated Motif variants) -Sharing a generated Motif variant needs an artifact the landing page can load reliably: -- **MIDI export**: smaller, fast, consistent with the project (music-as-structure) -- **Audio export**: most universal listening, but heavier and needs careful encoding/hosting - -### Recommended sequencing -1) **MIDI export first** (fast, small, easy to iterate)\n2) Add **audio export** once the flow proves value (and storage/limits are solved) - -### Proposed backend endpoints - -#### 1) Create export (returns a shareable play URL) -`POST /api/share/export` - -Input (example): -```json -{ - "u": "https://bitmidi.com/uploads/....mid", - "song": "Hotel California", - "title": "Hotel California - Eagles", - "preset": "dance|ambient|ominous|default", - "params": { "intensity": 0.5, "swing": 0.1 }, - "format": "midi|audio" -} -``` - -Output (example): -```json -{ - "id": "abc123", - "format": "midi", - "playUrl": "/play?m=abc123&title=Hotel%20California&song=Hotel%20California" -} -``` - -#### 2) Fetch export artifact -Option A (explicit): -- `GET /api/share/artifact?id=abc123` → returns bytes (MIDI or audio) - -Option B (format-specific): -- `GET /api/share/midi?id=abc123` -- `GET /api/share/audio?id=abc123` - -Landing params: -- MIDI artifact: `/play?m=&title=...&song=...` -- Audio artifact: `/play?a=&title=...&song=...` - -### Storage strategy (dev vs production) -- **Local dev**: filesystem under `server/cache/exports/` -- **Vercel production**: filesystem is not durable; prefer: - - blob storage (recommended), or - - KV/object store (if blob not available), or - - a small database row referencing a blob key - -### Security constraints (must-have) -- Validate `u` server-side: - - allow only `http`/`https` - - strongly consider allowlisting hosts (e.g. `bitmidi.com`) for v1 -- Enforce limits: - - max MIDI bytes - - max parsed duration - - max export size - - max export time/CPU -- Rate-limit export endpoints. -- Cache and dedupe exports: - - key by `(sourceMidiHash + preset + params + format)` to avoid repeated work - -## Files (expected) — implementation map -Frontend: -- `index.html`: Share UI elements -- `src/main.ts`: URL generation + clipboard + X intent -- `play.html` + `src/play.ts`: landing page UI + logic -- `vite.config.ts`: add `play.html` to multi-page inputs - -Backend: -- `server/src/server.ts`: route handlers for export endpoints -- `server/src/services/*`: reuse parsing/generation services as needed -- `server/cache/exports/`: local dev artifact cache - -Routing/deploy: -- `vercel.json`: add `/play` → `/play.html` similar to `/embed` - -## Test plan -- Desktop: - - Share from main app → open `/play?...` in new tab → loads → plays after click - - Copy link works; X intent opens with correct URL -- iOS Safari: - - `/play` shows Enable Audio when needed - - Tap Enable Audio → tap Play → sound -- Export flow: - - Export MIDI variant → receive `/play?m=...` → plays the exported MIDI reliably - - Export audio variant (if implemented) → `/play?a=...` → plays reliably - -## Success criteria -- A shared link reliably recreates “the thing” (preview or exported variant) and is playable with one tap. -- The system remains safe under abuse (no open proxy behavior, bounded cost). - diff --git a/public/.gitkeep b/public/.gitkeep deleted file mode 100644 index 1ab5c00..0000000 --- a/public/.gitkeep +++ /dev/null @@ -1 +0,0 @@ -# Save the social preview image as og-image.jpg in this directory diff --git a/public/chiptune_blog_piece_1000x.webp b/public/chiptune_blog_piece_1000x.webp deleted file mode 100644 index 869f356..0000000 Binary files a/public/chiptune_blog_piece_1000x.webp and /dev/null differ diff --git a/src/main-simple.ts b/src/main-simple.ts deleted file mode 100644 index 6dfdc5a..0000000 --- a/src/main-simple.ts +++ /dev/null @@ -1,42 +0,0 @@ -console.log('Main script loading...'); - -document.addEventListener('DOMContentLoaded', function() { - console.log('DOM loaded'); - - const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement; - const songInput = document.getElementById('songInput') as HTMLInputElement; - const status = document.getElementById('status')!; - - if (!searchBtn || !songInput || !status) { - console.error('UI elements not found!'); - return; - } - - console.log('UI elements found'); - - searchBtn.addEventListener('click', async function() { - console.log('Search button clicked!'); - - const songName = songInput.value.trim(); - if (!songName) return; - - status.textContent = 'Searching...'; - searchBtn.disabled = true; - - try { - const response = await fetch(`http://localhost:3001/api/midi/search?q=${encodeURIComponent(songName)}`); - const data = await response.json(); - - console.log('Search results:', data); - status.textContent = `Found ${data.count} results: ${data.results.map((r: any) => r.title).join(', ')}`; - - } catch (error) { - console.error('Search error:', error); - status.textContent = `Search error: ${error}`; - } finally { - searchBtn.disabled = false; - } - }); - - console.log('Event listeners attached'); -}); \ No newline at end of file diff --git a/src/main-working.ts b/src/main-working.ts deleted file mode 100644 index 6c2b3f5..0000000 --- a/src/main-working.ts +++ /dev/null @@ -1,189 +0,0 @@ -console.log('Full Motif app loading...'); - -interface SearchResult { - id: string; - title: string; - source: string; - pageUrl: string; - midiUrl: string; - confidence: number; -} - -class MotifApp { - private searchResults: SearchResult[] = []; - private selectedIndex = 0; - - constructor() { - document.addEventListener('DOMContentLoaded', () => { - this.initializeUI(); - }); - } - - private initializeUI(): void { - console.log('Initializing UI...'); - - const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement; - const songInput = document.getElementById('songInput') as HTMLInputElement; - - if (!searchBtn || !songInput) { - console.error('UI elements not found!'); - return; - } - - searchBtn.addEventListener('click', () => this.handleSearch()); - songInput.addEventListener('keypress', (e) => { - if (e.key === 'Enter') this.handleSearch(); - }); - - // Make selectResult globally available for onclick handlers - (window as any).app = this; - - console.log('UI initialized successfully'); - } - - private async handleSearch(): Promise { - const songInput = document.getElementById('songInput') as HTMLInputElement; - const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement; - const status = document.getElementById('status')!; - - const songName = songInput.value.trim(); - if (!songName) return; - - status.textContent = 'Searching for MIDI files...'; - searchBtn.disabled = true; - - try { - const response = await fetch(`http://localhost:3001/api/midi/search?q=${encodeURIComponent(songName)}`); - const data = await response.json(); - - console.log('Search results:', data); - - if (data.results.length === 0) { - status.textContent = 'No MIDI files found. Try a different search.'; - return; - } - - this.searchResults = data.results; - this.displayResults(); - status.textContent = `Found ${data.results.length} MIDI files. Select one to play.`; - - } catch (error) { - console.error('Search error:', error); - status.textContent = `Search error: ${error}`; - } finally { - searchBtn.disabled = false; - } - } - - private displayResults(): void { - const resultsSection = document.getElementById('resultsSection')!; - const resultsBody = document.getElementById('resultsBody')!; - - resultsBody.innerHTML = ''; - - this.searchResults.forEach((result, index) => { - const row = document.createElement('tr'); - if (index === this.selectedIndex) { - row.classList.add('selected'); - } - - row.innerHTML = ` - ${result.title} - ${result.source} - -
-
-
- - ? - ? - - - `; - - resultsBody.appendChild(row); - }); - - resultsSection.classList.add('visible'); - - // Auto-select first result - if (this.searchResults.length > 0) { - this.selectResult(0); - } - } - - public async selectResult(index: number): Promise { - if (index < 0 || index >= this.searchResults.length) return; - - this.selectedIndex = index; - const result = this.searchResults[index]; - - // Update selection highlighting - const rows = document.querySelectorAll('#resultsBody tr'); - rows.forEach((row, i) => { - row.classList.toggle('selected', i === index); - }); - - const status = document.getElementById('status')!; - const playerSection = document.getElementById('playerSection')!; - const selectedTitle = document.getElementById('selectedTitle')!; - const selectedMeta = document.getElementById('selectedMeta')!; - - status.textContent = 'Loading MIDI file...'; - - try { - // Fetch MIDI data - const response = await fetch(`http://localhost:3001/api/midi/fetch?u=${encodeURIComponent(result.midiUrl)}`); - - if (!response.ok) { - throw new Error(`Failed to fetch MIDI: ${response.status}`); - } - - const midiBuffer = await response.arrayBuffer(); - - // Update UI - selectedTitle.textContent = result.title; - selectedMeta.innerHTML = ` - Source: ${result.source} | - Size: ${(midiBuffer.byteLength / 1024).toFixed(1)}KB | - Confidence: ${Math.round(result.confidence * 100)}% - `; - - playerSection.classList.add('visible'); - - // Enable preview button - const previewBtn = document.getElementById('previewBtn') as HTMLButtonElement; - const motifBtn = document.getElementById('motifBtn') as HTMLButtonElement; - previewBtn.disabled = false; - motifBtn.disabled = false; - - // Store MIDI data for playback - (this as any).currentMIDI = { buffer: midiBuffer, result }; - - status.textContent = 'MIDI loaded. You can now preview or generate synthesis.'; - - } catch (error) { - console.error('Load error:', error); - status.textContent = `Load error: ${error}`; - } - } - - public async handlePreview(): Promise { - console.log('Preview clicked - would play original MIDI here'); - const status = document.getElementById('status')!; - status.textContent = 'Preview playback not yet implemented - but MIDI is loaded!'; - } - - public async handleMotif(): Promise { - console.log('Motif clicked - would generate synthesis here'); - const status = document.getElementById('status')!; - status.textContent = 'Motif synthesis not yet implemented - but MIDI is parsed!'; - } -} - -// Initialize app -new MotifApp(); - -// Expose handlers for buttons -(window as any).handlePreview = () => (window as any).app.handlePreview(); -(window as any).handleMotif = () => (window as any).app.handleMotif(); \ No newline at end of file diff --git a/test.html b/test.html deleted file mode 100644 index d936db7..0000000 --- a/test.html +++ /dev/null @@ -1,21 +0,0 @@ - - - - Test - - - -
Not loaded
- - - - \ No newline at end of file diff --git a/thursday-aims-progress-problems.md b/thursday-aims-progress-problems.md deleted file mode 100644 index 8ecc81f..0000000 --- a/thursday-aims-progress-problems.md +++ /dev/null @@ -1,107 +0,0 @@ -# MOTIF Project - Thursday Status Report - -## Project Overview - -**MOTIF** is a procedural music synthesis system that extracts structural information from existing MIDI files and recreates them as original, real-time audio using Web Audio API. The core concept: "music as executable structure, not static audio." - -### Workflow -1. User searches for a song by name -2. System finds MIDI files from multiple sources (BitMidi, Dongrays) -3. MIDI is parsed and analyzed for structural features (tempo, density, melodic patterns) -4. Notes are mapped to synthesis "roles" (bass, drone, ostinato, texture, accents) -5. Web Audio API generates procedural audio with similar "feel" but original sound - -## Current Status: Functional MVP - -### ✅ Completed Features - -**Complete Search Pipeline** -- Multi-source MIDI search with confidence scoring -- Real MIDI integration with fetching and parsing -- Graceful error handling and timeouts - -**Role-Based Synthesis Engine** -- Intelligent mapping of MIDI tracks to synthesis layers -- Role-specific oscillator types and filtering -- Velocity-sensitive ADSR envelopes -- Polyphonic chord support -- Automatic looping and proper cleanup - -**Polished User Interface** -- Search results table with confidence bars and quality analysis -- Dual player UI: Preview Original MIDI vs Generate Motif -- Real-time status updates and progress feedback -- "Try Next Result" workflow for easy A/B testing - -**Backend Infrastructure** -- Express + TypeScript server -- CORS proxy with validation and SHA256 disk caching -- Multi-source search (BitMidi, Dongrays, synthetic fallback) -- Quality assessment with penalties for problematic content - -### 📈 Recent Progress (Latest Commit) - -**Major UI/UX Improvements:** -- Implemented search results table with metadata display -- Added MIDI preview player with basic oscillator mapping -- Built dual transport controls for comparison -- Integrated ParsedMIDIInfo with comprehensive track analysis -- Enhanced confidence scoring system with quality penalties - -**Technical Enhancements:** -- Sophisticated role mapping with pitch range and density analysis -- Improved error handling across the pipeline -- Better synthesis scheduling with Web Audio lookahead -- Streamlined search-to-synthesis workflow - -## Current Problems & Limitations - -### 🔴 Performance Issues -- **~70% search success rate** (goal: >90%) -- **~30% musical similarity recognition** (goal: >70%) -- HTML regex parsing is fragile (should use DOM parsing) - -### 🟡 Feature Limitations -- Basic role mapping heuristics (lacks harmonic analysis) -- Simple synthesis timbres (basic oscillators only) -- Limited MIDI source coverage -- No user controls for synthesis parameters - -### 🟠 Technical Debt -- Need more robust parsing for edge cases -- Search confidence scoring could be more sophisticated -- Some synthesis roles need refinement - -## Next Sprint Priorities - -### Phase 1: Core Stability (Next 1-2 weeks) -1. **Improve search success rate** - better error handling, additional sources -2. **Enhance role mapping** - add harmonic analysis, rhythm detection -3. **Polish synthesis** - more interesting timbres, dynamic control -4. **Robust parsing** - replace regex with proper DOM parsing - -### Phase 2: Musical Intelligence (2-4 weeks) -1. **Smarter scoring** - melodic similarity, harmonic progression analysis -2. **Advanced synthesis** - effects, modulation, realistic instruments -3. **User controls** - synthesis parameter adjustment, role customization -4. **More MIDI sources** - expand search coverage - -### Phase 3: Production Ready (1-2 months) -1. **Performance optimization** - caching, preloading, worker threads -2. **Legal compliance** - proper attribution, copyright handling -3. **User uploads** - allow custom MIDI file analysis -4. **Production infrastructure** - deployment, monitoring, scaling - -## Technical Architecture - -**Backend:** Express + TypeScript with multi-source search, CORS proxy, and caching -**Frontend:** TypeScript + Vite + Web Audio with real-time synthesis -**Key Components:** MotifEngine, RoleMapper, SynthesisEngine, MIDIPlayer - -## Demo Status - -✅ **Ready to demonstrate** - Full end-to-end pipeline functional -✅ **User-friendly interface** - Polished search and playback experience -✅ **Comparative validation** - Side-by-side original vs synthesis preview - -The project successfully proves the core concept and is ready for user testing and iterative improvement. \ No newline at end of file