Clean up repo for public release
Removed: - Old planning docs (DEPLOYMENT.md, IMPLEMENTATION.md, etc.) - Dev notes (thursday-aims-progress-problems.md) - Old backup files (main-simple.ts, main-working.ts) - Test file (test.html) - Unused image (chiptune_blog_piece_1000x.webp) - docs/ folder with old audit/plan docs Updated: - Renamed claude.md to CLAUDE.md - Added .claude/ to gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -34,6 +34,7 @@ logs/
|
|||||||
.cache/
|
.cache/
|
||||||
.tmp/
|
.tmp/
|
||||||
.vercel
|
.vercel
|
||||||
|
.claude/
|
||||||
|
|
||||||
# Backend runtime cache (downloaded MIDI files)
|
# Backend runtime cache (downloaded MIDI files)
|
||||||
server/cache/
|
server/cache/
|
||||||
|
|||||||
-102
@@ -1,102 +0,0 @@
|
|||||||
# Deployment Guide for Motif
|
|
||||||
|
|
||||||
## iOS Fixes Applied
|
|
||||||
- ✓ Fixed auto-zoom on input focus (font-size: 16px, viewport locked)
|
|
||||||
- ✓ Prevented zoom on touch (maximum-scale=1.0, user-scalable=no)
|
|
||||||
|
|
||||||
## Backend Deployment
|
|
||||||
|
|
||||||
The MIDI search requires a backend server to be running. You have two options:
|
|
||||||
|
|
||||||
### Option 1: Deploy Backend Separately (Recommended for Production)
|
|
||||||
|
|
||||||
1. Deploy the backend to a service like Render, Railway, or Heroku:
|
|
||||||
```bash
|
|
||||||
cd server
|
|
||||||
npm install
|
|
||||||
npm run build
|
|
||||||
npm start
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Set the `VITE_API_URL` environment variable in your frontend deployment to point to your backend:
|
|
||||||
```
|
|
||||||
VITE_API_URL=https://your-backend-url.com
|
|
||||||
```
|
|
||||||
|
|
||||||
### Option 2: Run Backend Locally (Development Only)
|
|
||||||
|
|
||||||
1. In one terminal, start the backend:
|
|
||||||
```bash
|
|
||||||
npm run dev:backend
|
|
||||||
```
|
|
||||||
|
|
||||||
2. In another terminal, start the frontend:
|
|
||||||
```bash
|
|
||||||
npm run dev
|
|
||||||
```
|
|
||||||
|
|
||||||
## Vercel Deployment
|
|
||||||
|
|
||||||
### Deploy Frontend to Vercel:
|
|
||||||
|
|
||||||
1. Build the project:
|
|
||||||
```bash
|
|
||||||
npm run build
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Deploy to Vercel:
|
|
||||||
```bash
|
|
||||||
vercel --prod
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Set environment variable in Vercel dashboard:
|
|
||||||
- Variable name: `VITE_API_URL`
|
|
||||||
- Value: Your deployed backend URL (e.g., `https://motif-backend.onrender.com`)
|
|
||||||
|
|
||||||
### Deploy Backend to Render/Railway:
|
|
||||||
|
|
||||||
1. Create a new Web Service
|
|
||||||
2. Connect your GitHub repository
|
|
||||||
3. Set build command: `cd server && npm install && npm run build`
|
|
||||||
4. Set start command: `cd server && npm start`
|
|
||||||
5. Add environment variable `PORT` (usually auto-set)
|
|
||||||
6. Deploy
|
|
||||||
|
|
||||||
## Environment Variables
|
|
||||||
|
|
||||||
Create a `.env` file in the root directory for local development:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
VITE_API_URL=http://localhost:3001
|
|
||||||
```
|
|
||||||
|
|
||||||
For production, set this in your deployment platform (Vercel, Netlify, etc.):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
VITE_API_URL=https://your-backend-api-url.com
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing the Deployment
|
|
||||||
|
|
||||||
1. Open the deployed URL on iOS Safari
|
|
||||||
2. Search for a song (e.g., "Hotel California")
|
|
||||||
3. You should see MIDI results load
|
|
||||||
4. Select a result and play it
|
|
||||||
5. The page should NOT zoom when touching the search input
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### No search results on production:
|
|
||||||
- Check that backend is running (visit `https://your-backend-url.com/health`)
|
|
||||||
- Verify `VITE_API_URL` environment variable is set correctly in Vercel
|
|
||||||
- Check browser console for CORS errors
|
|
||||||
- Rebuild frontend after setting environment variables
|
|
||||||
|
|
||||||
### iOS zoom issue persists:
|
|
||||||
- Clear Safari cache
|
|
||||||
- Hard reload the page
|
|
||||||
- Check that the latest build is deployed
|
|
||||||
|
|
||||||
### CORS errors:
|
|
||||||
- Backend must allow requests from your frontend domain
|
|
||||||
- Check server CORS configuration in `server/src/server.ts`
|
|
||||||
@@ -1,251 +0,0 @@
|
|||||||
# MOTIF Implementation Status & Roadmap
|
|
||||||
|
|
||||||
**Current Status**: MVP functional with real MIDI search, parsing, and procedural synthesis
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✅ What's Been Built
|
|
||||||
|
|
||||||
### Backend API (Express + TypeScript)
|
|
||||||
|
|
||||||
**Endpoints:**
|
|
||||||
- `GET /api/midi/search?q=song` - Multi-source MIDI search
|
|
||||||
- `GET /api/midi/fetch?u=url` - CORS proxy with validation and caching
|
|
||||||
- `GET /health` - Service health check
|
|
||||||
|
|
||||||
**Architecture:**
|
|
||||||
```
|
|
||||||
server/src/
|
|
||||||
├── adapters/ # Search source implementations
|
|
||||||
│ ├── BitMidiAdapter # HTML parsing for bitmidi.com
|
|
||||||
│ └── DongraysAdapter # HTML parsing for dongrays.net
|
|
||||||
├── services/
|
|
||||||
│ ├── MIDISearchService # Orchestrates multi-source search
|
|
||||||
│ └── MIDIFetchService # Downloads, validates, caches MIDI
|
|
||||||
└── utils/
|
|
||||||
└── ScoreUtils # Confidence scoring & quality assessment
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Features:**
|
|
||||||
- **Confidence Scoring**: Token matching, quality penalties (karaoke, broken files)
|
|
||||||
- **Disk Caching**: SHA256-hashed files with JSON index
|
|
||||||
- **Validation**: MIDI header checks, file size limits (10MB max)
|
|
||||||
- **Error Handling**: Timeouts, graceful fallbacks
|
|
||||||
- **Deduplication**: Removes duplicate results across sources
|
|
||||||
|
|
||||||
### Frontend (TypeScript + Vite + Web Audio)
|
|
||||||
|
|
||||||
**Architecture:**
|
|
||||||
```
|
|
||||||
src/
|
|
||||||
├── core/
|
|
||||||
│ ├── MotifEngine # Main orchestrator
|
|
||||||
│ └── RoleMapper # MIDI track → synthesis role assignment
|
|
||||||
├── midi/
|
|
||||||
│ ├── MIDIProcessor # Feature extraction (tempo, density, etc.)
|
|
||||||
│ └── MIDIParser # @tonejs/midi wrapper
|
|
||||||
├── synthesis/
|
|
||||||
│ └── SynthesisEngine # Pure Web Audio procedural synthesis
|
|
||||||
├── services/
|
|
||||||
│ └── MIDIService # Backend API client
|
|
||||||
└── types/
|
|
||||||
└── index # TypeScript interfaces
|
|
||||||
```
|
|
||||||
|
|
||||||
**Synthesis Pipeline:**
|
|
||||||
```
|
|
||||||
Song Name → MIDI Search → Parse Events → Role Assignment → Web Audio Synthesis
|
|
||||||
```
|
|
||||||
|
|
||||||
### Integration Flow
|
|
||||||
|
|
||||||
1. **User enters song name** → `MotifEngine.generateFromSong()`
|
|
||||||
2. **MIDI Search** → `MIDIService.search()` → Backend `/search` endpoint
|
|
||||||
3. **Multi-source search** → BitMidi + Dongrays adapters in parallel
|
|
||||||
4. **Result ranking** → Confidence scoring, deduplication
|
|
||||||
5. **MIDI Fetch** → `MIDIService.fetchMIDI()` → Backend `/fetch` with caching
|
|
||||||
6. **MIDI Parsing** → `@tonejs/midi` → Normalized `NoteEvent[]` array
|
|
||||||
7. **Role Assignment** → `RoleMapper` → Bass/Drone/Ostinato/Texture/Accents
|
|
||||||
8. **Web Audio Synthesis** → `SynthesisEngine` → Real-time procedural audio
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 How It Actually Works
|
|
||||||
|
|
||||||
### MIDI Resolution Strategy
|
|
||||||
|
|
||||||
**Sources (MVP):**
|
|
||||||
- **BitMidi**: Regex parsing of search results, direct `.mid` links
|
|
||||||
- **Dongrays**: Similar approach, handles download endpoints
|
|
||||||
- **Synthetic Fallback**: Hash-based procedural generation if search fails
|
|
||||||
|
|
||||||
**Scoring Heuristics:**
|
|
||||||
- Token matching between query and title
|
|
||||||
- Penalties for "karaoke", "vocal", "broken"
|
|
||||||
- Bonus for direct `.mid` links
|
|
||||||
- Source preference (BitMidi slightly favored)
|
|
||||||
|
|
||||||
### Role-Based Synthesis
|
|
||||||
|
|
||||||
**Role Assignment:**
|
|
||||||
```typescript
|
|
||||||
// Heuristic rules:
|
|
||||||
pitch < 48 + short notes = Bass
|
|
||||||
long duration > 2s = Drone
|
|
||||||
short + repetitive = Ostinato
|
|
||||||
high velocity = Accents
|
|
||||||
everything else = Texture
|
|
||||||
```
|
|
||||||
|
|
||||||
**Synthesis Per Role:**
|
|
||||||
- **Bass**: Square wave, lowpass filter, punchy envelopes
|
|
||||||
- **Drone**: Sawtooth, bandpass, sustained notes
|
|
||||||
- **Ostinato**: Triangle, highpass, rhythmic patterns
|
|
||||||
- **Texture**: Sine, bandpass, atmospheric
|
|
||||||
- **Accents**: Sine, peaking filter, sharp attacks
|
|
||||||
|
|
||||||
**Web Audio Implementation:**
|
|
||||||
- Lookahead scheduling (100ms)
|
|
||||||
- MIDI note → Hz conversion: `440 * 2^((note-69)/12)`
|
|
||||||
- Velocity-sensitive envelopes
|
|
||||||
- Automatic looping when MIDI ends
|
|
||||||
- Per-note oscillator + gain envelope
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚨 Current Limitations
|
|
||||||
|
|
||||||
### Search Quality
|
|
||||||
- **HTML Parsing**: Fragile regex-based extraction (not DOM parsing)
|
|
||||||
- **Limited Sources**: Only 2 sources, no fallbacks if both fail
|
|
||||||
- **No Metadata**: Can't validate artist, album, year matching
|
|
||||||
- **Rate Limiting**: No request throttling or backoff
|
|
||||||
|
|
||||||
### MIDI Processing
|
|
||||||
- **Simple Role Mapping**: Basic pitch/duration heuristics only
|
|
||||||
- **No Harmonic Analysis**: Doesn't understand chord progressions
|
|
||||||
- **Track Correlation**: Doesn't detect melody vs accompaniment intelligently
|
|
||||||
- **Tempo Handling**: Assumes constant tempo, ignores tempo changes
|
|
||||||
|
|
||||||
### Synthesis Engine
|
|
||||||
- **Basic Timbres**: Simple oscillator types, no complex synthesis
|
|
||||||
- **No Dynamics**: Volume levels are role-based, not musically aware
|
|
||||||
- **Limited Effects**: Only basic filtering, no reverb/chorus/etc.
|
|
||||||
- **Monophonic Layers**: Each role plays one note at a time
|
|
||||||
|
|
||||||
### Frontend UX
|
|
||||||
- **No Progress Feedback**: Search/fetch happens in black box
|
|
||||||
- **No Result Preview**: Can't see what MIDI was found before synthesis
|
|
||||||
- **No Controls**: Can't adjust synthesis parameters
|
|
||||||
- **Error Messages**: Generic error handling
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🎯 Next Steps (Prioritized)
|
|
||||||
|
|
||||||
### Phase 1: Polish MVP
|
|
||||||
**Goal**: Make current system reliable and user-friendly
|
|
||||||
|
|
||||||
1. **Better Error Handling**
|
|
||||||
- Show search progress ("Searching BitMidi...", "Parsing MIDI...")
|
|
||||||
- Display actual MIDI file found before synthesis
|
|
||||||
- Graceful degradation with informative messages
|
|
||||||
|
|
||||||
2. **Improve Role Mapping**
|
|
||||||
- Add harmonic analysis (detect bass lines, chord patterns)
|
|
||||||
- Use track names/MIDI program changes as hints
|
|
||||||
- Smarter melody vs accompaniment detection
|
|
||||||
|
|
||||||
3. **Synthesis Polish**
|
|
||||||
- Add polyphony within roles (chords, multiple bass notes)
|
|
||||||
- Better envelopes (ADSR with release tails)
|
|
||||||
- Basic effects (simple reverb, subtle filtering LFOs)
|
|
||||||
|
|
||||||
### Phase 2: Search Enhancement
|
|
||||||
**Goal**: Higher success rate finding good MIDIs
|
|
||||||
|
|
||||||
4. **Robust Parsing**
|
|
||||||
- Switch to Cheerio for proper DOM parsing
|
|
||||||
- Handle dynamic content/JavaScript-loaded results
|
|
||||||
- Add more MIDI sources (MuseScore, IMSLP public domain)
|
|
||||||
|
|
||||||
5. **Smarter Scoring**
|
|
||||||
- Artist name matching with fuzzy string comparison
|
|
||||||
- Duration validation (reject 30-second clips, 20-minute symphonies)
|
|
||||||
- Key signature and time signature analysis
|
|
||||||
|
|
||||||
6. **Caching & Performance**
|
|
||||||
- Cache search results (not just MIDI files)
|
|
||||||
- Add request deduplication and rate limiting
|
|
||||||
- Background refresh of popular files
|
|
||||||
|
|
||||||
### Phase 3: Synthesis Sophistication
|
|
||||||
**Goal**: More recognizable and musical output
|
|
||||||
|
|
||||||
7. **Advanced Synthesis**
|
|
||||||
- Multiple synthesis modes per role (subtractive, FM, additive)
|
|
||||||
- Tempo-synced effects and modulation
|
|
||||||
- Cross-role interaction (bass and drums lock together)
|
|
||||||
|
|
||||||
8. **Musical Intelligence**
|
|
||||||
- Detect and preserve harmonic progressions
|
|
||||||
- Rhythmic pattern extraction and variation
|
|
||||||
- Dynamic arrangement (intro/verse/chorus detection)
|
|
||||||
|
|
||||||
9. **User Controls**
|
|
||||||
- Synthesis parameter sliders (brightness, warmth, density)
|
|
||||||
- Role muting/soloing
|
|
||||||
- Tempo adjustment and time-stretching
|
|
||||||
|
|
||||||
### Phase 4: Production Ready
|
|
||||||
**Goal**: Reliable service for real users
|
|
||||||
|
|
||||||
10. **Infrastructure**
|
|
||||||
- Database for MIDI metadata and search caching
|
|
||||||
- CDN for popular MIDI files
|
|
||||||
- Analytics and error monitoring
|
|
||||||
|
|
||||||
11. **Legal & Content**
|
|
||||||
- MIDI license validation
|
|
||||||
- User-uploaded MIDI support
|
|
||||||
- Integration with Creative Commons sources
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔬 Technical Debt
|
|
||||||
|
|
||||||
### Immediate
|
|
||||||
- Remove `crypto` dependency warning in backend package.json
|
|
||||||
- Add proper TypeScript strict mode compliance
|
|
||||||
- Implement proper error boundaries in frontend
|
|
||||||
|
|
||||||
### Medium Term
|
|
||||||
- Replace regex HTML parsing with proper DOM parsing
|
|
||||||
- Add comprehensive logging/telemetry
|
|
||||||
- Write unit tests for core algorithms (role mapping, scoring)
|
|
||||||
|
|
||||||
### Long Term
|
|
||||||
- Consider WebAssembly for intensive audio processing
|
|
||||||
- Evaluate Web Workers for MIDI parsing/analysis
|
|
||||||
- Implement WebRTC for real-time collaboration features
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Success Metrics
|
|
||||||
|
|
||||||
**Current State**:
|
|
||||||
- ✅ Searches return results ~70% of time
|
|
||||||
- ✅ Successfully parses most MIDI files found
|
|
||||||
- ✅ Generates audio output 100% of time (with fallback)
|
|
||||||
- ⚠️ Output recognizably similar to input ~30% of time
|
|
||||||
|
|
||||||
**Target State**:
|
|
||||||
- 🎯 Search success rate >90%
|
|
||||||
- 🎯 Musical similarity recognition >70%
|
|
||||||
- 🎯 User "that sounds like the song" reaction >60%
|
|
||||||
- 🎯 Sub-3-second generation time 95% of requests
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Built**: Functional end-to-end MVP with real MIDI integration
|
|
||||||
**Next**: Polish the core experience before expanding features
|
|
||||||
@@ -1,163 +0,0 @@
|
|||||||
## MIDI playback MVP — detailed technical integration plan
|
|
||||||
|
|
||||||
### Product target (MVP)
|
|
||||||
- **User flow**:
|
|
||||||
1) User types a song name (e.g. “Hotel California”)
|
|
||||||
2) App searches MIDI sources and shows a ranked list
|
|
||||||
3) User selects a result and can **play the MIDI “correctly”** using **General MIDI soundfonts**
|
|
||||||
4) User can then click **Generate Motif** to synthesize a “similar-but-different” version from the same parsed MIDI
|
|
||||||
|
|
||||||
### Current code reality (gaps to close)
|
|
||||||
- **Search is currently biased toward mock/synthetic**, not real MIDI:
|
|
||||||
- `server/src/services/MIDISearchService.ts` includes `MockAdapter` first.
|
|
||||||
- **Fetch can silently replace real URLs with synthetic MIDI**:
|
|
||||||
- `server/src/services/MIDIFetchService.ts` generates synthetic MIDI when URL contains `bitmidi.com/uploads`, preventing true BitMidi playback.
|
|
||||||
- **Frontend preview is oscillator-based**, not GM soundfont playback.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Architecture (what we’ll ship)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart LR
|
|
||||||
User -->|typesQuery| FrontendUI
|
|
||||||
FrontendUI -->|GET /api/midi/search?q=...| BackendSearch
|
|
||||||
BackendSearch -->|rankedResults| FrontendUI
|
|
||||||
FrontendUI -->|selectResult + GET /api/midi/fetch?u=...| BackendFetch
|
|
||||||
BackendFetch -->|midiBytes| FrontendParse
|
|
||||||
FrontendParse -->|NoteEvents + TrackMeta| PreviewPlayerGM
|
|
||||||
FrontendParse -->|NoteEvents| MotifEngine
|
|
||||||
MotifEngine -->|roles + chords| SynthesisEngine
|
|
||||||
PreviewPlayerGM --> AudioOut
|
|
||||||
SynthesisEngine --> AudioOut
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 0 — “Real MIDI mode” defaults (backend hardening)
|
|
||||||
|
|
||||||
### 0.1 Gate mock adapter behind env flag
|
|
||||||
- **Change**: In `server/src/services/MIDISearchService.ts`, make adapters:
|
|
||||||
- Default: `[BitMidiAdapter, DongraysAdapter]`
|
|
||||||
- Optional: prepend `MockAdapter` only if `USE_MOCK_ADAPTER=1` (or similar)
|
|
||||||
- **Acceptance**:
|
|
||||||
- Searching “Hotel California” returns **non-`synthetic:*`** results when internet is available.
|
|
||||||
- Devs can still run offline with `USE_MOCK_ADAPTER=1`.
|
|
||||||
|
|
||||||
### 0.2 Stop auto-synth overriding real URLs in fetch
|
|
||||||
- **Change**: In `server/src/services/MIDIFetchService.ts`, only synthesize when:
|
|
||||||
- `url.startsWith('synthetic:')` (and optionally if `USE_SYNTHETIC_FETCH=1`)
|
|
||||||
- **Remove**: `url.includes('bitmidi.com/uploads')` synthetic shortcut
|
|
||||||
- **Acceptance**:
|
|
||||||
- Selecting a BitMidi result triggers a real network fetch and caches the real bytes.
|
|
||||||
- If the remote file is invalid, it fails explicitly with a clear error.
|
|
||||||
|
|
||||||
### 0.3 Make backend behavior explicit in responses (optional but recommended)
|
|
||||||
- **Change**: Add response fields or headers indicating source:
|
|
||||||
- Example: `X-Motif-Source: real|synthetic|cache`
|
|
||||||
- **Acceptance**:
|
|
||||||
- Frontend can display “cached”/“live”/“synthetic fallback” badges (helps debugging + trust).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 1 — GM soundfont playback (frontend “Preview” becomes correct MIDI playback)
|
|
||||||
|
|
||||||
### 1.1 Dependency choice & asset strategy
|
|
||||||
- **Dependency**: add a browser-friendly soundfont player dependency (e.g. `soundfont-player`).
|
|
||||||
- **Soundfont hosting**:
|
|
||||||
- Prefer static hosting under `/public/soundfonts/` (versioned with the app)
|
|
||||||
- Or use a CDN, but pin versions and handle CORS
|
|
||||||
- **MVP instrument set**:
|
|
||||||
- Minimum viable: **Acoustic Grand Piano** for all melodic tracks, and a basic drum fallback
|
|
||||||
- Better: load instruments on demand per track program
|
|
||||||
|
|
||||||
### 1.2 Implement `SoundfontMIDIPlayer`
|
|
||||||
Create `src/synthesis/SoundfontMIDIPlayer.ts` with:
|
|
||||||
- **Responsibilities**
|
|
||||||
- Load instruments (program → soundfont instrument name)
|
|
||||||
- Schedule note-on/note-off with WebAudio timing
|
|
||||||
- Provide `load(midi)` / `play()` / `stop()` / `setVolume()` APIs
|
|
||||||
- **Inputs**
|
|
||||||
- Best: use `@tonejs/midi`’s `Midi` object (tracks include `instrument.number`, `notes`, `channel`)
|
|
||||||
- Alternate: keep using `NoteEvent[]`, but you’ll lose program/channel unless you extend the event model
|
|
||||||
- **Timing correctness**
|
|
||||||
- Use seconds-based timing from `@tonejs/midi` notes (`time`, `duration` are in seconds)
|
|
||||||
- Ensure AudioContext resumes on user gesture
|
|
||||||
- **Drums**
|
|
||||||
- If channel 9/10 is detected: either map to a percussion kit if supported, or skip drums for MVP (but be explicit in UI)
|
|
||||||
|
|
||||||
### 1.3 Wire UI to use soundfont preview
|
|
||||||
- In `src/main.ts`, replace/augment the current oscillator `MIDIPlayer` usage:
|
|
||||||
- Preview buttons should play via `SoundfontMIDIPlayer`
|
|
||||||
- Keep existing Motif buttons intact
|
|
||||||
- Keep oscillator preview only as a fallback if soundfonts fail to load (optional).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 2 — “Play the real song correctly” UX loop
|
|
||||||
|
|
||||||
### 2.1 Search UX requirements
|
|
||||||
- **Result list must show**:
|
|
||||||
- title, source, confidence
|
|
||||||
- parsed metadata: duration, track count, issues
|
|
||||||
- **Selection behavior**:
|
|
||||||
- Selecting a row fetches + parses once; enables Preview + Generate Motif
|
|
||||||
|
|
||||||
### 2.2 Error handling (user-facing)
|
|
||||||
Define user-facing error classes/messages:
|
|
||||||
- **Search**: “No results”, “Backend unavailable”, “Rate limited / source blocked”
|
|
||||||
- **Fetch**: “MIDI file blocked”, “Invalid MIDI header”, “Quality rejected”
|
|
||||||
- **Parse**: “Unsupported MIDI features” / “Parse failed”
|
|
||||||
- **Preview playback**: “Soundfont failed to load” / “Audio not allowed until click”
|
|
||||||
|
|
||||||
### 2.3 Observability (dev-facing)
|
|
||||||
- Backend: log adapter failures per source + timings
|
|
||||||
- Frontend: log selected MIDI URL, parse duration, instrument load times
|
|
||||||
- Optional: a small “Debug” accordion showing chosen URL, cache hit, parse issues, loaded instruments
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 3 — Motif generation stays step 2 (but align data model)
|
|
||||||
- Keep the current path:
|
|
||||||
- `MotifEngine.generateFromMIDI(NoteEvent[])` then `MotifEngine.play()`
|
|
||||||
- Recommended alignment work:
|
|
||||||
- Decide whether Motif should later consume richer track metadata (program/channel) to improve role mapping.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Integration milestones & acceptance checks
|
|
||||||
|
|
||||||
### Milestone A — Real MIDI end-to-end
|
|
||||||
- Search returns results from BitMidi/Dongrays with mock disabled by default
|
|
||||||
- Fetch returns real bytes and caches them
|
|
||||||
- Parse endpoint `/api/midi/parse` works for metadata
|
|
||||||
|
|
||||||
### Milestone B — “Correct” preview playback
|
|
||||||
- Preview produces recognizable instrument playback (piano at minimum)
|
|
||||||
- Stop reliably stops scheduled notes
|
|
||||||
- Works in Chrome/Safari with autoplay policies (requires click)
|
|
||||||
|
|
||||||
### Milestone C — Motif as second step
|
|
||||||
- Generate Motif still works on the same loaded MIDI
|
|
||||||
- Preview and Motif can be A/B tested without reloading the page
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Recommended team task breakdown
|
|
||||||
|
|
||||||
### Backend engineer
|
|
||||||
- Implement env gating for mock/synthetic
|
|
||||||
- Tighten fetch behavior + ensure BitMidi URLs are truly fetched
|
|
||||||
- Add explicit “source = real/cache/synthetic” marker
|
|
||||||
|
|
||||||
### Frontend engineer
|
|
||||||
- Add the chosen soundfont library
|
|
||||||
- Implement `SoundfontMIDIPlayer`
|
|
||||||
- Wire `src/main.ts` preview buttons to soundfont playback
|
|
||||||
- Add user-facing error messaging for soundfont failures
|
|
||||||
|
|
||||||
### QA / test harness
|
|
||||||
- Maintain a short list of known-good queries (3–5 songs) and confirm:
|
|
||||||
- search results appear
|
|
||||||
- at least one MIDI fetches and plays
|
|
||||||
- Motif plays afterward
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
# Home UX audit + improvements (arcade-first)
|
|
||||||
|
|
||||||
This doc is a saved copy of the current home UX audit plan (no code changes).
|
|
||||||
|
|
||||||
## Current UX issues (what’s making it feel uninspiring/confusing)
|
|
||||||
- **Unclear hierarchy**: “Search”, “Preview”, and “Synthesis Engine” compete; users don’t instantly see the main thing to do.
|
|
||||||
- **Weak step-by-step guidance**: The app is a 3-step flow (Search → Select → Run engine), but the UI doesn’t feel like a guided sequence.
|
|
||||||
- **Controls feel detached from state**: Buttons don’t always read as “locked until selection”; the reason for disabled state isn’t visible.
|
|
||||||
- **Results table is dense** on mobile: long titles wrap unpredictably, selection highlight is subtle.
|
|
||||||
- **Copy and concept drift**: “emulator/engine” and “MIDI scrape/analyse” copy varies; the story should be consistent and confident.
|
|
||||||
- **iOS audio state**: The “Enable Audio” banner is reactive but not integrated into the flow (and can appear “random”).
|
|
||||||
|
|
||||||
## Target outcome
|
|
||||||
Arcade cabinet vibe with a clear, dramatic main CTA:
|
|
||||||
- **Primary action**: “Run the engine” (Generate & Play)
|
|
||||||
- Secondary: preview the MIDI
|
|
||||||
- Stronger “insert coin” style guidance: show step chips and a big engine panel.
|
|
||||||
|
|
||||||
## Proposed UX changes (high impact, low risk)
|
|
||||||
|
|
||||||
### 1) Make the flow explicit
|
|
||||||
Update `index.html`:
|
|
||||||
- Add a compact “Steps” row near the top:
|
|
||||||
- Step 1: Search
|
|
||||||
- Step 2: Pick a MIDI
|
|
||||||
- Step 3: Run the engine
|
|
||||||
- Tie each step’s visual state to app state:
|
|
||||||
- Step 2/3 show “locked” styling until available.
|
|
||||||
|
|
||||||
### 2) Promote “Synthesis Engine” as the hero module
|
|
||||||
- Move the engine card visually above Preview (or keep order but make engine card visually dominant).
|
|
||||||
- Make engine CTA bigger and more arcade:
|
|
||||||
- Primary button: **Run the engine**
|
|
||||||
- Secondary: Stop
|
|
||||||
- Volume + progress remain but visually subordinate.
|
|
||||||
- Add a small one-line “what this does” under the button, not as a paragraph.
|
|
||||||
|
|
||||||
### 3) Preview becomes clearly “optional”
|
|
||||||
- Rename to “Listen to the source MIDI (optional)”
|
|
||||||
- Collapse preview by default on mobile (or add a “Show preview” toggle) to reduce overwhelm.
|
|
||||||
|
|
||||||
### 4) Results list readability + selection confidence
|
|
||||||
- Increase row tap targets and selection contrast.
|
|
||||||
- Add a right-side “Selected” chip on the selected row.
|
|
||||||
- On mobile:
|
|
||||||
- clamp long titles to 2 lines
|
|
||||||
- reduce columns (hide Source or Duration) based on width.
|
|
||||||
|
|
||||||
### 5) iOS audio UX integrated into the engine (subtle)
|
|
||||||
- Only on iOS-like browsers and only when audio is locked.
|
|
||||||
- Show a small under-section link: “Having trouble on iOS? Enable audio”
|
|
||||||
- Clicking expands a compact CTA row (Enable Audio button + tiny state line).
|
|
||||||
- Avoid a big persistent banner that could confuse non-iOS users.
|
|
||||||
|
|
||||||
### 6) Copy pass
|
|
||||||
Deferred for now: keep copy changes out of this pass.
|
|
||||||
|
|
||||||
## Implementation plan (concrete)
|
|
||||||
|
|
||||||
### Files
|
|
||||||
- Primary: `index.html`
|
|
||||||
- Minor TS adjustments (state-driven CSS classes): `src/main.ts`
|
|
||||||
|
|
||||||
### Approach
|
|
||||||
- Add a small set of **state CSS classes** on `body` (or on `.container`):
|
|
||||||
- `state-has-results`
|
|
||||||
- `state-has-selection`
|
|
||||||
- `state-audio-locked`
|
|
||||||
- In `src/main.ts`, toggle these classes when:
|
|
||||||
- results are loaded
|
|
||||||
- a selection is made
|
|
||||||
- audio unlock state changes
|
|
||||||
- Use CSS to:
|
|
||||||
- style locked sections
|
|
||||||
- highlight the engine CTA
|
|
||||||
- improve mobile responsiveness
|
|
||||||
|
|
||||||
## Test plan
|
|
||||||
- Desktop Chrome/Safari:
|
|
||||||
- search → select → engine run
|
|
||||||
- preview optional still works
|
|
||||||
- iOS Safari:
|
|
||||||
- first run shows “arm audio” only when needed
|
|
||||||
- no random banner when audio is already running
|
|
||||||
- Visual:
|
|
||||||
- mobile widths (320–430px)
|
|
||||||
- long titles wrapping
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
- No changes to the core synthesis engine behavior or backend.
|
|
||||||
- No copywriting/wording changes in this pass.
|
|
||||||
|
|
||||||
@@ -1,176 +0,0 @@
|
|||||||
# Sharing plan: links + player landing + Motif export
|
|
||||||
|
|
||||||
This document captures the implementation plan for **Share links** (Copy Link + Share to X), a minimal **player landing page**, and the next step: sharing **generated Motif variants** via **exported artifacts** (MIDI and/or audio).
|
|
||||||
|
|
||||||
## Goals
|
|
||||||
- **Shareable outcome**: users can share something they heard (original MIDI preview, or a generated Motif variant).
|
|
||||||
- **One-click-to-hear**: shared links load quickly and are ready to play (but never true autoplay; iOS requires a user gesture).
|
|
||||||
- **Lightweight**: keep the landing page minimal and stable.
|
|
||||||
- **Safe**: shared URLs must not turn the backend into an open proxy.
|
|
||||||
|
|
||||||
## MVP constraints (important)
|
|
||||||
- **iOS/Safari** blocks audio until a user gesture. The landing can preload, but playback needs a tap.
|
|
||||||
- **X/Twitter** shares are links + preview cards. No reliable in-post WebAudio playback.
|
|
||||||
- Search results can change over time, so **index-based sharing** is brittle.
|
|
||||||
|
|
||||||
## UX spec
|
|
||||||
|
|
||||||
### A) Main app: Share controls
|
|
||||||
Placement: near the “Selected MIDI”/result context (where users decide something is share-worthy).
|
|
||||||
|
|
||||||
Controls:
|
|
||||||
- **Copy link**: copies a permalink to clipboard + shows a small “Copied” toast.
|
|
||||||
- **Share to X**: opens the tweet composer with prefilled text + URL.
|
|
||||||
|
|
||||||
X intent URL format:
|
|
||||||
- `https://twitter.com/intent/tweet?text=<encodedText>&url=<encodedShareUrl>`
|
|
||||||
|
|
||||||
Suggested default tweet text:
|
|
||||||
- `Listening to “{title}” in MOTIF — try it`
|
|
||||||
|
|
||||||
### B) Share landing: `/play`
|
|
||||||
A minimal landing page that:
|
|
||||||
- reads parameters from the URL
|
|
||||||
- fetches the MIDI (or exported artifact)
|
|
||||||
- enables Play/Stop + volume
|
|
||||||
- shows iOS “Enable Audio” CTA if required
|
|
||||||
- includes a primary CTA: **Generate your own** → links back to the main app
|
|
||||||
|
|
||||||
## URL design (heart of the plan)
|
|
||||||
|
|
||||||
### v1: share the original MIDI preview (recommended MVP)
|
|
||||||
Use the MIDI URL as the stable identifier.
|
|
||||||
|
|
||||||
Landing URL:
|
|
||||||
- `/play?u=<encodedMidiUrl>&title=<encodedTitle>&song=<encodedQuery>`
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
- `u` is required for reproducibility.
|
|
||||||
- `title` is display-only (optional).
|
|
||||||
- `song` supports the CTA back to the main app (optional).
|
|
||||||
|
|
||||||
Example:
|
|
||||||
- `/play?u=https%3A%2F%2Fbitmidi.com%2Fuploads%2F...mid&title=Hotel%20California&song=Hotel%20California`
|
|
||||||
|
|
||||||
Main app CTA target:
|
|
||||||
- `/?song=<encodedQuery>`
|
|
||||||
|
|
||||||
### Why not share “result index”?
|
|
||||||
Because search ranking shifts; index links rot. If needed, index can be a fallback only when `u` is missing.
|
|
||||||
|
|
||||||
## `/play` landing behavior
|
|
||||||
|
|
||||||
### Required inputs
|
|
||||||
- **Preview share**: `u` (MIDI source URL)
|
|
||||||
- **Export share**: `m` (motif midi id) or `a` (audio id)
|
|
||||||
|
|
||||||
### State machine
|
|
||||||
- If `m` or `a` present → load exported artifact
|
|
||||||
- Else if `u` present → fetch + parse MIDI via backend proxy
|
|
||||||
- Else → show “Invalid link” + CTA to home
|
|
||||||
|
|
||||||
### Playback
|
|
||||||
- Never autoplay.
|
|
||||||
- Enable Audio CTA shown until AudioContext is running.
|
|
||||||
|
|
||||||
### Error cases
|
|
||||||
- Missing/invalid params → show “Invalid link”
|
|
||||||
- Backend fetch failure → show “Couldn’t load this MIDI”
|
|
||||||
- Parse failure → show “Unsupported or corrupted MIDI”
|
|
||||||
|
|
||||||
## Export-based sharing (generated Motif variants)
|
|
||||||
Sharing a generated Motif variant needs an artifact the landing page can load reliably:
|
|
||||||
- **MIDI export**: smaller, fast, consistent with the project (music-as-structure)
|
|
||||||
- **Audio export**: most universal listening, but heavier and needs careful encoding/hosting
|
|
||||||
|
|
||||||
### Recommended sequencing
|
|
||||||
1) **MIDI export first** (fast, small, easy to iterate)\n2) Add **audio export** once the flow proves value (and storage/limits are solved)
|
|
||||||
|
|
||||||
### Proposed backend endpoints
|
|
||||||
|
|
||||||
#### 1) Create export (returns a shareable play URL)
|
|
||||||
`POST /api/share/export`
|
|
||||||
|
|
||||||
Input (example):
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"u": "https://bitmidi.com/uploads/....mid",
|
|
||||||
"song": "Hotel California",
|
|
||||||
"title": "Hotel California - Eagles",
|
|
||||||
"preset": "dance|ambient|ominous|default",
|
|
||||||
"params": { "intensity": 0.5, "swing": 0.1 },
|
|
||||||
"format": "midi|audio"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Output (example):
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"id": "abc123",
|
|
||||||
"format": "midi",
|
|
||||||
"playUrl": "/play?m=abc123&title=Hotel%20California&song=Hotel%20California"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2) Fetch export artifact
|
|
||||||
Option A (explicit):
|
|
||||||
- `GET /api/share/artifact?id=abc123` → returns bytes (MIDI or audio)
|
|
||||||
|
|
||||||
Option B (format-specific):
|
|
||||||
- `GET /api/share/midi?id=abc123`
|
|
||||||
- `GET /api/share/audio?id=abc123`
|
|
||||||
|
|
||||||
Landing params:
|
|
||||||
- MIDI artifact: `/play?m=<id>&title=...&song=...`
|
|
||||||
- Audio artifact: `/play?a=<id>&title=...&song=...`
|
|
||||||
|
|
||||||
### Storage strategy (dev vs production)
|
|
||||||
- **Local dev**: filesystem under `server/cache/exports/`
|
|
||||||
- **Vercel production**: filesystem is not durable; prefer:
|
|
||||||
- blob storage (recommended), or
|
|
||||||
- KV/object store (if blob not available), or
|
|
||||||
- a small database row referencing a blob key
|
|
||||||
|
|
||||||
### Security constraints (must-have)
|
|
||||||
- Validate `u` server-side:
|
|
||||||
- allow only `http`/`https`
|
|
||||||
- strongly consider allowlisting hosts (e.g. `bitmidi.com`) for v1
|
|
||||||
- Enforce limits:
|
|
||||||
- max MIDI bytes
|
|
||||||
- max parsed duration
|
|
||||||
- max export size
|
|
||||||
- max export time/CPU
|
|
||||||
- Rate-limit export endpoints.
|
|
||||||
- Cache and dedupe exports:
|
|
||||||
- key by `(sourceMidiHash + preset + params + format)` to avoid repeated work
|
|
||||||
|
|
||||||
## Files (expected) — implementation map
|
|
||||||
Frontend:
|
|
||||||
- `index.html`: Share UI elements
|
|
||||||
- `src/main.ts`: URL generation + clipboard + X intent
|
|
||||||
- `play.html` + `src/play.ts`: landing page UI + logic
|
|
||||||
- `vite.config.ts`: add `play.html` to multi-page inputs
|
|
||||||
|
|
||||||
Backend:
|
|
||||||
- `server/src/server.ts`: route handlers for export endpoints
|
|
||||||
- `server/src/services/*`: reuse parsing/generation services as needed
|
|
||||||
- `server/cache/exports/`: local dev artifact cache
|
|
||||||
|
|
||||||
Routing/deploy:
|
|
||||||
- `vercel.json`: add `/play` → `/play.html` similar to `/embed`
|
|
||||||
|
|
||||||
## Test plan
|
|
||||||
- Desktop:
|
|
||||||
- Share from main app → open `/play?...` in new tab → loads → plays after click
|
|
||||||
- Copy link works; X intent opens with correct URL
|
|
||||||
- iOS Safari:
|
|
||||||
- `/play` shows Enable Audio when needed
|
|
||||||
- Tap Enable Audio → tap Play → sound
|
|
||||||
- Export flow:
|
|
||||||
- Export MIDI variant → receive `/play?m=...` → plays the exported MIDI reliably
|
|
||||||
- Export audio variant (if implemented) → `/play?a=...` → plays reliably
|
|
||||||
|
|
||||||
## Success criteria
|
|
||||||
- A shared link reliably recreates “the thing” (preview or exported variant) and is playable with one tap.
|
|
||||||
- The system remains safe under abuse (no open proxy behavior, bounded cost).
|
|
||||||
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
# Save the social preview image as og-image.jpg in this directory
|
|
||||||
Binary file not shown.
|
Before Width: | Height: | Size: 178 KiB |
@@ -1,42 +0,0 @@
|
|||||||
console.log('Main script loading...');
|
|
||||||
|
|
||||||
document.addEventListener('DOMContentLoaded', function() {
|
|
||||||
console.log('DOM loaded');
|
|
||||||
|
|
||||||
const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement;
|
|
||||||
const songInput = document.getElementById('songInput') as HTMLInputElement;
|
|
||||||
const status = document.getElementById('status')!;
|
|
||||||
|
|
||||||
if (!searchBtn || !songInput || !status) {
|
|
||||||
console.error('UI elements not found!');
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('UI elements found');
|
|
||||||
|
|
||||||
searchBtn.addEventListener('click', async function() {
|
|
||||||
console.log('Search button clicked!');
|
|
||||||
|
|
||||||
const songName = songInput.value.trim();
|
|
||||||
if (!songName) return;
|
|
||||||
|
|
||||||
status.textContent = 'Searching...';
|
|
||||||
searchBtn.disabled = true;
|
|
||||||
|
|
||||||
try {
|
|
||||||
const response = await fetch(`http://localhost:3001/api/midi/search?q=${encodeURIComponent(songName)}`);
|
|
||||||
const data = await response.json();
|
|
||||||
|
|
||||||
console.log('Search results:', data);
|
|
||||||
status.textContent = `Found ${data.count} results: ${data.results.map((r: any) => r.title).join(', ')}`;
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Search error:', error);
|
|
||||||
status.textContent = `Search error: ${error}`;
|
|
||||||
} finally {
|
|
||||||
searchBtn.disabled = false;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
console.log('Event listeners attached');
|
|
||||||
});
|
|
||||||
@@ -1,189 +0,0 @@
|
|||||||
console.log('Full Motif app loading...');
|
|
||||||
|
|
||||||
interface SearchResult {
|
|
||||||
id: string;
|
|
||||||
title: string;
|
|
||||||
source: string;
|
|
||||||
pageUrl: string;
|
|
||||||
midiUrl: string;
|
|
||||||
confidence: number;
|
|
||||||
}
|
|
||||||
|
|
||||||
class MotifApp {
|
|
||||||
private searchResults: SearchResult[] = [];
|
|
||||||
private selectedIndex = 0;
|
|
||||||
|
|
||||||
constructor() {
|
|
||||||
document.addEventListener('DOMContentLoaded', () => {
|
|
||||||
this.initializeUI();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
private initializeUI(): void {
|
|
||||||
console.log('Initializing UI...');
|
|
||||||
|
|
||||||
const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement;
|
|
||||||
const songInput = document.getElementById('songInput') as HTMLInputElement;
|
|
||||||
|
|
||||||
if (!searchBtn || !songInput) {
|
|
||||||
console.error('UI elements not found!');
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
searchBtn.addEventListener('click', () => this.handleSearch());
|
|
||||||
songInput.addEventListener('keypress', (e) => {
|
|
||||||
if (e.key === 'Enter') this.handleSearch();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Make selectResult globally available for onclick handlers
|
|
||||||
(window as any).app = this;
|
|
||||||
|
|
||||||
console.log('UI initialized successfully');
|
|
||||||
}
|
|
||||||
|
|
||||||
private async handleSearch(): Promise<void> {
|
|
||||||
const songInput = document.getElementById('songInput') as HTMLInputElement;
|
|
||||||
const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement;
|
|
||||||
const status = document.getElementById('status')!;
|
|
||||||
|
|
||||||
const songName = songInput.value.trim();
|
|
||||||
if (!songName) return;
|
|
||||||
|
|
||||||
status.textContent = 'Searching for MIDI files...';
|
|
||||||
searchBtn.disabled = true;
|
|
||||||
|
|
||||||
try {
|
|
||||||
const response = await fetch(`http://localhost:3001/api/midi/search?q=${encodeURIComponent(songName)}`);
|
|
||||||
const data = await response.json();
|
|
||||||
|
|
||||||
console.log('Search results:', data);
|
|
||||||
|
|
||||||
if (data.results.length === 0) {
|
|
||||||
status.textContent = 'No MIDI files found. Try a different search.';
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
this.searchResults = data.results;
|
|
||||||
this.displayResults();
|
|
||||||
status.textContent = `Found ${data.results.length} MIDI files. Select one to play.`;
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Search error:', error);
|
|
||||||
status.textContent = `Search error: ${error}`;
|
|
||||||
} finally {
|
|
||||||
searchBtn.disabled = false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private displayResults(): void {
|
|
||||||
const resultsSection = document.getElementById('resultsSection')!;
|
|
||||||
const resultsBody = document.getElementById('resultsBody')!;
|
|
||||||
|
|
||||||
resultsBody.innerHTML = '';
|
|
||||||
|
|
||||||
this.searchResults.forEach((result, index) => {
|
|
||||||
const row = document.createElement('tr');
|
|
||||||
if (index === this.selectedIndex) {
|
|
||||||
row.classList.add('selected');
|
|
||||||
}
|
|
||||||
|
|
||||||
row.innerHTML = `
|
|
||||||
<td>${result.title}</td>
|
|
||||||
<td>${result.source}</td>
|
|
||||||
<td>
|
|
||||||
<div class="confidence-bar">
|
|
||||||
<div class="confidence-fill" style="width: ${result.confidence * 100}%"></div>
|
|
||||||
</div>
|
|
||||||
</td>
|
|
||||||
<td>?</td>
|
|
||||||
<td>?</td>
|
|
||||||
<td></td>
|
|
||||||
<td><button onclick="window.app.selectResult(${index})">Select</button></td>
|
|
||||||
`;
|
|
||||||
|
|
||||||
resultsBody.appendChild(row);
|
|
||||||
});
|
|
||||||
|
|
||||||
resultsSection.classList.add('visible');
|
|
||||||
|
|
||||||
// Auto-select first result
|
|
||||||
if (this.searchResults.length > 0) {
|
|
||||||
this.selectResult(0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public async selectResult(index: number): Promise<void> {
|
|
||||||
if (index < 0 || index >= this.searchResults.length) return;
|
|
||||||
|
|
||||||
this.selectedIndex = index;
|
|
||||||
const result = this.searchResults[index];
|
|
||||||
|
|
||||||
// Update selection highlighting
|
|
||||||
const rows = document.querySelectorAll('#resultsBody tr');
|
|
||||||
rows.forEach((row, i) => {
|
|
||||||
row.classList.toggle('selected', i === index);
|
|
||||||
});
|
|
||||||
|
|
||||||
const status = document.getElementById('status')!;
|
|
||||||
const playerSection = document.getElementById('playerSection')!;
|
|
||||||
const selectedTitle = document.getElementById('selectedTitle')!;
|
|
||||||
const selectedMeta = document.getElementById('selectedMeta')!;
|
|
||||||
|
|
||||||
status.textContent = 'Loading MIDI file...';
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Fetch MIDI data
|
|
||||||
const response = await fetch(`http://localhost:3001/api/midi/fetch?u=${encodeURIComponent(result.midiUrl)}`);
|
|
||||||
|
|
||||||
if (!response.ok) {
|
|
||||||
throw new Error(`Failed to fetch MIDI: ${response.status}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
const midiBuffer = await response.arrayBuffer();
|
|
||||||
|
|
||||||
// Update UI
|
|
||||||
selectedTitle.textContent = result.title;
|
|
||||||
selectedMeta.innerHTML = `
|
|
||||||
<strong>Source:</strong> ${result.source} |
|
|
||||||
<strong>Size:</strong> ${(midiBuffer.byteLength / 1024).toFixed(1)}KB |
|
|
||||||
<strong>Confidence:</strong> ${Math.round(result.confidence * 100)}%
|
|
||||||
`;
|
|
||||||
|
|
||||||
playerSection.classList.add('visible');
|
|
||||||
|
|
||||||
// Enable preview button
|
|
||||||
const previewBtn = document.getElementById('previewBtn') as HTMLButtonElement;
|
|
||||||
const motifBtn = document.getElementById('motifBtn') as HTMLButtonElement;
|
|
||||||
previewBtn.disabled = false;
|
|
||||||
motifBtn.disabled = false;
|
|
||||||
|
|
||||||
// Store MIDI data for playback
|
|
||||||
(this as any).currentMIDI = { buffer: midiBuffer, result };
|
|
||||||
|
|
||||||
status.textContent = 'MIDI loaded. You can now preview or generate synthesis.';
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Load error:', error);
|
|
||||||
status.textContent = `Load error: ${error}`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public async handlePreview(): Promise<void> {
|
|
||||||
console.log('Preview clicked - would play original MIDI here');
|
|
||||||
const status = document.getElementById('status')!;
|
|
||||||
status.textContent = 'Preview playback not yet implemented - but MIDI is loaded!';
|
|
||||||
}
|
|
||||||
|
|
||||||
public async handleMotif(): Promise<void> {
|
|
||||||
console.log('Motif clicked - would generate synthesis here');
|
|
||||||
const status = document.getElementById('status')!;
|
|
||||||
status.textContent = 'Motif synthesis not yet implemented - but MIDI is parsed!';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Initialize app
|
|
||||||
new MotifApp();
|
|
||||||
|
|
||||||
// Expose handlers for buttons
|
|
||||||
(window as any).handlePreview = () => (window as any).app.handlePreview();
|
|
||||||
(window as any).handleMotif = () => (window as any).app.handleMotif();
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
<!DOCTYPE html>
|
|
||||||
<html>
|
|
||||||
<head>
|
|
||||||
<title>Test</title>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<button id="testBtn">Test Button</button>
|
|
||||||
<div id="output">Not loaded</div>
|
|
||||||
|
|
||||||
<script>
|
|
||||||
console.log('JavaScript is loading...');
|
|
||||||
document.addEventListener('DOMContentLoaded', function() {
|
|
||||||
console.log('DOM loaded');
|
|
||||||
document.getElementById('output').textContent = 'JavaScript loaded!';
|
|
||||||
document.getElementById('testBtn').addEventListener('click', function() {
|
|
||||||
alert('Button works!');
|
|
||||||
});
|
|
||||||
});
|
|
||||||
</script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
# MOTIF Project - Thursday Status Report
|
|
||||||
|
|
||||||
## Project Overview
|
|
||||||
|
|
||||||
**MOTIF** is a procedural music synthesis system that extracts structural information from existing MIDI files and recreates them as original, real-time audio using Web Audio API. The core concept: "music as executable structure, not static audio."
|
|
||||||
|
|
||||||
### Workflow
|
|
||||||
1. User searches for a song by name
|
|
||||||
2. System finds MIDI files from multiple sources (BitMidi, Dongrays)
|
|
||||||
3. MIDI is parsed and analyzed for structural features (tempo, density, melodic patterns)
|
|
||||||
4. Notes are mapped to synthesis "roles" (bass, drone, ostinato, texture, accents)
|
|
||||||
5. Web Audio API generates procedural audio with similar "feel" but original sound
|
|
||||||
|
|
||||||
## Current Status: Functional MVP
|
|
||||||
|
|
||||||
### ✅ Completed Features
|
|
||||||
|
|
||||||
**Complete Search Pipeline**
|
|
||||||
- Multi-source MIDI search with confidence scoring
|
|
||||||
- Real MIDI integration with fetching and parsing
|
|
||||||
- Graceful error handling and timeouts
|
|
||||||
|
|
||||||
**Role-Based Synthesis Engine**
|
|
||||||
- Intelligent mapping of MIDI tracks to synthesis layers
|
|
||||||
- Role-specific oscillator types and filtering
|
|
||||||
- Velocity-sensitive ADSR envelopes
|
|
||||||
- Polyphonic chord support
|
|
||||||
- Automatic looping and proper cleanup
|
|
||||||
|
|
||||||
**Polished User Interface**
|
|
||||||
- Search results table with confidence bars and quality analysis
|
|
||||||
- Dual player UI: Preview Original MIDI vs Generate Motif
|
|
||||||
- Real-time status updates and progress feedback
|
|
||||||
- "Try Next Result" workflow for easy A/B testing
|
|
||||||
|
|
||||||
**Backend Infrastructure**
|
|
||||||
- Express + TypeScript server
|
|
||||||
- CORS proxy with validation and SHA256 disk caching
|
|
||||||
- Multi-source search (BitMidi, Dongrays, synthetic fallback)
|
|
||||||
- Quality assessment with penalties for problematic content
|
|
||||||
|
|
||||||
### 📈 Recent Progress (Latest Commit)
|
|
||||||
|
|
||||||
**Major UI/UX Improvements:**
|
|
||||||
- Implemented search results table with metadata display
|
|
||||||
- Added MIDI preview player with basic oscillator mapping
|
|
||||||
- Built dual transport controls for comparison
|
|
||||||
- Integrated ParsedMIDIInfo with comprehensive track analysis
|
|
||||||
- Enhanced confidence scoring system with quality penalties
|
|
||||||
|
|
||||||
**Technical Enhancements:**
|
|
||||||
- Sophisticated role mapping with pitch range and density analysis
|
|
||||||
- Improved error handling across the pipeline
|
|
||||||
- Better synthesis scheduling with Web Audio lookahead
|
|
||||||
- Streamlined search-to-synthesis workflow
|
|
||||||
|
|
||||||
## Current Problems & Limitations
|
|
||||||
|
|
||||||
### 🔴 Performance Issues
|
|
||||||
- **~70% search success rate** (goal: >90%)
|
|
||||||
- **~30% musical similarity recognition** (goal: >70%)
|
|
||||||
- HTML regex parsing is fragile (should use DOM parsing)
|
|
||||||
|
|
||||||
### 🟡 Feature Limitations
|
|
||||||
- Basic role mapping heuristics (lacks harmonic analysis)
|
|
||||||
- Simple synthesis timbres (basic oscillators only)
|
|
||||||
- Limited MIDI source coverage
|
|
||||||
- No user controls for synthesis parameters
|
|
||||||
|
|
||||||
### 🟠 Technical Debt
|
|
||||||
- Need more robust parsing for edge cases
|
|
||||||
- Search confidence scoring could be more sophisticated
|
|
||||||
- Some synthesis roles need refinement
|
|
||||||
|
|
||||||
## Next Sprint Priorities
|
|
||||||
|
|
||||||
### Phase 1: Core Stability (Next 1-2 weeks)
|
|
||||||
1. **Improve search success rate** - better error handling, additional sources
|
|
||||||
2. **Enhance role mapping** - add harmonic analysis, rhythm detection
|
|
||||||
3. **Polish synthesis** - more interesting timbres, dynamic control
|
|
||||||
4. **Robust parsing** - replace regex with proper DOM parsing
|
|
||||||
|
|
||||||
### Phase 2: Musical Intelligence (2-4 weeks)
|
|
||||||
1. **Smarter scoring** - melodic similarity, harmonic progression analysis
|
|
||||||
2. **Advanced synthesis** - effects, modulation, realistic instruments
|
|
||||||
3. **User controls** - synthesis parameter adjustment, role customization
|
|
||||||
4. **More MIDI sources** - expand search coverage
|
|
||||||
|
|
||||||
### Phase 3: Production Ready (1-2 months)
|
|
||||||
1. **Performance optimization** - caching, preloading, worker threads
|
|
||||||
2. **Legal compliance** - proper attribution, copyright handling
|
|
||||||
3. **User uploads** - allow custom MIDI file analysis
|
|
||||||
4. **Production infrastructure** - deployment, monitoring, scaling
|
|
||||||
|
|
||||||
## Technical Architecture
|
|
||||||
|
|
||||||
**Backend:** Express + TypeScript with multi-source search, CORS proxy, and caching
|
|
||||||
**Frontend:** TypeScript + Vite + Web Audio with real-time synthesis
|
|
||||||
**Key Components:** MotifEngine, RoleMapper, SynthesisEngine, MIDIPlayer
|
|
||||||
|
|
||||||
## Demo Status
|
|
||||||
|
|
||||||
✅ **Ready to demonstrate** - Full end-to-end pipeline functional
|
|
||||||
✅ **User-friendly interface** - Polished search and playback experience
|
|
||||||
✅ **Comparative validation** - Side-by-side original vs synthesis preview
|
|
||||||
|
|
||||||
The project successfully proves the core concept and is ready for user testing and iterative improvement.
|
|
||||||
Reference in New Issue
Block a user