Clean up repo for public release
Removed: - Old planning docs (DEPLOYMENT.md, IMPLEMENTATION.md, etc.) - Dev notes (thursday-aims-progress-problems.md) - Old backup files (main-simple.ts, main-working.ts) - Test file (test.html) - Unused image (chiptune_blog_piece_1000x.webp) - docs/ folder with old audit/plan docs Updated: - Renamed claude.md to CLAUDE.md - Added .claude/ to gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
b1rdmania
@@ -34,6 +34,7 @@ logs/
|
||||
.cache/
|
||||
.tmp/
|
||||
.vercel
|
||||
.claude/
|
||||
|
||||
# Backend runtime cache (downloaded MIDI files)
|
||||
server/cache/
|
||||
|
||||
-102
@@ -1,102 +0,0 @@
|
||||
# Deployment Guide for Motif
|
||||
|
||||
## iOS Fixes Applied
|
||||
- ✓ Fixed auto-zoom on input focus (font-size: 16px, viewport locked)
|
||||
- ✓ Prevented zoom on touch (maximum-scale=1.0, user-scalable=no)
|
||||
|
||||
## Backend Deployment
|
||||
|
||||
The MIDI search requires a backend server to be running. You have two options:
|
||||
|
||||
### Option 1: Deploy Backend Separately (Recommended for Production)
|
||||
|
||||
1. Deploy the backend to a service like Render, Railway, or Heroku:
|
||||
```bash
|
||||
cd server
|
||||
npm install
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
2. Set the `VITE_API_URL` environment variable in your frontend deployment to point to your backend:
|
||||
```
|
||||
VITE_API_URL=https://your-backend-url.com
|
||||
```
|
||||
|
||||
### Option 2: Run Backend Locally (Development Only)
|
||||
|
||||
1. In one terminal, start the backend:
|
||||
```bash
|
||||
npm run dev:backend
|
||||
```
|
||||
|
||||
2. In another terminal, start the frontend:
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
## Vercel Deployment
|
||||
|
||||
### Deploy Frontend to Vercel:
|
||||
|
||||
1. Build the project:
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
2. Deploy to Vercel:
|
||||
```bash
|
||||
vercel --prod
|
||||
```
|
||||
|
||||
3. Set environment variable in Vercel dashboard:
|
||||
- Variable name: `VITE_API_URL`
|
||||
- Value: Your deployed backend URL (e.g., `https://motif-backend.onrender.com`)
|
||||
|
||||
### Deploy Backend to Render/Railway:
|
||||
|
||||
1. Create a new Web Service
|
||||
2. Connect your GitHub repository
|
||||
3. Set build command: `cd server && npm install && npm run build`
|
||||
4. Set start command: `cd server && npm start`
|
||||
5. Add environment variable `PORT` (usually auto-set)
|
||||
6. Deploy
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Create a `.env` file in the root directory for local development:
|
||||
|
||||
```bash
|
||||
VITE_API_URL=http://localhost:3001
|
||||
```
|
||||
|
||||
For production, set this in your deployment platform (Vercel, Netlify, etc.):
|
||||
|
||||
```bash
|
||||
VITE_API_URL=https://your-backend-api-url.com
|
||||
```
|
||||
|
||||
## Testing the Deployment
|
||||
|
||||
1. Open the deployed URL on iOS Safari
|
||||
2. Search for a song (e.g., "Hotel California")
|
||||
3. You should see MIDI results load
|
||||
4. Select a result and play it
|
||||
5. The page should NOT zoom when touching the search input
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No search results on production:
|
||||
- Check that backend is running (visit `https://your-backend-url.com/health`)
|
||||
- Verify `VITE_API_URL` environment variable is set correctly in Vercel
|
||||
- Check browser console for CORS errors
|
||||
- Rebuild frontend after setting environment variables
|
||||
|
||||
### iOS zoom issue persists:
|
||||
- Clear Safari cache
|
||||
- Hard reload the page
|
||||
- Check that the latest build is deployed
|
||||
|
||||
### CORS errors:
|
||||
- Backend must allow requests from your frontend domain
|
||||
- Check server CORS configuration in `server/src/server.ts`
|
||||
@@ -1,251 +0,0 @@
|
||||
# MOTIF Implementation Status & Roadmap
|
||||
|
||||
**Current Status**: MVP functional with real MIDI search, parsing, and procedural synthesis
|
||||
|
||||
---
|
||||
|
||||
## ✅ What's Been Built
|
||||
|
||||
### Backend API (Express + TypeScript)
|
||||
|
||||
**Endpoints:**
|
||||
- `GET /api/midi/search?q=song` - Multi-source MIDI search
|
||||
- `GET /api/midi/fetch?u=url` - CORS proxy with validation and caching
|
||||
- `GET /health` - Service health check
|
||||
|
||||
**Architecture:**
|
||||
```
|
||||
server/src/
|
||||
├── adapters/ # Search source implementations
|
||||
│ ├── BitMidiAdapter # HTML parsing for bitmidi.com
|
||||
│ └── DongraysAdapter # HTML parsing for dongrays.net
|
||||
├── services/
|
||||
│ ├── MIDISearchService # Orchestrates multi-source search
|
||||
│ └── MIDIFetchService # Downloads, validates, caches MIDI
|
||||
└── utils/
|
||||
└── ScoreUtils # Confidence scoring & quality assessment
|
||||
```
|
||||
|
||||
**Key Features:**
|
||||
- **Confidence Scoring**: Token matching, quality penalties (karaoke, broken files)
|
||||
- **Disk Caching**: SHA256-hashed files with JSON index
|
||||
- **Validation**: MIDI header checks, file size limits (10MB max)
|
||||
- **Error Handling**: Timeouts, graceful fallbacks
|
||||
- **Deduplication**: Removes duplicate results across sources
|
||||
|
||||
### Frontend (TypeScript + Vite + Web Audio)
|
||||
|
||||
**Architecture:**
|
||||
```
|
||||
src/
|
||||
├── core/
|
||||
│ ├── MotifEngine # Main orchestrator
|
||||
│ └── RoleMapper # MIDI track → synthesis role assignment
|
||||
├── midi/
|
||||
│ ├── MIDIProcessor # Feature extraction (tempo, density, etc.)
|
||||
│ └── MIDIParser # @tonejs/midi wrapper
|
||||
├── synthesis/
|
||||
│ └── SynthesisEngine # Pure Web Audio procedural synthesis
|
||||
├── services/
|
||||
│ └── MIDIService # Backend API client
|
||||
└── types/
|
||||
└── index # TypeScript interfaces
|
||||
```
|
||||
|
||||
**Synthesis Pipeline:**
|
||||
```
|
||||
Song Name → MIDI Search → Parse Events → Role Assignment → Web Audio Synthesis
|
||||
```
|
||||
|
||||
### Integration Flow
|
||||
|
||||
1. **User enters song name** → `MotifEngine.generateFromSong()`
|
||||
2. **MIDI Search** → `MIDIService.search()` → Backend `/search` endpoint
|
||||
3. **Multi-source search** → BitMidi + Dongrays adapters in parallel
|
||||
4. **Result ranking** → Confidence scoring, deduplication
|
||||
5. **MIDI Fetch** → `MIDIService.fetchMIDI()` → Backend `/fetch` with caching
|
||||
6. **MIDI Parsing** → `@tonejs/midi` → Normalized `NoteEvent[]` array
|
||||
7. **Role Assignment** → `RoleMapper` → Bass/Drone/Ostinato/Texture/Accents
|
||||
8. **Web Audio Synthesis** → `SynthesisEngine` → Real-time procedural audio
|
||||
|
||||
---
|
||||
|
||||
## 🔧 How It Actually Works
|
||||
|
||||
### MIDI Resolution Strategy
|
||||
|
||||
**Sources (MVP):**
|
||||
- **BitMidi**: Regex parsing of search results, direct `.mid` links
|
||||
- **Dongrays**: Similar approach, handles download endpoints
|
||||
- **Synthetic Fallback**: Hash-based procedural generation if search fails
|
||||
|
||||
**Scoring Heuristics:**
|
||||
- Token matching between query and title
|
||||
- Penalties for "karaoke", "vocal", "broken"
|
||||
- Bonus for direct `.mid` links
|
||||
- Source preference (BitMidi slightly favored)
|
||||
|
||||
### Role-Based Synthesis
|
||||
|
||||
**Role Assignment:**
|
||||
```typescript
|
||||
// Heuristic rules:
|
||||
pitch < 48 + short notes = Bass
|
||||
long duration > 2s = Drone
|
||||
short + repetitive = Ostinato
|
||||
high velocity = Accents
|
||||
everything else = Texture
|
||||
```
|
||||
|
||||
**Synthesis Per Role:**
|
||||
- **Bass**: Square wave, lowpass filter, punchy envelopes
|
||||
- **Drone**: Sawtooth, bandpass, sustained notes
|
||||
- **Ostinato**: Triangle, highpass, rhythmic patterns
|
||||
- **Texture**: Sine, bandpass, atmospheric
|
||||
- **Accents**: Sine, peaking filter, sharp attacks
|
||||
|
||||
**Web Audio Implementation:**
|
||||
- Lookahead scheduling (100ms)
|
||||
- MIDI note → Hz conversion: `440 * 2^((note-69)/12)`
|
||||
- Velocity-sensitive envelopes
|
||||
- Automatic looping when MIDI ends
|
||||
- Per-note oscillator + gain envelope
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Current Limitations
|
||||
|
||||
### Search Quality
|
||||
- **HTML Parsing**: Fragile regex-based extraction (not DOM parsing)
|
||||
- **Limited Sources**: Only 2 sources, no fallbacks if both fail
|
||||
- **No Metadata**: Can't validate artist, album, year matching
|
||||
- **Rate Limiting**: No request throttling or backoff
|
||||
|
||||
### MIDI Processing
|
||||
- **Simple Role Mapping**: Basic pitch/duration heuristics only
|
||||
- **No Harmonic Analysis**: Doesn't understand chord progressions
|
||||
- **Track Correlation**: Doesn't detect melody vs accompaniment intelligently
|
||||
- **Tempo Handling**: Assumes constant tempo, ignores tempo changes
|
||||
|
||||
### Synthesis Engine
|
||||
- **Basic Timbres**: Simple oscillator types, no complex synthesis
|
||||
- **No Dynamics**: Volume levels are role-based, not musically aware
|
||||
- **Limited Effects**: Only basic filtering, no reverb/chorus/etc.
|
||||
- **Monophonic Layers**: Each role plays one note at a time
|
||||
|
||||
### Frontend UX
|
||||
- **No Progress Feedback**: Search/fetch happens in black box
|
||||
- **No Result Preview**: Can't see what MIDI was found before synthesis
|
||||
- **No Controls**: Can't adjust synthesis parameters
|
||||
- **Error Messages**: Generic error handling
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Next Steps (Prioritized)
|
||||
|
||||
### Phase 1: Polish MVP
|
||||
**Goal**: Make current system reliable and user-friendly
|
||||
|
||||
1. **Better Error Handling**
|
||||
- Show search progress ("Searching BitMidi...", "Parsing MIDI...")
|
||||
- Display actual MIDI file found before synthesis
|
||||
- Graceful degradation with informative messages
|
||||
|
||||
2. **Improve Role Mapping**
|
||||
- Add harmonic analysis (detect bass lines, chord patterns)
|
||||
- Use track names/MIDI program changes as hints
|
||||
- Smarter melody vs accompaniment detection
|
||||
|
||||
3. **Synthesis Polish**
|
||||
- Add polyphony within roles (chords, multiple bass notes)
|
||||
- Better envelopes (ADSR with release tails)
|
||||
- Basic effects (simple reverb, subtle filtering LFOs)
|
||||
|
||||
### Phase 2: Search Enhancement
|
||||
**Goal**: Higher success rate finding good MIDIs
|
||||
|
||||
4. **Robust Parsing**
|
||||
- Switch to Cheerio for proper DOM parsing
|
||||
- Handle dynamic content/JavaScript-loaded results
|
||||
- Add more MIDI sources (MuseScore, IMSLP public domain)
|
||||
|
||||
5. **Smarter Scoring**
|
||||
- Artist name matching with fuzzy string comparison
|
||||
- Duration validation (reject 30-second clips, 20-minute symphonies)
|
||||
- Key signature and time signature analysis
|
||||
|
||||
6. **Caching & Performance**
|
||||
- Cache search results (not just MIDI files)
|
||||
- Add request deduplication and rate limiting
|
||||
- Background refresh of popular files
|
||||
|
||||
### Phase 3: Synthesis Sophistication
|
||||
**Goal**: More recognizable and musical output
|
||||
|
||||
7. **Advanced Synthesis**
|
||||
- Multiple synthesis modes per role (subtractive, FM, additive)
|
||||
- Tempo-synced effects and modulation
|
||||
- Cross-role interaction (bass and drums lock together)
|
||||
|
||||
8. **Musical Intelligence**
|
||||
- Detect and preserve harmonic progressions
|
||||
- Rhythmic pattern extraction and variation
|
||||
- Dynamic arrangement (intro/verse/chorus detection)
|
||||
|
||||
9. **User Controls**
|
||||
- Synthesis parameter sliders (brightness, warmth, density)
|
||||
- Role muting/soloing
|
||||
- Tempo adjustment and time-stretching
|
||||
|
||||
### Phase 4: Production Ready
|
||||
**Goal**: Reliable service for real users
|
||||
|
||||
10. **Infrastructure**
|
||||
- Database for MIDI metadata and search caching
|
||||
- CDN for popular MIDI files
|
||||
- Analytics and error monitoring
|
||||
|
||||
11. **Legal & Content**
|
||||
- MIDI license validation
|
||||
- User-uploaded MIDI support
|
||||
- Integration with Creative Commons sources
|
||||
|
||||
---
|
||||
|
||||
## 🔬 Technical Debt
|
||||
|
||||
### Immediate
|
||||
- Remove `crypto` dependency warning in backend package.json
|
||||
- Add proper TypeScript strict mode compliance
|
||||
- Implement proper error boundaries in frontend
|
||||
|
||||
### Medium Term
|
||||
- Replace regex HTML parsing with proper DOM parsing
|
||||
- Add comprehensive logging/telemetry
|
||||
- Write unit tests for core algorithms (role mapping, scoring)
|
||||
|
||||
### Long Term
|
||||
- Consider WebAssembly for intensive audio processing
|
||||
- Evaluate Web Workers for MIDI parsing/analysis
|
||||
- Implement WebRTC for real-time collaboration features
|
||||
|
||||
---
|
||||
|
||||
## 📊 Success Metrics
|
||||
|
||||
**Current State**:
|
||||
- ✅ Searches return results ~70% of time
|
||||
- ✅ Successfully parses most MIDI files found
|
||||
- ✅ Generates audio output 100% of time (with fallback)
|
||||
- ⚠️ Output recognizably similar to input ~30% of time
|
||||
|
||||
**Target State**:
|
||||
- 🎯 Search success rate >90%
|
||||
- 🎯 Musical similarity recognition >70%
|
||||
- 🎯 User "that sounds like the song" reaction >60%
|
||||
- 🎯 Sub-3-second generation time 95% of requests
|
||||
|
||||
---
|
||||
|
||||
**Built**: Functional end-to-end MVP with real MIDI integration
|
||||
**Next**: Polish the core experience before expanding features
|
||||
@@ -1,163 +0,0 @@
|
||||
## MIDI playback MVP — detailed technical integration plan
|
||||
|
||||
### Product target (MVP)
|
||||
- **User flow**:
|
||||
1) User types a song name (e.g. “Hotel California”)
|
||||
2) App searches MIDI sources and shows a ranked list
|
||||
3) User selects a result and can **play the MIDI “correctly”** using **General MIDI soundfonts**
|
||||
4) User can then click **Generate Motif** to synthesize a “similar-but-different” version from the same parsed MIDI
|
||||
|
||||
### Current code reality (gaps to close)
|
||||
- **Search is currently biased toward mock/synthetic**, not real MIDI:
|
||||
- `server/src/services/MIDISearchService.ts` includes `MockAdapter` first.
|
||||
- **Fetch can silently replace real URLs with synthetic MIDI**:
|
||||
- `server/src/services/MIDIFetchService.ts` generates synthetic MIDI when URL contains `bitmidi.com/uploads`, preventing true BitMidi playback.
|
||||
- **Frontend preview is oscillator-based**, not GM soundfont playback.
|
||||
|
||||
---
|
||||
|
||||
## Architecture (what we’ll ship)
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
User -->|typesQuery| FrontendUI
|
||||
FrontendUI -->|GET /api/midi/search?q=...| BackendSearch
|
||||
BackendSearch -->|rankedResults| FrontendUI
|
||||
FrontendUI -->|selectResult + GET /api/midi/fetch?u=...| BackendFetch
|
||||
BackendFetch -->|midiBytes| FrontendParse
|
||||
FrontendParse -->|NoteEvents + TrackMeta| PreviewPlayerGM
|
||||
FrontendParse -->|NoteEvents| MotifEngine
|
||||
MotifEngine -->|roles + chords| SynthesisEngine
|
||||
PreviewPlayerGM --> AudioOut
|
||||
SynthesisEngine --> AudioOut
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — “Real MIDI mode” defaults (backend hardening)
|
||||
|
||||
### 0.1 Gate mock adapter behind env flag
|
||||
- **Change**: In `server/src/services/MIDISearchService.ts`, make adapters:
|
||||
- Default: `[BitMidiAdapter, DongraysAdapter]`
|
||||
- Optional: prepend `MockAdapter` only if `USE_MOCK_ADAPTER=1` (or similar)
|
||||
- **Acceptance**:
|
||||
- Searching “Hotel California” returns **non-`synthetic:*`** results when internet is available.
|
||||
- Devs can still run offline with `USE_MOCK_ADAPTER=1`.
|
||||
|
||||
### 0.2 Stop auto-synth overriding real URLs in fetch
|
||||
- **Change**: In `server/src/services/MIDIFetchService.ts`, only synthesize when:
|
||||
- `url.startsWith('synthetic:')` (and optionally if `USE_SYNTHETIC_FETCH=1`)
|
||||
- **Remove**: `url.includes('bitmidi.com/uploads')` synthetic shortcut
|
||||
- **Acceptance**:
|
||||
- Selecting a BitMidi result triggers a real network fetch and caches the real bytes.
|
||||
- If the remote file is invalid, it fails explicitly with a clear error.
|
||||
|
||||
### 0.3 Make backend behavior explicit in responses (optional but recommended)
|
||||
- **Change**: Add response fields or headers indicating source:
|
||||
- Example: `X-Motif-Source: real|synthetic|cache`
|
||||
- **Acceptance**:
|
||||
- Frontend can display “cached”/“live”/“synthetic fallback” badges (helps debugging + trust).
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — GM soundfont playback (frontend “Preview” becomes correct MIDI playback)
|
||||
|
||||
### 1.1 Dependency choice & asset strategy
|
||||
- **Dependency**: add a browser-friendly soundfont player dependency (e.g. `soundfont-player`).
|
||||
- **Soundfont hosting**:
|
||||
- Prefer static hosting under `/public/soundfonts/` (versioned with the app)
|
||||
- Or use a CDN, but pin versions and handle CORS
|
||||
- **MVP instrument set**:
|
||||
- Minimum viable: **Acoustic Grand Piano** for all melodic tracks, and a basic drum fallback
|
||||
- Better: load instruments on demand per track program
|
||||
|
||||
### 1.2 Implement `SoundfontMIDIPlayer`
|
||||
Create `src/synthesis/SoundfontMIDIPlayer.ts` with:
|
||||
- **Responsibilities**
|
||||
- Load instruments (program → soundfont instrument name)
|
||||
- Schedule note-on/note-off with WebAudio timing
|
||||
- Provide `load(midi)` / `play()` / `stop()` / `setVolume()` APIs
|
||||
- **Inputs**
|
||||
- Best: use `@tonejs/midi`’s `Midi` object (tracks include `instrument.number`, `notes`, `channel`)
|
||||
- Alternate: keep using `NoteEvent[]`, but you’ll lose program/channel unless you extend the event model
|
||||
- **Timing correctness**
|
||||
- Use seconds-based timing from `@tonejs/midi` notes (`time`, `duration` are in seconds)
|
||||
- Ensure AudioContext resumes on user gesture
|
||||
- **Drums**
|
||||
- If channel 9/10 is detected: either map to a percussion kit if supported, or skip drums for MVP (but be explicit in UI)
|
||||
|
||||
### 1.3 Wire UI to use soundfont preview
|
||||
- In `src/main.ts`, replace/augment the current oscillator `MIDIPlayer` usage:
|
||||
- Preview buttons should play via `SoundfontMIDIPlayer`
|
||||
- Keep existing Motif buttons intact
|
||||
- Keep oscillator preview only as a fallback if soundfonts fail to load (optional).
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — “Play the real song correctly” UX loop
|
||||
|
||||
### 2.1 Search UX requirements
|
||||
- **Result list must show**:
|
||||
- title, source, confidence
|
||||
- parsed metadata: duration, track count, issues
|
||||
- **Selection behavior**:
|
||||
- Selecting a row fetches + parses once; enables Preview + Generate Motif
|
||||
|
||||
### 2.2 Error handling (user-facing)
|
||||
Define user-facing error classes/messages:
|
||||
- **Search**: “No results”, “Backend unavailable”, “Rate limited / source blocked”
|
||||
- **Fetch**: “MIDI file blocked”, “Invalid MIDI header”, “Quality rejected”
|
||||
- **Parse**: “Unsupported MIDI features” / “Parse failed”
|
||||
- **Preview playback**: “Soundfont failed to load” / “Audio not allowed until click”
|
||||
|
||||
### 2.3 Observability (dev-facing)
|
||||
- Backend: log adapter failures per source + timings
|
||||
- Frontend: log selected MIDI URL, parse duration, instrument load times
|
||||
- Optional: a small “Debug” accordion showing chosen URL, cache hit, parse issues, loaded instruments
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Motif generation stays step 2 (but align data model)
|
||||
- Keep the current path:
|
||||
- `MotifEngine.generateFromMIDI(NoteEvent[])` then `MotifEngine.play()`
|
||||
- Recommended alignment work:
|
||||
- Decide whether Motif should later consume richer track metadata (program/channel) to improve role mapping.
|
||||
|
||||
---
|
||||
|
||||
## Integration milestones & acceptance checks
|
||||
|
||||
### Milestone A — Real MIDI end-to-end
|
||||
- Search returns results from BitMidi/Dongrays with mock disabled by default
|
||||
- Fetch returns real bytes and caches them
|
||||
- Parse endpoint `/api/midi/parse` works for metadata
|
||||
|
||||
### Milestone B — “Correct” preview playback
|
||||
- Preview produces recognizable instrument playback (piano at minimum)
|
||||
- Stop reliably stops scheduled notes
|
||||
- Works in Chrome/Safari with autoplay policies (requires click)
|
||||
|
||||
### Milestone C — Motif as second step
|
||||
- Generate Motif still works on the same loaded MIDI
|
||||
- Preview and Motif can be A/B tested without reloading the page
|
||||
|
||||
---
|
||||
|
||||
## Recommended team task breakdown
|
||||
|
||||
### Backend engineer
|
||||
- Implement env gating for mock/synthetic
|
||||
- Tighten fetch behavior + ensure BitMidi URLs are truly fetched
|
||||
- Add explicit “source = real/cache/synthetic” marker
|
||||
|
||||
### Frontend engineer
|
||||
- Add the chosen soundfont library
|
||||
- Implement `SoundfontMIDIPlayer`
|
||||
- Wire `src/main.ts` preview buttons to soundfont playback
|
||||
- Add user-facing error messaging for soundfont failures
|
||||
|
||||
### QA / test harness
|
||||
- Maintain a short list of known-good queries (3–5 songs) and confirm:
|
||||
- search results appear
|
||||
- at least one MIDI fetches and plays
|
||||
- Motif plays afterward
|
||||
@@ -1,92 +0,0 @@
|
||||
# Home UX audit + improvements (arcade-first)
|
||||
|
||||
This doc is a saved copy of the current home UX audit plan (no code changes).
|
||||
|
||||
## Current UX issues (what’s making it feel uninspiring/confusing)
|
||||
- **Unclear hierarchy**: “Search”, “Preview”, and “Synthesis Engine” compete; users don’t instantly see the main thing to do.
|
||||
- **Weak step-by-step guidance**: The app is a 3-step flow (Search → Select → Run engine), but the UI doesn’t feel like a guided sequence.
|
||||
- **Controls feel detached from state**: Buttons don’t always read as “locked until selection”; the reason for disabled state isn’t visible.
|
||||
- **Results table is dense** on mobile: long titles wrap unpredictably, selection highlight is subtle.
|
||||
- **Copy and concept drift**: “emulator/engine” and “MIDI scrape/analyse” copy varies; the story should be consistent and confident.
|
||||
- **iOS audio state**: The “Enable Audio” banner is reactive but not integrated into the flow (and can appear “random”).
|
||||
|
||||
## Target outcome
|
||||
Arcade cabinet vibe with a clear, dramatic main CTA:
|
||||
- **Primary action**: “Run the engine” (Generate & Play)
|
||||
- Secondary: preview the MIDI
|
||||
- Stronger “insert coin” style guidance: show step chips and a big engine panel.
|
||||
|
||||
## Proposed UX changes (high impact, low risk)
|
||||
|
||||
### 1) Make the flow explicit
|
||||
Update `index.html`:
|
||||
- Add a compact “Steps” row near the top:
|
||||
- Step 1: Search
|
||||
- Step 2: Pick a MIDI
|
||||
- Step 3: Run the engine
|
||||
- Tie each step’s visual state to app state:
|
||||
- Step 2/3 show “locked” styling until available.
|
||||
|
||||
### 2) Promote “Synthesis Engine” as the hero module
|
||||
- Move the engine card visually above Preview (or keep order but make engine card visually dominant).
|
||||
- Make engine CTA bigger and more arcade:
|
||||
- Primary button: **Run the engine**
|
||||
- Secondary: Stop
|
||||
- Volume + progress remain but visually subordinate.
|
||||
- Add a small one-line “what this does” under the button, not as a paragraph.
|
||||
|
||||
### 3) Preview becomes clearly “optional”
|
||||
- Rename to “Listen to the source MIDI (optional)”
|
||||
- Collapse preview by default on mobile (or add a “Show preview” toggle) to reduce overwhelm.
|
||||
|
||||
### 4) Results list readability + selection confidence
|
||||
- Increase row tap targets and selection contrast.
|
||||
- Add a right-side “Selected” chip on the selected row.
|
||||
- On mobile:
|
||||
- clamp long titles to 2 lines
|
||||
- reduce columns (hide Source or Duration) based on width.
|
||||
|
||||
### 5) iOS audio UX integrated into the engine (subtle)
|
||||
- Only on iOS-like browsers and only when audio is locked.
|
||||
- Show a small under-section link: “Having trouble on iOS? Enable audio”
|
||||
- Clicking expands a compact CTA row (Enable Audio button + tiny state line).
|
||||
- Avoid a big persistent banner that could confuse non-iOS users.
|
||||
|
||||
### 6) Copy pass
|
||||
Deferred for now: keep copy changes out of this pass.
|
||||
|
||||
## Implementation plan (concrete)
|
||||
|
||||
### Files
|
||||
- Primary: `index.html`
|
||||
- Minor TS adjustments (state-driven CSS classes): `src/main.ts`
|
||||
|
||||
### Approach
|
||||
- Add a small set of **state CSS classes** on `body` (or on `.container`):
|
||||
- `state-has-results`
|
||||
- `state-has-selection`
|
||||
- `state-audio-locked`
|
||||
- In `src/main.ts`, toggle these classes when:
|
||||
- results are loaded
|
||||
- a selection is made
|
||||
- audio unlock state changes
|
||||
- Use CSS to:
|
||||
- style locked sections
|
||||
- highlight the engine CTA
|
||||
- improve mobile responsiveness
|
||||
|
||||
## Test plan
|
||||
- Desktop Chrome/Safari:
|
||||
- search → select → engine run
|
||||
- preview optional still works
|
||||
- iOS Safari:
|
||||
- first run shows “arm audio” only when needed
|
||||
- no random banner when audio is already running
|
||||
- Visual:
|
||||
- mobile widths (320–430px)
|
||||
- long titles wrapping
|
||||
|
||||
## Non-goals
|
||||
- No changes to the core synthesis engine behavior or backend.
|
||||
- No copywriting/wording changes in this pass.
|
||||
|
||||
@@ -1,176 +0,0 @@
|
||||
# Sharing plan: links + player landing + Motif export
|
||||
|
||||
This document captures the implementation plan for **Share links** (Copy Link + Share to X), a minimal **player landing page**, and the next step: sharing **generated Motif variants** via **exported artifacts** (MIDI and/or audio).
|
||||
|
||||
## Goals
|
||||
- **Shareable outcome**: users can share something they heard (original MIDI preview, or a generated Motif variant).
|
||||
- **One-click-to-hear**: shared links load quickly and are ready to play (but never true autoplay; iOS requires a user gesture).
|
||||
- **Lightweight**: keep the landing page minimal and stable.
|
||||
- **Safe**: shared URLs must not turn the backend into an open proxy.
|
||||
|
||||
## MVP constraints (important)
|
||||
- **iOS/Safari** blocks audio until a user gesture. The landing can preload, but playback needs a tap.
|
||||
- **X/Twitter** shares are links + preview cards. No reliable in-post WebAudio playback.
|
||||
- Search results can change over time, so **index-based sharing** is brittle.
|
||||
|
||||
## UX spec
|
||||
|
||||
### A) Main app: Share controls
|
||||
Placement: near the “Selected MIDI”/result context (where users decide something is share-worthy).
|
||||
|
||||
Controls:
|
||||
- **Copy link**: copies a permalink to clipboard + shows a small “Copied” toast.
|
||||
- **Share to X**: opens the tweet composer with prefilled text + URL.
|
||||
|
||||
X intent URL format:
|
||||
- `https://twitter.com/intent/tweet?text=<encodedText>&url=<encodedShareUrl>`
|
||||
|
||||
Suggested default tweet text:
|
||||
- `Listening to “{title}” in MOTIF — try it`
|
||||
|
||||
### B) Share landing: `/play`
|
||||
A minimal landing page that:
|
||||
- reads parameters from the URL
|
||||
- fetches the MIDI (or exported artifact)
|
||||
- enables Play/Stop + volume
|
||||
- shows iOS “Enable Audio” CTA if required
|
||||
- includes a primary CTA: **Generate your own** → links back to the main app
|
||||
|
||||
## URL design (heart of the plan)
|
||||
|
||||
### v1: share the original MIDI preview (recommended MVP)
|
||||
Use the MIDI URL as the stable identifier.
|
||||
|
||||
Landing URL:
|
||||
- `/play?u=<encodedMidiUrl>&title=<encodedTitle>&song=<encodedQuery>`
|
||||
|
||||
Notes:
|
||||
- `u` is required for reproducibility.
|
||||
- `title` is display-only (optional).
|
||||
- `song` supports the CTA back to the main app (optional).
|
||||
|
||||
Example:
|
||||
- `/play?u=https%3A%2F%2Fbitmidi.com%2Fuploads%2F...mid&title=Hotel%20California&song=Hotel%20California`
|
||||
|
||||
Main app CTA target:
|
||||
- `/?song=<encodedQuery>`
|
||||
|
||||
### Why not share “result index”?
|
||||
Because search ranking shifts; index links rot. If needed, index can be a fallback only when `u` is missing.
|
||||
|
||||
## `/play` landing behavior
|
||||
|
||||
### Required inputs
|
||||
- **Preview share**: `u` (MIDI source URL)
|
||||
- **Export share**: `m` (motif midi id) or `a` (audio id)
|
||||
|
||||
### State machine
|
||||
- If `m` or `a` present → load exported artifact
|
||||
- Else if `u` present → fetch + parse MIDI via backend proxy
|
||||
- Else → show “Invalid link” + CTA to home
|
||||
|
||||
### Playback
|
||||
- Never autoplay.
|
||||
- Enable Audio CTA shown until AudioContext is running.
|
||||
|
||||
### Error cases
|
||||
- Missing/invalid params → show “Invalid link”
|
||||
- Backend fetch failure → show “Couldn’t load this MIDI”
|
||||
- Parse failure → show “Unsupported or corrupted MIDI”
|
||||
|
||||
## Export-based sharing (generated Motif variants)
|
||||
Sharing a generated Motif variant needs an artifact the landing page can load reliably:
|
||||
- **MIDI export**: smaller, fast, consistent with the project (music-as-structure)
|
||||
- **Audio export**: most universal listening, but heavier and needs careful encoding/hosting
|
||||
|
||||
### Recommended sequencing
|
||||
1) **MIDI export first** (fast, small, easy to iterate)\n2) Add **audio export** once the flow proves value (and storage/limits are solved)
|
||||
|
||||
### Proposed backend endpoints
|
||||
|
||||
#### 1) Create export (returns a shareable play URL)
|
||||
`POST /api/share/export`
|
||||
|
||||
Input (example):
|
||||
```json
|
||||
{
|
||||
"u": "https://bitmidi.com/uploads/....mid",
|
||||
"song": "Hotel California",
|
||||
"title": "Hotel California - Eagles",
|
||||
"preset": "dance|ambient|ominous|default",
|
||||
"params": { "intensity": 0.5, "swing": 0.1 },
|
||||
"format": "midi|audio"
|
||||
}
|
||||
```
|
||||
|
||||
Output (example):
|
||||
```json
|
||||
{
|
||||
"id": "abc123",
|
||||
"format": "midi",
|
||||
"playUrl": "/play?m=abc123&title=Hotel%20California&song=Hotel%20California"
|
||||
}
|
||||
```
|
||||
|
||||
#### 2) Fetch export artifact
|
||||
Option A (explicit):
|
||||
- `GET /api/share/artifact?id=abc123` → returns bytes (MIDI or audio)
|
||||
|
||||
Option B (format-specific):
|
||||
- `GET /api/share/midi?id=abc123`
|
||||
- `GET /api/share/audio?id=abc123`
|
||||
|
||||
Landing params:
|
||||
- MIDI artifact: `/play?m=<id>&title=...&song=...`
|
||||
- Audio artifact: `/play?a=<id>&title=...&song=...`
|
||||
|
||||
### Storage strategy (dev vs production)
|
||||
- **Local dev**: filesystem under `server/cache/exports/`
|
||||
- **Vercel production**: filesystem is not durable; prefer:
|
||||
- blob storage (recommended), or
|
||||
- KV/object store (if blob not available), or
|
||||
- a small database row referencing a blob key
|
||||
|
||||
### Security constraints (must-have)
|
||||
- Validate `u` server-side:
|
||||
- allow only `http`/`https`
|
||||
- strongly consider allowlisting hosts (e.g. `bitmidi.com`) for v1
|
||||
- Enforce limits:
|
||||
- max MIDI bytes
|
||||
- max parsed duration
|
||||
- max export size
|
||||
- max export time/CPU
|
||||
- Rate-limit export endpoints.
|
||||
- Cache and dedupe exports:
|
||||
- key by `(sourceMidiHash + preset + params + format)` to avoid repeated work
|
||||
|
||||
## Files (expected) — implementation map
|
||||
Frontend:
|
||||
- `index.html`: Share UI elements
|
||||
- `src/main.ts`: URL generation + clipboard + X intent
|
||||
- `play.html` + `src/play.ts`: landing page UI + logic
|
||||
- `vite.config.ts`: add `play.html` to multi-page inputs
|
||||
|
||||
Backend:
|
||||
- `server/src/server.ts`: route handlers for export endpoints
|
||||
- `server/src/services/*`: reuse parsing/generation services as needed
|
||||
- `server/cache/exports/`: local dev artifact cache
|
||||
|
||||
Routing/deploy:
|
||||
- `vercel.json`: add `/play` → `/play.html` similar to `/embed`
|
||||
|
||||
## Test plan
|
||||
- Desktop:
|
||||
- Share from main app → open `/play?...` in new tab → loads → plays after click
|
||||
- Copy link works; X intent opens with correct URL
|
||||
- iOS Safari:
|
||||
- `/play` shows Enable Audio when needed
|
||||
- Tap Enable Audio → tap Play → sound
|
||||
- Export flow:
|
||||
- Export MIDI variant → receive `/play?m=...` → plays the exported MIDI reliably
|
||||
- Export audio variant (if implemented) → `/play?a=...` → plays reliably
|
||||
|
||||
## Success criteria
|
||||
- A shared link reliably recreates “the thing” (preview or exported variant) and is playable with one tap.
|
||||
- The system remains safe under abuse (no open proxy behavior, bounded cost).
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
# Save the social preview image as og-image.jpg in this directory
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 178 KiB |
@@ -1,42 +0,0 @@
|
||||
console.log('Main script loading...');
|
||||
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
console.log('DOM loaded');
|
||||
|
||||
const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement;
|
||||
const songInput = document.getElementById('songInput') as HTMLInputElement;
|
||||
const status = document.getElementById('status')!;
|
||||
|
||||
if (!searchBtn || !songInput || !status) {
|
||||
console.error('UI elements not found!');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('UI elements found');
|
||||
|
||||
searchBtn.addEventListener('click', async function() {
|
||||
console.log('Search button clicked!');
|
||||
|
||||
const songName = songInput.value.trim();
|
||||
if (!songName) return;
|
||||
|
||||
status.textContent = 'Searching...';
|
||||
searchBtn.disabled = true;
|
||||
|
||||
try {
|
||||
const response = await fetch(`http://localhost:3001/api/midi/search?q=${encodeURIComponent(songName)}`);
|
||||
const data = await response.json();
|
||||
|
||||
console.log('Search results:', data);
|
||||
status.textContent = `Found ${data.count} results: ${data.results.map((r: any) => r.title).join(', ')}`;
|
||||
|
||||
} catch (error) {
|
||||
console.error('Search error:', error);
|
||||
status.textContent = `Search error: ${error}`;
|
||||
} finally {
|
||||
searchBtn.disabled = false;
|
||||
}
|
||||
});
|
||||
|
||||
console.log('Event listeners attached');
|
||||
});
|
||||
@@ -1,189 +0,0 @@
|
||||
console.log('Full Motif app loading...');
|
||||
|
||||
interface SearchResult {
|
||||
id: string;
|
||||
title: string;
|
||||
source: string;
|
||||
pageUrl: string;
|
||||
midiUrl: string;
|
||||
confidence: number;
|
||||
}
|
||||
|
||||
class MotifApp {
|
||||
private searchResults: SearchResult[] = [];
|
||||
private selectedIndex = 0;
|
||||
|
||||
constructor() {
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
this.initializeUI();
|
||||
});
|
||||
}
|
||||
|
||||
private initializeUI(): void {
|
||||
console.log('Initializing UI...');
|
||||
|
||||
const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement;
|
||||
const songInput = document.getElementById('songInput') as HTMLInputElement;
|
||||
|
||||
if (!searchBtn || !songInput) {
|
||||
console.error('UI elements not found!');
|
||||
return;
|
||||
}
|
||||
|
||||
searchBtn.addEventListener('click', () => this.handleSearch());
|
||||
songInput.addEventListener('keypress', (e) => {
|
||||
if (e.key === 'Enter') this.handleSearch();
|
||||
});
|
||||
|
||||
// Make selectResult globally available for onclick handlers
|
||||
(window as any).app = this;
|
||||
|
||||
console.log('UI initialized successfully');
|
||||
}
|
||||
|
||||
private async handleSearch(): Promise<void> {
|
||||
const songInput = document.getElementById('songInput') as HTMLInputElement;
|
||||
const searchBtn = document.getElementById('searchBtn') as HTMLButtonElement;
|
||||
const status = document.getElementById('status')!;
|
||||
|
||||
const songName = songInput.value.trim();
|
||||
if (!songName) return;
|
||||
|
||||
status.textContent = 'Searching for MIDI files...';
|
||||
searchBtn.disabled = true;
|
||||
|
||||
try {
|
||||
const response = await fetch(`http://localhost:3001/api/midi/search?q=${encodeURIComponent(songName)}`);
|
||||
const data = await response.json();
|
||||
|
||||
console.log('Search results:', data);
|
||||
|
||||
if (data.results.length === 0) {
|
||||
status.textContent = 'No MIDI files found. Try a different search.';
|
||||
return;
|
||||
}
|
||||
|
||||
this.searchResults = data.results;
|
||||
this.displayResults();
|
||||
status.textContent = `Found ${data.results.length} MIDI files. Select one to play.`;
|
||||
|
||||
} catch (error) {
|
||||
console.error('Search error:', error);
|
||||
status.textContent = `Search error: ${error}`;
|
||||
} finally {
|
||||
searchBtn.disabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
private displayResults(): void {
|
||||
const resultsSection = document.getElementById('resultsSection')!;
|
||||
const resultsBody = document.getElementById('resultsBody')!;
|
||||
|
||||
resultsBody.innerHTML = '';
|
||||
|
||||
this.searchResults.forEach((result, index) => {
|
||||
const row = document.createElement('tr');
|
||||
if (index === this.selectedIndex) {
|
||||
row.classList.add('selected');
|
||||
}
|
||||
|
||||
row.innerHTML = `
|
||||
<td>${result.title}</td>
|
||||
<td>${result.source}</td>
|
||||
<td>
|
||||
<div class="confidence-bar">
|
||||
<div class="confidence-fill" style="width: ${result.confidence * 100}%"></div>
|
||||
</div>
|
||||
</td>
|
||||
<td>?</td>
|
||||
<td>?</td>
|
||||
<td></td>
|
||||
<td><button onclick="window.app.selectResult(${index})">Select</button></td>
|
||||
`;
|
||||
|
||||
resultsBody.appendChild(row);
|
||||
});
|
||||
|
||||
resultsSection.classList.add('visible');
|
||||
|
||||
// Auto-select first result
|
||||
if (this.searchResults.length > 0) {
|
||||
this.selectResult(0);
|
||||
}
|
||||
}
|
||||
|
||||
public async selectResult(index: number): Promise<void> {
|
||||
if (index < 0 || index >= this.searchResults.length) return;
|
||||
|
||||
this.selectedIndex = index;
|
||||
const result = this.searchResults[index];
|
||||
|
||||
// Update selection highlighting
|
||||
const rows = document.querySelectorAll('#resultsBody tr');
|
||||
rows.forEach((row, i) => {
|
||||
row.classList.toggle('selected', i === index);
|
||||
});
|
||||
|
||||
const status = document.getElementById('status')!;
|
||||
const playerSection = document.getElementById('playerSection')!;
|
||||
const selectedTitle = document.getElementById('selectedTitle')!;
|
||||
const selectedMeta = document.getElementById('selectedMeta')!;
|
||||
|
||||
status.textContent = 'Loading MIDI file...';
|
||||
|
||||
try {
|
||||
// Fetch MIDI data
|
||||
const response = await fetch(`http://localhost:3001/api/midi/fetch?u=${encodeURIComponent(result.midiUrl)}`);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to fetch MIDI: ${response.status}`);
|
||||
}
|
||||
|
||||
const midiBuffer = await response.arrayBuffer();
|
||||
|
||||
// Update UI
|
||||
selectedTitle.textContent = result.title;
|
||||
selectedMeta.innerHTML = `
|
||||
<strong>Source:</strong> ${result.source} |
|
||||
<strong>Size:</strong> ${(midiBuffer.byteLength / 1024).toFixed(1)}KB |
|
||||
<strong>Confidence:</strong> ${Math.round(result.confidence * 100)}%
|
||||
`;
|
||||
|
||||
playerSection.classList.add('visible');
|
||||
|
||||
// Enable preview button
|
||||
const previewBtn = document.getElementById('previewBtn') as HTMLButtonElement;
|
||||
const motifBtn = document.getElementById('motifBtn') as HTMLButtonElement;
|
||||
previewBtn.disabled = false;
|
||||
motifBtn.disabled = false;
|
||||
|
||||
// Store MIDI data for playback
|
||||
(this as any).currentMIDI = { buffer: midiBuffer, result };
|
||||
|
||||
status.textContent = 'MIDI loaded. You can now preview or generate synthesis.';
|
||||
|
||||
} catch (error) {
|
||||
console.error('Load error:', error);
|
||||
status.textContent = `Load error: ${error}`;
|
||||
}
|
||||
}
|
||||
|
||||
public async handlePreview(): Promise<void> {
|
||||
console.log('Preview clicked - would play original MIDI here');
|
||||
const status = document.getElementById('status')!;
|
||||
status.textContent = 'Preview playback not yet implemented - but MIDI is loaded!';
|
||||
}
|
||||
|
||||
public async handleMotif(): Promise<void> {
|
||||
console.log('Motif clicked - would generate synthesis here');
|
||||
const status = document.getElementById('status')!;
|
||||
status.textContent = 'Motif synthesis not yet implemented - but MIDI is parsed!';
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize app
|
||||
new MotifApp();
|
||||
|
||||
// Expose handlers for buttons
|
||||
(window as any).handlePreview = () => (window as any).app.handlePreview();
|
||||
(window as any).handleMotif = () => (window as any).app.handleMotif();
|
||||
@@ -1,21 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Test</title>
|
||||
</head>
|
||||
<body>
|
||||
<button id="testBtn">Test Button</button>
|
||||
<div id="output">Not loaded</div>
|
||||
|
||||
<script>
|
||||
console.log('JavaScript is loading...');
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
console.log('DOM loaded');
|
||||
document.getElementById('output').textContent = 'JavaScript loaded!';
|
||||
document.getElementById('testBtn').addEventListener('click', function() {
|
||||
alert('Button works!');
|
||||
});
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,107 +0,0 @@
|
||||
# MOTIF Project - Thursday Status Report
|
||||
|
||||
## Project Overview
|
||||
|
||||
**MOTIF** is a procedural music synthesis system that extracts structural information from existing MIDI files and recreates them as original, real-time audio using Web Audio API. The core concept: "music as executable structure, not static audio."
|
||||
|
||||
### Workflow
|
||||
1. User searches for a song by name
|
||||
2. System finds MIDI files from multiple sources (BitMidi, Dongrays)
|
||||
3. MIDI is parsed and analyzed for structural features (tempo, density, melodic patterns)
|
||||
4. Notes are mapped to synthesis "roles" (bass, drone, ostinato, texture, accents)
|
||||
5. Web Audio API generates procedural audio with similar "feel" but original sound
|
||||
|
||||
## Current Status: Functional MVP
|
||||
|
||||
### ✅ Completed Features
|
||||
|
||||
**Complete Search Pipeline**
|
||||
- Multi-source MIDI search with confidence scoring
|
||||
- Real MIDI integration with fetching and parsing
|
||||
- Graceful error handling and timeouts
|
||||
|
||||
**Role-Based Synthesis Engine**
|
||||
- Intelligent mapping of MIDI tracks to synthesis layers
|
||||
- Role-specific oscillator types and filtering
|
||||
- Velocity-sensitive ADSR envelopes
|
||||
- Polyphonic chord support
|
||||
- Automatic looping and proper cleanup
|
||||
|
||||
**Polished User Interface**
|
||||
- Search results table with confidence bars and quality analysis
|
||||
- Dual player UI: Preview Original MIDI vs Generate Motif
|
||||
- Real-time status updates and progress feedback
|
||||
- "Try Next Result" workflow for easy A/B testing
|
||||
|
||||
**Backend Infrastructure**
|
||||
- Express + TypeScript server
|
||||
- CORS proxy with validation and SHA256 disk caching
|
||||
- Multi-source search (BitMidi, Dongrays, synthetic fallback)
|
||||
- Quality assessment with penalties for problematic content
|
||||
|
||||
### 📈 Recent Progress (Latest Commit)
|
||||
|
||||
**Major UI/UX Improvements:**
|
||||
- Implemented search results table with metadata display
|
||||
- Added MIDI preview player with basic oscillator mapping
|
||||
- Built dual transport controls for comparison
|
||||
- Integrated ParsedMIDIInfo with comprehensive track analysis
|
||||
- Enhanced confidence scoring system with quality penalties
|
||||
|
||||
**Technical Enhancements:**
|
||||
- Sophisticated role mapping with pitch range and density analysis
|
||||
- Improved error handling across the pipeline
|
||||
- Better synthesis scheduling with Web Audio lookahead
|
||||
- Streamlined search-to-synthesis workflow
|
||||
|
||||
## Current Problems & Limitations
|
||||
|
||||
### 🔴 Performance Issues
|
||||
- **~70% search success rate** (goal: >90%)
|
||||
- **~30% musical similarity recognition** (goal: >70%)
|
||||
- HTML regex parsing is fragile (should use DOM parsing)
|
||||
|
||||
### 🟡 Feature Limitations
|
||||
- Basic role mapping heuristics (lacks harmonic analysis)
|
||||
- Simple synthesis timbres (basic oscillators only)
|
||||
- Limited MIDI source coverage
|
||||
- No user controls for synthesis parameters
|
||||
|
||||
### 🟠 Technical Debt
|
||||
- Need more robust parsing for edge cases
|
||||
- Search confidence scoring could be more sophisticated
|
||||
- Some synthesis roles need refinement
|
||||
|
||||
## Next Sprint Priorities
|
||||
|
||||
### Phase 1: Core Stability (Next 1-2 weeks)
|
||||
1. **Improve search success rate** - better error handling, additional sources
|
||||
2. **Enhance role mapping** - add harmonic analysis, rhythm detection
|
||||
3. **Polish synthesis** - more interesting timbres, dynamic control
|
||||
4. **Robust parsing** - replace regex with proper DOM parsing
|
||||
|
||||
### Phase 2: Musical Intelligence (2-4 weeks)
|
||||
1. **Smarter scoring** - melodic similarity, harmonic progression analysis
|
||||
2. **Advanced synthesis** - effects, modulation, realistic instruments
|
||||
3. **User controls** - synthesis parameter adjustment, role customization
|
||||
4. **More MIDI sources** - expand search coverage
|
||||
|
||||
### Phase 3: Production Ready (1-2 months)
|
||||
1. **Performance optimization** - caching, preloading, worker threads
|
||||
2. **Legal compliance** - proper attribution, copyright handling
|
||||
3. **User uploads** - allow custom MIDI file analysis
|
||||
4. **Production infrastructure** - deployment, monitoring, scaling
|
||||
|
||||
## Technical Architecture
|
||||
|
||||
**Backend:** Express + TypeScript with multi-source search, CORS proxy, and caching
|
||||
**Frontend:** TypeScript + Vite + Web Audio with real-time synthesis
|
||||
**Key Components:** MotifEngine, RoleMapper, SynthesisEngine, MIDIPlayer
|
||||
|
||||
## Demo Status
|
||||
|
||||
✅ **Ready to demonstrate** - Full end-to-end pipeline functional
|
||||
✅ **User-friendly interface** - Polished search and playback experience
|
||||
✅ **Comparative validation** - Side-by-side original vs synthesis preview
|
||||
|
||||
The project successfully proves the core concept and is ready for user testing and iterative improvement.
|
||||
Reference in New Issue
Block a user