README.md (6478B)
1 # FluidStudio 2 3 Programmable music notation with audio+video frame sync — write music as text, compile to MIDI + WAV stems, generate per-frame data for visuals. 4 5 For Hermes agents: see [SKILL.md](SKILL.md) for the full reference (notation, frame sync, creative guidance). 6 7 ## Quick Start 8 9 ```bash 10 git clone https://git.corbia.net/repos/fluidstudio.git fluidstudio 11 cd fluidstudio 12 13 # Install system dependencies (Debian/Ubuntu) 14 sudo apt install fluidsynth ffmpeg 15 16 # Create virtual environment and install Python deps 17 python3 -m venv .venv 18 source .venv/bin/activate 19 pip install -r requirements.txt 20 21 # Compile a project 22 python -m tools.compiler projects/my_song 23 ``` 24 25 ## Requirements 26 27 **System packages** (Debian/Ubuntu): 28 - `fluidsynth` — MIDI to WAV rendering (needs a SoundFont) 29 - `ffmpeg` — stem mixing, audio trimming 30 31 ```bash 32 sudo apt install fluidsynth ffmpeg 33 ``` 34 35 **Python packages** (in `requirements.txt`): 36 - `mido` — MIDI file generation 37 - `numpy` — audio processing 38 - `Pillow` — video frame generation 39 40 ## Project Structure 41 42 ``` 43 fluidstudio/ 44 ├── asset/ 45 │ └── soundfonts/ # SF2 soundfont files 46 │ ├── misc/ 47 │ └── rkhive/ 48 ├── tools/ # Python tools (package) 49 │ ├── __init__.py 50 │ ├── parser.py # .flsp notation parser 51 │ ├── midi_gen.py # MIDI generation + FluidSynth rendering 52 │ ├── compiler.py # Project compiler (partition → stems → mix) 53 │ ├── clips.py # Per-frame CSV for video sync 54 │ ├── fps.py # BPM to FPS calculator 55 │ ├── video.py # Video generation from clips 56 │ └── chromatic_check.py # Chord validation utility 57 ├── projects/ # Your songs go here 58 ├── requirements.txt 59 └── README.md 60 ``` 61 62 ## Creating a Project 63 64 Each project lives in `projects/<song_name>/` with this structure: 65 66 ``` 67 projects/<song_name>/ 68 ├── partition/ 69 │ ├── header.json # BPM, octave, mode, channels 70 │ └── <voice>.flsp # One file per stem (filename = channel name) 71 ├── stems/ # Compiled WAV stems (generated) 72 ├── mix/ # Mixed WAV (generated) 73 ├── midi/ # Compiled MIDI files (generated) 74 └── clip/ # Per-frame CSVs for video sync (generated) 75 ``` 76 77 ### Header Format (`partition/header.json`) 78 79 ```json 80 { 81 "project": "my_song", 82 "bpm": 120, 83 "key": 0, 84 "octave": 4, 85 "mode": "ionian", 86 "channels": [ 87 {"name": "piano", "sf2": "rkhive/jrhodes3a-looped/jrhodes3a-looped.sf2", "octave": 5}, 88 {"name": "drums", "sf2": "rkhive/ghosdrum/ghosdrum.sf2", "octave": 1} 89 ] 90 } 91 ``` 92 93 - `key`: key root in semitones (0=C, 7=G, etc.). Combined with `mode`, suggests which chords fit well together. Since the notation is fully chromatic, any note works anywhere — this is a guide, not a restriction. 94 - `octave`: base octave (0 = C at MIDI 12). `octave: 4` = C4 = MIDI 60. 95 - `mode`: combined with `key`, determines which chords are "scale-pure". Use `chromatic_check.py` to list them. 96 - Per-channel `octave`: optional override. 97 - `sf2`: path relative to `asset/soundfonts/`. 98 99 ### Choosing Instruments 100 101 Check `asset/soundfonts/INDEX.md` for the full catalog with notes on optimal ranges. Some SoundFonts only sound good in specific octaves (e.g. Classic Hit is best between G2–G3, Crypt Organ has chords on C0 and bass on C-2–B-1). Always verify the instrument range before assigning an octave. 102 103 ### Notation Format (`.flsp`) 104 105 **Chromatic semitone system:** 106 ``` 107 0=C 1=C# 2=D 3=D# 4=E 5=F 6=F# 7=G 8=G# 9=A 10=A# 11=B 108 ``` 109 110 **Rules:** 111 - `---` = beat divider (one section between dividers = one beat) 112 - Lines between dividers are subdivisions of that beat 113 - `- ` prefix for subdivision lines (stripped during parse) 114 - `//` for inline comments, `#` for full-line comments 115 116 **Cell syntax:** `note[x][chord_type][:duration] [(v:velocity)] [(legato: on|off)]` 117 - `note` = chromatic semitone (0-11) 118 - `x` after note = play chord 119 - Duration: `b` = 1 beat, `b/2` = half beat, `b/3` = triplet, `2b` = 2 beats 120 - Velocity: `(v:NN)` — default 100 121 - Legato: `(legato: on)` / `(legato: off)` — toggles portamento 122 123 **Example (8ths subdivision):** 124 ``` 125 # melody — octave 4 126 - 0:b/2 (legato: on) // C, portamento on 127 - 4:b/2 // E 128 --- 129 - 7:b // G 130 --- 131 - 9:b/2 // A 132 - 11:b/2 // B 133 --- 134 - 0:b (legato: off) // C, portamento off 135 --- 136 - 0:b (v:60) // C, quiet 137 ``` 138 139 **Empty sections (rests):** 140 ``` 141 - 0 // kick 142 --- 143 - // rest (1 beat) 144 --- 145 - 1 // snare 146 ``` 147 148 ## Compiling 149 150 ```bash 151 # From project root 152 python -m tools.compiler projects/my_song 153 154 # From Python 155 from tools.compiler import compile_project 156 compile_project("projects/my_song") 157 ``` 158 159 This generates: MIDI files → WAV stems (exact beat duration) → mixed WAV. 160 161 ## Video Sync 162 163 ```bash 164 # Generate per-frame CSVs 165 python -m tools.clips projects/my_song 166 167 # Generate video from clips 168 python -m tools.video projects/my_song 169 ``` 170 171 ## Chord Validation 172 173 ```bash 174 python tools/chromatic_check.py 7 misheberak 175 ``` 176 177 Lists all scale-pure chords per degree for a given root and mode. 178 179 ## Available Modes 180 181 ionian, dorian, phrygian, lydian, mixolydian, aeolian, locrian, misheberak, freygish, hungarian_minor, hungarian_major, persian, whole_tone 182 183 ## Available Chord Types 184 185 M, M6, M69, M7m, M7M, M9, Madd2, Madd9, m, m6, m7m, m7M, m9, m69, madd2, madd9, sus2, sus26, sus27, sus27M, sus4, sus46, sus47, sus47M, dimb3, dim, dim7, b5, aug, aug7, p4, TT, p5 186 187 ## Design Philosophy 188 189 **Simplification over special cases.** Chromatic semitones (0-11) with octave-based offset eliminates the need for a separate percussion channel type — drums use the same system with a different octave. 190 191 **Modes as validation, not enforcement.** Modes define which chords are "scale-pure" for reference, but partitions can use any notes. 192 193 **Unified systems.** All channels (melodic, chord, percussion) use the same notation and resolution logic. 194 195 ## Pitfalls 196 197 - **Stem length mismatch:** `amix` with `duration=first` truncates to shortest input. All stems must be same length — loop patterns to match longest part. 198 - **Trim accuracy:** Use `ffmpeg -af "atrim=end=N"` (sample-accurate), not `-t` (codec-frame-accurate, ~ms off). 199 - **WAV for stitching:** MP3 adds padding (~0.05s). Keep stems as WAV until final delivery.