ComputerCraft / Create Mod Pipe Organ

[Dartania]

So something I've been low-key obsessed with for a long time is creating a music box in ComputerCraft.  I thought about it before Create existed, before Minecraft increased the number of instruments available on NoteBlocks.  It's definitely been a slow-burn between not-enough-time and too-many-hobbies.  But my interest in this has only grown as I've seen content creators like Tonestas and others make Create Mod Pipe Organs.  Unfortunately, the stronger my interest, the more complex the project becomes.  I start a new job tomorrow, so I might not get to play around with this again for a while.  Sound of Steam might come through with a better interface.  Who knows.  But I just want to ramble for a bit on my obstacles and successes.

Of course I do get distracted by a few things in-game... Whenever I actually go into Minecraft to work on it, I get distracted by either A) Trains or B) aesthetics of the organ.  Sound of Steam came out, and I decided to include all the sounds.  I had finally gotten a display layout I really liked (using the mirrored floor illusion to have half the organ under the floor, and making sure everything was within 2.5x2.5 chunks to keep the sound in-range, and decided to build a mix of a viking longhouse and gothic cathedral), only for Sound of Steam to double the number of pipes.  So... that finally got me out of the aesthetics rut and into just focusing on the programming.  For the actual build, I'll just have compressed rows with wind-chests.  And maybe later, I'll update the layouts to be prettier.

I like working on my laptop so I can watch TV with my wife, but I can't easily play modded on this machine.  So this week, I'm using CraftOS-PC to emulate CC computers.  But how do I play sounds you ask??  A) I'm making this whole thing modular.  The functions that control inputs, outputs, etc are all going to be interchangeable.  For instance, in the actual pipe organ, note_on and note_off will control bits in bundled redstone cable.  But in the emulator, note_on will use periphemu to create speaker peripherals and use speaker.playNote or speaker.playSound .  I'm still debating about whether the emulation will try to play the note multiple times for sustained notes, or whether I'm just going to let it play once.  B) How do I actually find sounds?  They're loaded into (mac) /Applications/CraftOS-PC.app/Contents/Resources/sounds with similar format as a resource pack.  I got the resources from the latest Sound of Steam github and from... I think Curseforge for the Minecraft base assets.  So I can play all the pipe organs and all the note blocks.

The most difficult part is figuring out the inputs.

What I have in mind is definitely far too ambitious for an MVP, but I can't get it out of my head.  I want to do all of: A) Download + Play Midi files, A.1) Download + Play NBS files, B) accept continuous input for both holding individual keys and toggling stops, C) the ability to edit and save tracks, which can be played individually or together, once or looped, probably saved in MIDI format, D) the ability to edit stops visually, E) B and C together (eg: edit and save a loop track and then play on top of it).

I'm going to use a combination of advanced-monitor touch/mouse and keyboard inputs.  Don't exactly know what all I'm going to do with that, though.  Probably numbers to toggle frequently-used stops, arrows to switch keyboard locations, arrows to change instruments and how far up in the range you are.  Visuals on the screen to indicate where in the list you are and where you can go, and how to escape to the menu.

So I also want the keys in use to be configurable because my laptop and desktop have different keyboard layouts, not to mention my wife's and brother-in-law's keyboards.  So like... the on-screen keyboard layouts would be in slightly different locations, in addition to using different computer keys.  The Nasard starts at C, for instance, so the range for that is different.  Different organ creators leave out different sets of pipes from their organs.  For instance, the highest range on the piccolo is very intense, while the lowest ranges in some of the bass sections are barely audible.  Some people are going to just use the original pipes, some will use Sound of Steam pre-0.8, some will use Sound of Steam post-0.8, some will want double the pipes for volume control, etc etc etc.  I also kinda want the flexibility to either do everything on a single screen, or to have multiple monitors so that it's similar to real organ pits, where you have multiple rows of keys laid out in front of you, and a wall of stops to either side, and maybe have a row of stops on the back.  But of course I can't do any of _that_ in the emulator.  I get one screen.

I want the final result to be fun and usable for people who don't have the same level of commitment, musical ability, or programming ability, as I do.  Eg: a friend of a friend on the server who's just there for Hexerei and who hasn't heard me rant about pipe organs, and has no interest in Create or ComputerCraft should be able to find the secret entrance to the organist's pit and start playing around and make music that sounds cool or at least scary.  The setup can require commitment, Create knowledge, CC knowledge, WorldEdit knowledge, etc.  But after the setup, you should be able to reprogram Stops, save and add multiple Tracks, download new midi files, and play saved songs with minimal non-music knowledge.

And the setup itself shouldn't take all week to connect each and every pipe.  I frequently have to start new worlds due to changes in our friendsgroup server's available mods and world corruption.  And when my friends are into Minecraft again, I want to be able to build this thing and play with the organ with my friends before they get bored of the worldsave.  The builder should definitely be able to choose, for each range of sounds, whether the notes are played via bundled-signal redstone relays, single-signal redstone relays (eg if bundles aren't available), or via speaker peripherals (eg for testing, or if it's too small of a space for the complete organ).  Since we can't change or copy the redstone relay numbers as we lay them down, to make it easier for single-signal relays, you'll be able to define a range of notes that have a range of sequentially-numbered relays.  So like `{mode: 'redstone_single', details: {side: 'top', peripheralPrefix: 'redstone_relay_'}, first: {note: 'f#3', redstone_relay_id: 324}, last: {note: 'f#4, redstone_relay_id: 336}}`.  It's still a lot of definitions to edit.  (~5 octaves per instrument across 17 windpipes (34 if using double-volume), as well as 1 or 2 ranges (depending on method) across 15 noteblocks, totaling 115 ranges, which is a lot less than the nearly 1500 entries required for individual blocks, and a lot easier to spot typos).

There'll be a settings menu to do more-common things post-build like select one of a few physical keyboard layouts, change the order that you cycle through instruments, etc.

The ability to use speaker peripherals (and especially emulated peripherals) is definitely on the path to MVP, even though it's technically extra and it doesn't seem like it should be MVP.  Being able to test on CC Computers without setting up the whole organ definitely makes it easier to stay on task without getting distracted by the beauty that is Minecraft and the fun of trains and Create mod production.

I've got https://forums.computercraft.cc/index.php?topic=597.msg1887#msg1887 bookmarked.  The Midi project found here: https://github.com/danielathome19/MIDIToComputerCraft is cool, but... since it's in Dotnet instead of Java or ComputerCraft Lua, I'm the only one of my friendsgroup who could reasonably use it.  And it hardcodes the whole thing to require a pastebin login, require a controller/agent setup with a separate computer for every range, assumes that the wireless modem is on top of the computer (as a general rule for building, mine are on the back or bottom), and has one and only one way of actually running the script.  But it's good to look at what it's doing.  I'm sure https://github.com/melanchall/drywetmidi has a more thorough example of how to read MIDI files in its source code.

I feel like I might be reinventing the wheel a little, but I very much would like to read download and read MIDI files directly from online sources into CC Computers without going through extra software.  I'm probably going to use a lot more code from SquidDev's Simple Piano https://github.com/SquidDev-CC/blanketcon-2023/blob/main/src/piano/piano.lua than from the C# implementation.

When converting arbitrary MIDI files to fit nicely inside a custom pipe organ, I suspect I'll have to make some big changes.  Either by storing separate meta data about the translation, or by creating a new MIDI file.  The ability to change which set of pipes play a specific instrument, or move sounds up/down an octave may be necessary per-song.  But MVP I'll probably just do something like website font families.  Have a file containing (in either .csv format or lua code returning a table) an array of 128 lists (matching up with the list of MIDI instruments), and each list will contain instrument names, in order from most-appropriate / least-likely-to-exist to least-appropriate / most-likely-to-exist.  So maybe your music box has Minecraft's guitar as an instrument, and the program will pick that when it sees 'guitar'.  But someone else's doesn't have guitar, so it picks the next best instrument.  And you can override it when building to pick whatever instrument you want to be there.

I need to get organized about what order I'll be developing things in.  For instance, here I am thinking about MIDI formats, and my current task is actually reading realtime inputs and translating them to note_on/note_off signals, and then my next step is seeing if I can get the speaker peripherals to seem like they're looping the audio.  I'm probably going to have a separate emulated speaker peripheral for every note so that I can use `speaker.stop()` for note_off events.

I think I'm done rambling for now.  I'll maybe add a comment if I make significant progress.