This MR is going to be broken up into a number of smaller changes for the 0.7.x release and a subsequent 0.8.x release.

this change was originally started for a simple reason: code refactoring/reuse.

there were a bunch of SomethingArray types in parts and patterns data models, e.g. AudioTrackTrigsArray and MidiTrackTrigsArray

#[derive(PartialEq, Debug, Serialize, Deserialize, Clone, ContainerArrayMethods)]
pub struct AudioTrackTrigsArray(pub [AudioTrackTrigs; 8]);

#[derive(PartialEq, Debug, Serialize, Deserialize, Clone, ContainerArrayMethods)]
pub struct MidiTrackTrigsArray(pub [MidiTrackTrigs; 8]);

these are essentially the same thing -- only the inner array element types change. so i figured i would refactor the parts and patterns code, introducing some generic newtypes to remove the duplicated implementations.

anyway ... yeah ... then i went a bit mad and binge-coded a bunch of stuff over multiple weeks.

but the end result is

parts/patterns types are a lot easier to manage for me (fewer code duplicates)
downstream binaries/libraries can reuse array newtypes for their own purpose (see example below)
delegated methods and extra traits are implemented for array newtypes -- no more thing.0.0[0] to access array data (yippee!)
a panic-free indexing mechanism for array newtypes
improvements to slots data access

the new array newtypes

newtype	size	serde compatible?	`Box`ed inner array?	has identifier based indexing?	description
`Trigs<T>`	64	✅	✅	✅ `TrigId`	wraps any array data related to trigs
`Tracks<T>`	8	✅	❌	✅ `TrackId`	wraps any array data related to tracks
`Scenes<T>`	16	✅	❌	✅ `SceneId`	wraps any array data related to scenes
`Parts<T>`	4	✅	✅	✅ `PartId`	wraps any array data related to parts
`Patterns<T>`	16	✅	✅	✅ `PatternId`	wraps any array data related to patterns
`Banks<T>`	16	❌	✅	✅ `BankId`	wraps banks and enables reading/writing all project bank files from/to a project directory
`Arrangements<T>`	8	❌	✅	✅ `ArrId`	wraps arrangements and enables reading/writing all project arrangement files from/to a project directory
`FlexSlots<T>`	126	✅	✅	✅ `SlotId`	wraps any array data related to slots (flex)
`StaticSlots<T>`	128	✅	✅	✅ `SlotId`	wraps any array data related to slots (static)
`RecordingBuffer<T>`	8	✅	✅	✅ `RecBufId`	wraps any array data related to recording buffer slots

downstream reuse

all the new array newtypes abstract/generic, meaning you can re-use them in downstream libraries with your own custom newtypes

// imagine that we've decided to dump part data to yaml files ...
//
// we could keep track of where those files are stored in a `Vec<PathBuf>` ... 
//
// but then we could end up with 5x elements, or a panic if we use 
// `Vec::<PathBuf>::with_capacity(4)` and exceed the allowed size ...
//
// instead, we can implement a newtype to use with the `Parts` type

use ot_tools_io::types::Patterns;
use ot_tools_io::identifiers::PatternId;
use ot_tools_io::Defaults;
use ot_tools_io_derive::{BoxedArrayDefaults, ArrayDefaults};
use std::path::PathBuf;

// must implement `Default` and `Defaults`
//
// - `PathBuf` has a `Default` implementation
// - `ArrayDefaults` and `BoxedArrayDefaults` derive the `Defaults` implementation

#[derive(Debug, PartialEq, Default, ArrayDefaults, BoxedArrayDefaults)]
pub struct PartYamlPath(PathBuf);

let mut part_yaml_paths = Parts::<PartYamlPath>::default();

// can use `iter()` methods
for part in part_yaml_paths.iter_mut() {
    *part = PartYamlPath(PathBuf::from("non-default-path"));
}

// can use `id*()` methods
let first = part_yaml_paths.id_mut(&PartId::One);
*first = PartYamlPath(PathBuf::from("first"));

assert_eq!(part_yaml_paths.id(&PartId::One), PartYamlPath(PathBuf::from("first")));
assert_eq!(part_yaml_paths.id(&PartId::Two), PartYamlPath(PathBuf::from("non-default-path")));
assert_eq!(part_yaml_paths.id(&PartId::Three), PartYamlPath(PathBuf::from("non-default-path")));
assert_eq!(part_yaml_paths.id(&PartId::Four), PartYamlPath(PathBuf::from("non-default-path")));

`serde_big_array::Array` issues

missing useful traits

the Array type is missing a lot of useful traits, like AsRef.

// not possible
fn asref_function<T: AsRef<Array<u8, 256>>>(value: T) {
    println!("`Array` does not have `AsRef`, so we can't do this");
}

we can't add/modify these traits because Array is from a remote crate.

luckily the Array type is less than 100 lines of code, basically implementing 6 traits:

Serialize
Deserialize
Index
IndexMut
Deref
DerefMut

so, that's not too much work.

no method delegation

there are no standard collection methods like iter() or iter_mut() delegating from Array type into the inner array type. so it becomes just really plain annoying having to muck around every time we want to use some iter() method on array data ... which happens quite often when using Array as this is collection data we're dealing with here!

use serde_big_array::Array;

let some_arr: [u8; 256] = std::array::from_fn(|_| 42);
let boxed_serde_arr: Box<Array<u8, 256>> = Box::new(Array::new(some_arr));

// ewwwwwww .0.0
for (idx, element) in boxed_serde_arr.0.0.iter().enumerate() {
    println!("more fields!")
}

we cannot add these methods to Array because, again, Array is from a remote crate.

replacing `Array` usage with individual array newtypes

instead of relying on an type which cannot be modified (remote crate), i decided to implement the traits and method delegation on a series of new array newtypes. the new newtypes basically have two forms:

pub struct NotBoxeds<T>([T; <some fixed size>]>);
pub struct Boxeds<T>(Box<[T; <some fixed size>]>);

// e.g.
pub struct Tracks<T>([T; 8]>);
pub struct Parts<T>(Box<[T; 4]>);

so, after creating the individual newtype structs i just needed to add the 6 traits with custom implementations and then add method delegation. unfortunately that needs to be done for all seven newtypes and means circa 2,500 lines of very repetitive code that would have to be added.

the ContainerArrayMethods procedural macro from the ot-tools-io-derive crate previously handled this for existing SomethingArray types. it added method delegation and a number of useful traits (like IntoIterator). however, we cannot provide information to the proc_macro about the size of the inner array, unless we allow for generic array sizes

#[proc_macro_derive(ContainerArrayMethods)]
pub fn container_type_derive(input: TokenStream) -> TokenStream {

    let non_arr_name = name.to_string().replace("Array", "");

    let expanded = quote! {
        impl<T> #name {
            // ...
            pub fn each_ref(&self) -> [&T; ???????] {  // we can't tell the proc_macro what the array size should be
                self.0.each_ref()
            }
            // ...
        }
    }
}

i don't want to allow generic array size for these types -- they've been created because they wrap types that always have the same number of elements. so i implemented a bunch of macro_rules to remove a bunch of the repetitive code.

macro	description	example
`generic_newtype_method_delegate`	adds a bunch of `iter()` methods to the generic array newtype	`generic_newtype_method_delegate!(Parts, 4)`
`generic_newtype_id_lookups`	adds methods for panic-free index via 'identifier' enums (see later section)	`generic_newtype_id_lookups!(Parts, PartId)`
`generic_newtype_unbox`	adds the various methods to 'unbox' (deref a `Box`) on generic array newtypes that have an inner type of `Box<[T; N]`	`generic_newtype_unbox!(Parts, 4)`
`generic_newtype_asmut`	adds the `AsMut` trait to the generic array newtype	`generic_newtype_asmut!(Parts)`
`generic_newtype_asref`	adds the `AsRef` trait to the generic array newtype	`generic_newtype_asref!(Parts)`
`generic_newtype_deref`	adds the `Deref` trait to the generic array newtype	`generic_newtype_deref!(Parts, 4)`
`generic_newtype_derefmut`	adds the `DerefMut` trait to the generic array newtype	`generic_newtype_derefmut!(Parts)`
`generic_newtype_index`	adds the `Index` trait to the generic array newtype	`generic_newtype_index!(Parts)`
`generic_newtype_indexmut`	adds the `IndexMut` trait to the generic array newtype	`generic_newtype_indexmut!(Parts)`

for generic_newtype_method_delegate -- the majority of std iter() methods are included with the exception of any method that changes the size of the inner array as changing the size of the array would break serialization/deserialization. see the end of the MR description for a list of iter() methods that are added.

panic-free indexing via 'identifier' enums

Adds the following identifier enums to provide a panic-free/no-option method for accessing an element of the relevant array newtype

BankId
PartId
SceneId
ArrId
PatternId
TrackId

use ot_tools_io::types::{Parts, Part};
use ot_tools_io::identifiers::PartId;

let parts = Parts::<Part>::default();

// --------------------------------------------

// will panic
parts[256];

// --------------------------------------------

// impossible to panic -- no variant for invalid IDs
parts.id_ref(&PartId::One);
parts.id_ref(&PartId::Two);
parts.id_ref(&PartId::Three);
parts.id_ref(&PartId::Four);

// --------------------------------------------

// these are all fine ...
parts[PartId::try_from(0_usize)?.as_index()];
parts[PartId::try_from(1_usize)?.as_index()];
parts[PartId::try_from(2_usize)?.as_index()];
parts[PartId::try_from(3_usize)?.as_index()];

// throws an `InvalidIndex` error instead of a panic
parts[PartId::try_from(4_usize)?.as_index()];

improvements to slots data access

firstly, array newtypes for interacting with slot data have been added. the same benefits from the above sections apply to generic array newtypes for slot data (FlexSlots and StaticSlots).

additionally, a 'god'-like Slots type has been added, which basically wraps FlexSlots and StaticSlots.

let slots = Slots<SlotMarkers>::default();
let flexs = slots.flex_slots;
let statics = slots.static_slots;

secondly, there are new slot data types to handle dealing with slots in a way that more closely tracks how the octatrack UI treats slots.

one of the tricky parts of the Octatrack's data model is the splitting of SlotMarkers and SlotsAttributes types. in the Octatrack UI the combination of both for a specific index (slot ID) is considered one slot. similarly, we get a SampleSettingsFile by combining the data from a SlotMarkers and a SlotAttributes for a specific slot id. this means that if you wish to get all the relevant data for a specific slot you end up having to get the values from two different files, and it becomes a real pain because you end up having to pass slot_marks and slot_attrs variables to functions / methods all the time.

this change adds the ActiveSlot type which wraps both the SlotMarkers and SlotAttributes data into a single type

let x = ActiveSlot::new(slot_attrs, SlotMarkers::default());

x.attrs()      // &SlotAttributes
x.attrs_mut()  // &SlotAttributes
x.marks()      // &SlotMarkers
x.marks_mut()  // &SlotMarkers

// before
fn my_func(attrs: &SlotAttributes, marks: &SlotMarkers) -> Something;

// now
fn my_func(slot: &ActiveSlot) -> Something;

An ActiveSlot is named specifically because it can only exist if a sample has been "loaded" into a specific slot id. There are cases where no sample has been loaded into a specific slot id. for those instances we have the Slot type, which is an Option-like enum:

match slot {
    Slot::Active(x) => println!("slot gain: {}", x.attrs().gain),  // x is the `ActiveSlot` type
    Slot::Inactive => println!("no slot")
}

There is also the SlotPath type, which is another Option-like helper enum to handle the path of the slot. Unfortunately we can end up with "null" paths for flex slots (recording tracks). rather than having to match against PathBuf("") everywhere, the SlotPath lets us handle things a bit more idiomatically.

match slot.path {
    SlotPath::Path(x) => println!("slot path: {x:#?}"),  // x is the `PathBuf` type
    SlotPath::Empty => println!("slot path null -- recording slot?")
}

list of iter() methods added via generic_newtype_method_delegate

len
is_empty
element_offset
iter
iter_mut
fill_with
first
first_mut
last
last_mut
as_slice
as_mut_slice
as_array
as_mut_array
each_ref
each_mut
get
get_mut
as_chunks
as_chunks_mut
split
split_mut
split_at
split_at_mut
split_at_checked
split_at_mut_checked
split_first
split_first_mut
split_last
split_last_mut
split_inclusive
split_inclusive_mut
contains (requires type implements PartialEq)
into_iter (requires Copy)
map (requires Copy)
repeat (requires Copy)
copy_from_slice (requires Copy)
fill (requires Clone)
to_vec (requires Clone)
clone_from_slice (requires Clone)
clone_into (requires Clone)

Edited May 09, 2026 by dijksterhuis

Draft: Major Library Improvements