Implement an audio visualization system in the SysInfo project
Problem to solve
ScreenPlay lacks a nice way to display the current audio leves live in bars like rainmeter and wallpaper engine.
Further details
Because this is one of our most important features, this must be easy to use and extendable/theme-able.
Proposal
For example something like this:
// Non visual item that is written in c++ and is only used as a data provider
AudioInput {
id: audioInput
interval: 10 // Updates the current system audio ever 10ms
}
BarChartVisualizer {
id: visualizer
source: audioInput // Uses the audioInput as source
width:300
height:300
anchors.CenterIn: parent
color: "orange"
}
- A c++ class that reads the current audio levels of the OS
- This class should be part of the SysInfo project
- Register this class to be easily used in qml (For example like the AudioInput code above. Just an example can be named different if you find a more fitting name!)
- Is a list of properties or a list model better for this?
- Test what implementation is easier to use and if we hit some performance penatiels
- A QML Visualizer example element like a simple bar chart visualizer
What does success look like, and how can we measure that?
Success means when a user can toy around with a little code snipped that is provided by use.
Work
- Clone repo
- Get familiar with the audio level calculations from
onDurationChanged: {
// Load the pre-calculated magnitude data for the visualizer
var request = new XMLHttpRequest()
request.responseType = 'arraybuffer'
request.onreadystatechange = function() {
if (request.readyState === XMLHttpRequest.DONE) {
if (request.status == 200 || request.status == 0) {
var arrayBuffer = request.response
if (arrayBuffer) {
magnitudeArray = new Uint16Array(arrayBuffer)
visualizer.startVisualization()
}
} else {
console.warn("Couldn't load magnitude data for bars.")
}
request = null
}
};
request.open('GET', magnitudeDataSourceFile, true)
request.send(null)
}
function getNextAudioLevel(offsetMs) {
if (magnitudeArray === null)
return 0.0;
// Calculate the integer index position in to the magnitude array
var index = ((mediaPlayer.position + offsetMs) /
mainview.millisecondsPerBar) | 0;
if (index < 0 || index >= magnitudeArray.length)
return 0.0;
return (magnitudeArray[index] / 63274.0);
}
- Port the code to c++
- Check performance
- Create basic test qml implementation
Links / references
- Wallpaper example: https://youtu.be/05TQESLydE0?t=463
- Rainmeter code into nice Qt c++ https://github.com/rainmeter/rainmeter/tree/master/Plugins/PluginAudioLevel
- JS based audio visualization https://codepen.io/nfj525/pen/rVBaab
- Qt has an ugly 3d audio visualizer example https://doc.qt.io/qt-5/qt3d-audio-visualizer-qml-example.html that uses an audio file as source https://code.qt.io/cgit/qt/qt3d.git/tree/examples/qt3d/audio-visualizer-qml/main.qml?h=5.14#n118 we need to use the system audio
Edited by Elias Steurer