Dominik Ukolov
Dominik Ukolov is a musicologist and doctoral researcher at Leipzig University, where he taught as a tutor for electroacoustics, followed by research activities in the TASTEN and DISKOS projects.
He specializes in the multimodal virtualization of historical musical instruments and received several scholarships from the German National Academic Foundation, most recently for his PhD Project MODAVIS, which focuses on pipe organs and the development of the Virtual Acoustic Object standard. He is currently a research assistant at the Research Center DIGITAL ORGANOLOGY at the MIMUL and received the Frederick R. Selch Award by the American Musical Instrument Society in 2022.
Session
The emergence of highly efficient and accessible technologies opens up new opportunities for generating multimodal virtual representations of musical instruments. These primarily consist of audio recordings, three-dimensional models, simulation data and documents. Using the standard for Virtual Acoustic Objects (VAO) - which is also capable of incorporating historical acoustical spaces and situations - virtual objects can be used for artistic, educational, and research purposes as interactive representations and virtually playable musical instruments.
The framework not only supports the recording, analysis, segmentation and structuring of multimodal data including AI approaches, but also the conversion of photogrammetric interior models to simulate their acoustics at any virtual position. With the internal data relations and interfaces for various environments and applications, it is possible to interactively perform animations on segmented parts of a 3D model (e.g. piano keys), triggered by interactions or MIDI signals, resulting in real-time auralisations while observing the correlating mechanical actions. With its linked information system, the access and visualisation of person and institution networks, object and material provenance or other information classes can be achieved and implemented in Web-/App-Environments.
Attendees of this session will gain insights into multimodal data relations of musical instruments, acoustical spaces and interactive virtual representations. Discussions of the VAO-standard will be focused on its potential mapping to the Europeana Data Model. The session will present the outcome of the digitisation projects TASTEN and DISKOS, providing a comprehensive overview of workflows, challenges and use cases of VAOs in creative and remote research situations.