3Dmusic UI design

Tools & Trade: 3Dmusic UI Design

Introduction: assignment

A new 'tool/medium' medium (Bespoke/DIY production technology, critical or speculative artefact. The tool focuses on the design aspect. How can design be used for a tool? What is the experience like, what is the purpose of it?

Animation

In my studies at the department over animation, we are allows creating moving images. We use form, colour, contrast, compassion and motion, as are visual language for tell a story. But there is one aspect of animation creators ignore, “sound”. Sound is a essential part of the medium, it can invoke all kinds of reactions, like atmosphere, suspense, a emotional reaction and so fort. It’s 50% of the animation. There are also a lot of similarities between moving images and sound: - It can evoke a emotional reaction - Both can tell a story - Pattern: repeating certain elements create a pattern - Rhythm: defines the peas of the story - Both mediums can define a certain time & place - Both mediums can show something very real or abstract So why are most animators not familiar with sound design?

Animation: the study of movement and rhythm

The Problem: UI design

There’s a distinct different between the programs. Sound programs look very chaotic with all kinds of different UI designs. Often this effect and plugins are based on fiscal devices. These devices were used before digital sound design programs and so it can be very confusing for the user to use. Programs for designing visuals are userfriendly because they use icon design to specify certain actions. They use a visual language that users can understand more easily.

So why not use a better visual language for sound design programs? That’s because sound is very hard to visualise. My first thought was: ‘why not make my own visual language for sound design.’ So I started a quick research, looking for projects that did something similar thinks.

Similar project

A demo of an interactive piece of "Visual Sound: Project Visual Sound Wave Builder (Max/MSP/Jitter): Project Sound Art Installation Mark Cosgravehttps: Project

A demo of an interactive piece of "Visual Sound

After a while I found a video about “Cymatic”. With the use of vibration on a flat serves covert with sand. The vibration makes the sand move in a certain pattern.

Cymatics, is a subset of modal vibrational phenomena. The term was coined by Hans Jenny (1904-1972), a Swiss follower of the philosophical school known as anthroposophy. Typically the surface of a plate, diaphragm or membrane is vibrated, and regions of maximum and minimum displacement are made visible in a thin coating of particles, paste or liquid. Different patterns emerge in the excitatory medium depending on the geometry of the plate and the driving frequency.

Cymatic visualisation

Cymatic visualisation

CYMATIC sound wave from a violin

Cymatic Music Visualizer - Animated Software

Cymatics - Bringing Matter To Life With Sound

Tool: Cymatic inspired UI

Is a tool that lets you where a VR head, placing you in an environment. Here you can manipulate vibrating particles moving in a Cymatic structure. The Cymatic structure creates a sound. By changing the structure the sound will change with it. You can save these structures and mesh them up in to a melody. The function of the tool is to create sound by the use of visual effects. The goal is to show that design can changes the way we create sound in a visual way.

Inspiration

Inspiration: Cymatic

Inspiration: Particle UI Design

Inspiration: Visual Sound Design

Inspiration: Trons Fictional UI Design

Tool explanation: How will it function

The user will be put in a virtual environment with a user interface to control particle vibrating in in a Cymatic formation. By looking at the particle you can select them and change their behaviour and sound. For example you can lower their volume, pitch, delay or starch the sound, eco and even multiply the particles. When you finish you can save your visual sound structure and make more. When you have enough sounds you can create a melody or a beat by mashing up the visuals together.

Every thing will be built in unreal, an engine that supports programming virtual environments for all kinds of applications. The hardware contains a VR gear, Kinnect and a computer

Why use VR?

I believe VR brings new opportunistic for reinventing UI design. Going from a 2d interface to 3D interface brings new functionalities to the UI. Like working around the UI, or use head movement for navigation. It’s an opportunity to create new design language for UI functionality.

VR UI Design example

Research & studies: UI design, Cymatic & programming For this project ill will research Cymatics & 3D UI design. Building the tool, I’ll need to learn Programming with Blueprint and using the Ocular VR and the Kinnect motion sensor.

Unreal Engine: building and programming interactive spaces

Source: Oculus Rift DK2 - Unreal Engine 4

Source: Programing sound notes in Unreal Engine 4

[Source: www.youtube.com/watch?v=8jXojr7zeR8 Motion capture camera with the unreal Unreal Engine]

Book: abstracting craft, the practiced digital hand "marcom mccullough" The book offers a detailed analysis of the technical and psychological aspects of digital interfaces. In three sections, McCullough deals first with physiological and cognitive issues (hands, eyes, tools), moves on to representational and technological questions (symbols, interfaces, constructions) and ends with aspects of the practical usage of computers (medium, play, practice). The professed aim of the study is to re-root digital work in physical human agency and to develop a critical understanding of the ways in which the computer as a medium requires a new set of creative skills, especially regarding the handling of complex symbolic abstractions and the ability to construct mental models of objects and processes.

Source: [books.google.nl/books?id=PcwH1WricJEC&printsec=frontcover&hl=nl#v=onepage&q&f=false abstracting craft, the practiced digital hand "marcom mccullough"]

immersive design learning to let go of the screen

Approach: Projects Process

1. Start with Research on the subjects mention in “Research & studies.” 2. Design UI, functions and usability 3. Build the design in Unreal and program it with Blueprint 4. Alpha test 5. Fix the bugs and, work out the details 6. Beta test 7. Tweak bugs 8. Build an installation for the presentation 9. Document: screen recorders, screenshots, test demos, interviewing peoples reaction 10. Right 4000-word assay 11. Deadline

Result: Tool

The end result is a VR installation meant for making sounds by the use of a visuals UI design, a 4000-word assay and documentation video.

Is a tool that lets you where a VR head, placing you in an environment. Here you can manipulate vibrating particles moving in a Cymatic structure. The Cymatic structure creates a sound. By changing the structure the sound will change with it. You can save these structures and mesh them up in to a melody. The function of the tool is to create sound by the use of visual effects. The goal is to show that design can changes the way we create sound in a visual way.