3Dmusic UI design

Tools & Trade: 3Dmusic UI Design

Introduction: assignment

A new 'tool/medium' medium (Bespoke/DIY production technology, critical or speculative artefact. The tool focuses on the design aspect. How can design be used for a tool? What is the experience like, what is the purpose of it?

Animation

In my studies at the department over animation, we are allows creating moving images. We use form, colour, contrast, compassion and motion, as are visual language for tell a story. But there is one aspect of animation creators ignore, “sound”. Sound is a essential part of the medium, it can invoke all kinds of reactions, like atmosphere, suspense, a emotional reaction and so fort. It’s 50% of the animation. There are also a lot of similarities between moving images and sound: - It can evoke a emotional reaction - Both can tell a story - Pattern: repeating certain elements create a pattern - Rhythm: defines the peas of the story - Both mediums can define a certain time & place - Both mediums can show something very real or abstract So why are most animators not familiar with sound design?

Animation: the study of movement and rhythm

The Problem: UI design

There’s a distinct different between the programs. Sound programs look very chaotic with all kinds of different UI designs. Often this effect and plugins are based on fiscal devices. These devices were used before digital sound design programs and so it can be very confusing for the user to use. Programs for designing visuals are userfriendly because they use icon design to specify certain actions. They use a visual language that users can understand more easily.

So why not use a better visual language for sound design programs? That’s because sound is very hard to visualise. My first thought was: ‘why not make my own visual language for sound design.’ So I started a quick research, looking for projects that did something similar thinks.

Similar project

A demo of an interactive piece of "Visual Sound: Project Visual Sound Wave Builder (Max/MSP/Jitter): Project Sound Art Installation Mark Cosgravehttps: Project

A demo of an interactive piece of "Visual Sound

After a while I found a video about “Cymatic”. With the use of vibration on a flat serves covert with sand. The vibration makes the sand move in a certain pattern.

Cymatics, is a subset of modal vibrational phenomena. The term was coined by Hans Jenny (1904-1972), a Swiss follower of the philosophical school known as anthroposophy. Typically the surface of a plate, diaphragm or membrane is vibrated, and regions of maximum and minimum displacement are made visible in a thin coating of particles, paste or liquid. Different patterns emerge in the excitatory medium depending on the geometry of the plate and the driving frequency.

CYMATIC sound wave from a violin

Cymatic Music Visualizer - Animated Software

Cymatics - Bringing Matter To Life With Sound

Tool: Cymatic inspired UI

Is a tool that lets you where a VR head, placing you in an environment. Here you can manipulate vibrating particles moving in a Cymatic structure. The Cymatic structure creates a sound. By changing the structure the sound will change with it. You can save these structures and mesh them up in to a melody. The function of the tool is to create sound by the use of visual effects. The goal is to show that design can changes the way we create sound in a visual way.

Why use VR?

I believe VR brings new opportunistic for reinventing UI design. Going from a 2d interface to 3D interface brings new functionalities to the UI. Like working around the UI, or use head movement for navigation. It’s an opportunity to create new design language for UI functionality.

VR UI Design example

Research & studies: UI design, Cymatic & programming For this project ill will research Cymatics & 3D UI design. Building the tool, I’ll need to learn Programming with Blueprint and using the Ocular VR and the Kinnect motion sensor.

Unreal Engine: building and programming interactive spaces

Source: Oculus Rift DK2 - Unreal Engine 4

Source: Programing sound notes in Unreal Engine 4

[Source: www.youtube.com/watch?v=8jXojr7zeR8 Motion capture camera with the unreal Unreal Engine]

Book: abstracting craft, the practiced digital hand "marcom mccullough" The book offers a detailed analysis of the technical and psychological aspects of digital interfaces. In three sections, McCullough deals first with physiological and cognitive issues (hands, eyes, tools), moves on to representational and technological questions (symbols, interfaces, constructions) and ends with aspects of the practical usage of computers (medium, play, practice). The professed aim of the study is to re-root digital work in physical human agency and to develop a critical understanding of the ways in which the computer as a medium requires a new set of creative skills, especially regarding the handling of complex symbolic abstractions and the ability to construct mental models of objects and processes.

Source: [books.google.nl/books?id=PcwH1WricJEC&printsec=frontcover&hl=nl#v=onepage&q&f=false abstracting craft, the practiced digital hand "marcom mccullough"]

Approach: Projects Process

1. Start with Research on the subjects mention in “Research & studies.” 2. Design UI, functions and usability 3. Build the design in Unreal and program it with Blueprint 4. Alpha test 5. Fix the bugs and, work out the details 6. Beta test 7. Tweak bugs 8. Build an installation for the presentation 9. Document: screen recorders, screenshots, test demos, interviewing peoples reaction 10. Right 4000-word assay 11. Deadline

Result: Tool

The end result is a VR installation meant for making sounds by the use of a visuals UI design, a 4000-word assay and documentation video.

Is a tool that lets you where a VR head, placing you in an environment. Here you can manipulate vibrating particles moving in a Cymatic structure. The Cymatic structure creates a sound. By changing the structure the sound will change with it. You can save these structures and mesh them up in to a melody. The function of the tool is to create sound by the use of visual effects. The goal is to show that design can changes the way we create sound in a visual way.

---

Design usability (for VR)

immersive design learning to let go of the screen

These are my key takeaways from experimenting with VR at Instrument:

THINK LIKE HUMAN

We’ve moved from open grasslands where we could see danger or reward…to urban spaces where we rely on signs to inform ultimately to computers where we rely on GUI to communicate.

Each of these can be seen as an interaction model in its own right:

The Savannah

The oldest of interaction models. We can see everything, we are grounded. Content obeys the space. Objects in the present are close at hand. The future is on the horizon before us, the past is behind.

The Shop

Like the savannah, the shop implies a space that you can move around in but with a higher level of density. Content can be locked to the walls or planes inside of a space.

The Abstract The last 40 years have seen the rise of the digital landscape; a two dimensional plane that abstracts familiar real-world concepts like writing, using a calendar, storing documents in folders into user interface elements (UI). This approach allows for a high level of information density and multitasking. The down-side is that new interaction models need to be learned and there is a higher cognitive load to decision making.

Use perspective to your advantage Designers use size, contrast and color to denote hierarchy. These tools are still available in VR, but they are a little different. Size is based on the distance between the user and a piece of content.

Designers now have a full field of vision to play with, and humans are used to turning their heads or whole bodies.

Despite this, designers are trying to force 2D solutions into a 3D space, just like the Virtual Boy.

1 Flat: A common solution

The interface is skinned for the 3D space. It’s difficult to read text or images in perspective. There is no sense of grounding in the space. It’s a wall.

2 Curved: Marginally better

The content is curved around the user, so the tiles always face the user — making it much easier to read text or images.

3 Less content: Better

Less content is better, even if that requires some way to move through it.

4 Surrounded: Best

Hierarchy can be implied by nearness to the cone of focus. Secondary content can be pushed out of immediate view but still remain accessible.

Virtual Reality is immersive; design should support and enhance the user’s sense of presence in the virtual environment.

- Avoid rapid movement, it makes people sick.

- If there is a horizon line, keep it steady. A rolling horizon in VR is like a rolling horizon when you’re on a ship — not good.

- Avoid rapid or abrupt transitions to the world space, they are very disorientating.

- Do not require the user to move their head or body too much. Not only is this disorienting, but the user may be wearing their headset in an environment that they cannot turn around in, like on a plane.

- Be careful about mixing 2D GUI and 3D, the change can be jarring.

- Keep the density of information and objects on the screen low, much lower than in standard screen design. Not everything has to be in view. - Use real-world cues when appropriate.

- Bright scenes are fatiguing. (vermoeiend)

- When in doubt; test, test, test.

UI Design for VR (so far)

Action Sheets

StyleSheets

Inspiration Color Sheet

Inspiration Visuals

Inspiration UIdesign

Color Pallet

Research Questions

1. What is the theme I am investigating?

Experimental UI design for VR space's

2. What is my research question?

How can I redefine interaction, with the use of 'Virtual Reality'

3. How am I structuring my research?

- Researching design experience, people have with every day objects. - Researching practical and user-friendly UI design. - Researching Interaction for Virtual Reality

- 1st Draft interaction design in VR - 2st Draft: integrate feedback into design - 3ste Draft: stylize design, import components into Unreal Engine - Program installation - Beta test installation - integrate feedback

4. What tools or methods could I/will I use to conduct my research?

- Books, Web articles, researching related projects, product testing…

5. What is the foreseen result?

The VR installation should prove: That the interaction with objects & UI (for a certain goal), can be easier and helps the uses to have a better experience.

6. What is the difference between my foreseen result and the result which came through the process?

My first foreseen result was to build a visual language for sound. Often sounddesign-programs are very hard to learn for beginners. I like to use VR as a medium for exploring new possibilities for interaction with UI design.

- But throughout the project I was advised to choose one ’thema’. So I wanted to focus on ‘interaction within the virtual space’. My goal is to show that interaction within a virtual space can be a lot like the real world. Instead of using abstract UI design like with the Iphone for example.

7. (question 1/6) Is this relevant? (is every question relevant for you project)

---

Reverse culture shock

Reverse culture shock is the process of readjusting, reacculturating, and reassimilating into one's own home culture after living in a different culture for a significant period of time.

Returnees have also been reported to experience alienation, disorientation, stress, value confusion, anger, hostility, compulsive fears, helplessness, disenchantment, and discrimination.

The diverse returnee population has been commonly organized into primary wage-earner (i.e., parent) occupational/sponsorship subgroups, such as missionaries, non-governmental organization workers, federal government employees, educators, volunteers, business and military personnel, and international students (Gerner et al., 1992)

Theories of reverse culture shock

Reverse culture shock received scholarly attention as early as 1944 when Scheutz (1944) examined the difficulties of returning armed forces veterans. Austin and Jones (1987) identified earlier sources that indirectly addressed reentry issues, dating from as early as 1935. Culture shock itself first received critical attention in the late 1950's and early 1960's and for the most part was studied through qualitative research methods. Lysgaard, 1955 and Oberg, 1960, and Gullahorn and Gullahorn (1963) were the first to describe culture shock and reverse culture shock qualitatively as intercultural adjustment.

Defining reverse culture shock begins with acknowledging reverse culture shock's “parent” construct, culture shock. Oberg's (1960) early definition was: “Culture shock is precipitated by the anxiety that results from losing all our familiar signs and symbols of social intercourse” (p. 177). P. Adler's (1975) definition of culture shock is psychologically more descriptive and explanatory:

Culture shock is primarily a set of emotional reactions to the loss of perceptual reinforcements from one's own culture, to new cultural stimuli which have little or no meaning, and to the misunderstanding of new and diverse experiences. It may encompass feelings of helplessness, irritability, and fears of being cheated, contaminated, injured or disregarded. (p. 13.)

Source: http://www.sciencedirect.com/science/article/pii/S0147176799000243

Reverse Reality shock is een term die ik heb bedacht op basis van 'Reverse Culture shock'. Het idee er achter is dat mensen die langdurig in VR leven moeiten hebben om zich aan te passen aan de realiteit. In een wereld waar mensen in twee totaal verschillende realiteiten leven, kan het mogelijk zijn dat we niet normaal kunnen functioneren in de echte wereld. Mijn focus richt zich voornamelijk op de physics tussen de realiteit en de virtuele wereld. De manier hoe mensen interactie hebben in de virtuele wereld kan erg verschillen met de interactie in de echte wereld. Denk bijvoorbeeld aan zwaarte kracht, distributie van massa, tijd en ruimte.

Voor nu heb ik al een aantal experimenten gedaan binnen de Unreal Engine.

- Ik heb een code gevonden die er voor zorgt dat je op muren en het plafon kan lopen - In mijn tweede experiment kan je je zelf teleporteren naar elke plek in space - En in mijn 3de experiment kun je object dupliceren door ze op te pakken Voor mijn tool wil ik uiteindelijk een 'therapiesessie installatie' maken waar mensen het onderschijt zien tussen de virtuele wereld en realiteit. Het doel van de tool is het verschillen zien, maar ook om discussie te creëren. Wat zou het effect zijn als VR een onderdeel word van onze maatschappij. Wat heeft deze tool te maken met mijn vak? Binnen animatie wordt veel digitaal werk geproduceerd, ik geloof dat VR daar een grote rol in gaat spelen. De kans dat ik later werk maak in een virtual omgevingen is niet heel ver gezocht Daarom denk ik dat het belangrijk is om alvast over te discussiëren. Met de komst van VR zijn mensen net als ik erg enthousiast over de mogelijkheden. Door mijn onderzoek heen lees ik vrij weinig over de negative impact die VR kan brengen, daarom geloof ik dat deze tool daar bij kan helpen

Voor nu wil ik meer experimenten gaan doen in VR, uitzoeken hoe interactie binnen VR kan verschillen met de echte wereld.