Digital Craft Minor 4th Year: Cybernetics

Project 1: Critical Making exercise

Reimagine an existing technology or platform using the provided sets of cards. For this first project, I grouped up with Tutu, Thijs and Naima and we picked three random cards each from different category.

Inspiration

Our starting point

Prototype

Project 2: Cybernetic Prosthetics

Starting Point

Our first inspiration from nature, as a biomimicry was how whales communicate underwater. They communicate ultrasonic sound.

Research

There were several interesting facts on how whales communicate and various examples how the ultrasonic can be used in real world.

Insight from Research

However, we could not really drag one new idea we can work on derived from ultrasonic. We were watching some youtube video with whale sound and found that it is actually used for calming effect, and meditation usage. From this, we were thinking it would be nice to actually feel and hear as if we were under the sea. What if we make a device transform our surrounding to underwater by means of sound?

Plan

We decided to use a hydrophone, a jar, and a recorder with which we can hear the input directly monitored to the headphone. However a hydrophone was too expensive, so we asked Rob in Image and Sound station and he recommended us to cover a regular microphone with a condom.

First Testing

Making Prototype

Final Result

Aquatic Acoustic

Video Link : https://vimeo.com/292687674

Project 3: From Devices to systems

Cassette recorder to Electromagnetic noise detector https://electronics.howstuffworks.com/gadgets/audio-music/cassette2.htm

Final Project (RESEARCH DOCUMENT ✗ UNFINISHED ✗)

Position Paper_H Jung

Introduction

Extended from my former study English Language and Literature, I experiment with postmodern literary ideas in my works, such as stream of consciousness, fragmentation, and non-narrativity. I have been experimenting with different media, mainly photography, text, and sculpture. They demonstrate my perception of memory, time, and narrative, either linear or non linear. I interpret them as my trial to understand how we, as human beings, are conscious of ourselves, and surroundings to define our existence. I try to capture how I am conscious with my surroundings and intrigue audiences to question their own. Therefore, physicality, in terms of being present in the moment at the surrounding, and immersion into real space became my main subject matter. My work is about providing an environment by sending an invitation to audiences to be in the moment with the artwork. However, my work has been more of presenting art objects which derived from my own experience and has narratives behind. Since I become more aware of my recurring theme, I am eager to find the most effective way to express my contemplation. The immersive effect of new media technology is very powerful and brought us new consciousness in our daily lives. I want to make use of new media technology to provide audiences immersive effects and question the meaning of being conscious, mindful in reality at the same time. My current interest is the immersive effect of sound and video, also when they are combined.

Research Question

Sound is the most immediate and immersive media to me. It is abstract yet very tangible and physical at the same time. We respond on in directly which I conceive as a bodily reaction. Therefore, my starting point of this project was 'sound' which developed into making use of sound for an interactive installation to make people conscious of their surroundings combined with other new media technologies. Sound has been my all time biggest interest. I have been always fascinated by weird noise and electronic music. Therefore I decided to research on digital audio processing and how to produce electronic music. Besides, I chose to do something has to do with audio for the former exercises to narrow down which aspects and elements of sound I want to further develop specifically. In exercise 2, as a group, we made a device to experience being underwater, using water as an analogue sound filter for surrounding. In exercise 3, I dissembled a tape player and transformed it into an electromagnetic noise detector using the tape head reads sound when it was close to magnetic. From exercise 2 and 3, I dragged my research topic as our perception of sound around us. I pointed the hypnotising and meditating effect of sound since it brings us to another state of mind/consciousness. I also wanted to identify myself what is the border between sound art and music. Therefore, my research question became “How do we perceive sound around us? How does it affect our consciousness?”

Research Approach

My research process is mainly divided into two parts. Firstly, I read much literatures and information to find some interesting and relevant feature to me from the broad context and then narrow down to few things from which I would develop further, connecting them to do some practical exercises. Second, I do practical research on certain material, methodologies, skills by working on them. This two researches can go back and forth since I actually always come across with unexpected features and have new idea by doing practical research. It never really worked out as I assumed with theoretical and thought research, and the conclusion has always changed. Therefore, I would like to keep my research question open, but still limiting the main theme since I can be lost easily from too broad context and topics. By keeping the research question open, I can analyse the process and formulate rather specific questions inside. At the beginning of this project, I didn't know which elements and aspects of sound I wanted to further work on yet. However from the exercises I dragged my research question as "How we perceive sound around us and how does it affect us?" From this point, I will do more experiments extended from the former exercise of using hydrophone water as an analogue filter. From this, I will figure out if it works. If it doesn't, at least I know what works and not and there should be some other aspects come up afterwards, for instance from the contact microphone or the material and size of the jar, etc.

Inspiration/Reference

Sound art, using sound as an art medium, has been explored a lot in last decades. There are also many well-known and overlapping terms such as Experimental music, Avant-garde music, Electroacoustic music etc. To contextualise my topic in contemporary art, I researched the relevant history of Experimental music and Sound art, and practitioners to whom I can relate. I am quite familiar with the history of Electronic music (Dance music especially) so it was interesting to make connection between them.

Historical context

Electroacoustic Music

Electroacoustic music is a style of Western art music which originated around the middle of the 20th century, following the incorporation of electric sound production into compositional practice. All electroacoustic music is made with electronic technology, specifically a device, usually a loudspeaker, that transduce electrical energy to acoustic energy. I found this relevant to my practice since I wanted to combine electronic sound with physical, acoustic elements. Tape music was an integral part of electroacoustic music and I always wanted to try it since all digital sampling, editing techniques came from analogue tape manipulation. Tape music utilises pre-recorded fragments, loops, and sampled sounds, altering and manipulating them through techniques such as speed manipulation. In the context of electroacoustic music from the past, I found the artists John Cage and Pierre Schaeffer as references to my practice.

John Cage

John Cage was an American composer and music theorist. Cage was one of the leading, the most influential figures of the post-war avant-garde, lauded as one of the most influential composers of the 20th century. I know him from his best known composition work 4′33″ from 1952, which is performed in the absence of deliberate sound. Musicians who present the work do nothing aside from being present for the duration specified by the title. The content of the composition is not the silence of the duration, but rather the sounds of the environment heard by the audience during performance. The work's challenge to assumed definitions about musicianship and musical experience made it a popular and controversial topic both in musicology and the broader aesthetics of art and performance. This concept of Indeterminacy in music is a composing approach in which some aspects of a musical work are left open to chance or to the interpreter's free choice. I didn't know about 'prepared piano' before but it also fascinated me in the sense that it creates unusual sounds out of the piano, an instrument I can most relate and be familiar with. I listened to some of his composition works for prepared piano such and Landscape series, and it produces very weird, creepy sounds which you cannot really imagine that it comes from the piano. He explored the possibilities of prepared piano in the series of composition Imaginary Landscape. This is a video of a performance of the first one; Imaginary Landscape No. 1. image source: https://www.newyorker.com/magazine/2010/10/04/searching-for-silence

Pierre Schaeffer

Pierre Schaeffer as a French composer, writer, broadcaster, engineer, musicologist and acoustician. He is most widely and currently recognised for his accomplishments in electronic and experimental music, at the core of which stands his role as the chief developer of a unique and early form of avant-garde music known as musique concrète.
Music Concrète is a form of experimental music that exploits acousmatic listening, meaning sound identities can often be intentionally obscured or appear unconnected to their source cause. Schaeffer realised there was a rich vein of musicality hidden within such seemingly mundane sounds: snippets of complex rhythms, unique timbral characteristics, tonal peculiarities, strange and interesting textures. Our perception of these qualities, he recognized, was hindered by the associations and references that sounds inherently carry.

Today, Schaeffer is considered one of the most influential experimental, electroacoustic and subsequently electronic musicians, having been the first composer to utilize a number of contemporary recording and sampling techniques that are now used worldwide by nearly all record production companies.


image source: http://www.furious.com/perfect/pierreschaeffer.html

Contemporary practitioners

When I was researching to figure out on which aspect of sound I could work, I came across with several musicians and artists inspired me. The Netherlands itself has been a leading scene for electronic music pioneers.

Dick Raaijmakers

Phono-kinetic object

He is considered as someone who combines disciplines such as visual art, film, literature and theatre with the world of music. One of his most important books is The Method (1985), in which he describes in an exact but also poetic way how motion, cause and effect and their perception are interrelated.

Caterina Barbieri

Caterina Barbieri is a Berlin-based Italian composer who explores themes related to machine intelligence and object oriented perception in sound through a focus on minimalism. Using modular synthesisers as the main tool, she explores the psycho-physical effects of repetition and pattern-based operations in music, by investigating the polyphonic and polyrhythmic potential of sequencers to draw severe, complex geometries in time and space. Approaching music practice as an integrative cognitive feedback between humans and technology lies at the core of her current sonic research. I found her practice relevant to me since she approaches sound as a medium to stimulate consciousness. Besides, the way she uses sound pattern to create the whole piece is very interesting because it shows the process of development of the very first elements. Her concept inspired me to use patterns and repetition in the sound piece in the effective way as I intended and gave me insight about minimalism in music and process music, in which the process is audible to the listener. These processes consist of specific systems of choosing and arranging notes through pitch and time, often involving a long-term progress with a limited musical materials.

image source: https://www.ableton.com/en/blog/caterina-barbieri-minimalism-modular-and-live/

Tomoko Sauvage

image source: https://www.japantimes.co.jp/life/2017/09/30/people/musician-tomoko-sauvage-searches-freedom-sound/#.XD2LR88zbq0

Japanese musician and artist active since mid 2000’s, investigates the sculpturality of sound and improvisation in relation to the environment. Mainly known for a musical / visual research about ‘natural synthesizer’ of her invention, composed with diverse fluid, bowls, ceramic, light and underwater amplification, Sauvage’s approach is attached to questions of alchemy, meditation and balance between hazard and mastery. Under the form of performances, installations and musical compositions, her work is regularly presented in Europe, Asia and America. For more than ten years, Tomoko Sauvage has been investigating the sound and visual properties of water in different states, as well as those of ceramics, combined with electronics. China bowls of different sizes, filled with water and amplified via hydrophones (underwater microphones), waterbowls is a kind of natural synthesizer that generates fluid timbre using waves, drops and bubbles. These recipients resonate and also produce subaquatic feedback, an acoustic phenomenon that requires fine tuning depending on the amount of water, a subtle volume control and interaction with the acoustic space. Through primordial materials and playful gestures, Tomoko Sauvage searches for a fragile balance between randomness and discipline, chaos and order.

Field Research/Events

While doing researches and working on the project, I visited several events, performances and exhibitions for inspirations and see how the relevant practices are held in real world. I think it is really important to visit and see these kinds of events as much as possible because it gives me an insight of the field which I cannot learn inside the school. I want to figure out in what kind of exhibition, event, performance I would be able to position myself after the study. By doing field research, I get to know people who are engaged in this field and involved organisations which I can potentially contact later on.

Ryoji Ikeda at EYE Museum

https://www.eyefilm.nl/en/exhibition/ryojiikeda
image source: https://kunstmeisjes.com/2018/10/02/go-no-go-132-dansende-data-en-elektronische-bliepjes/

V2 Klankvorm #7

http://v2.nl/events/klankvorm-7
http://www.klankvorm.nl
image source: https://www.facebook.com/events/151927015748358/?active_tab=discussion
I am doing volunteer work in V2 and had a chance to do a shift during this event. I was interested in the organisation Klankvorm already from their concept and performance idea. There were 3 different commissioned performances which were specifically installed in V2 space. Klankvorm is a Rotterdam-based platform for the realisation and presentation of experimental audiovisual art. Klankvorm events take place in different locations and feature audiovisual performances and related presentations. Klankvorm aims to initiate, commission and present new works together with sound artists and visual artists who collaboratively explore the audiovisual domain.
  

The Rest is Noise - Alva Noto

https://www.muziekgebouw.nl/agenda/8004/_object_blue_Valery_Vermeulen/Alva_Noto_UNIEQAV/
image source: https://www.facebook.com/events/340189403221857/?active_tab=discussion

This event was part of FIBER weekends 2018, a new, decentralized art event dedicated to emerging developments in audiovisual art, electronic music, performances, creative coding, and advanced technologies. FIBER Weekends 2018 opens with a night of mesmerizing live performances at Muziekgebouw aan ‘t IJ on Friday, 23 November. Presented in collaboration with The Rest Is Noise, the opening concert sees pioneering German sound and visual artist Alva Noto unveil the Dutch premiere of his latest project, UNIEQAV, a live set by Beijing-born DJ and producer object blue, and a lecture-performance by Belgian electronic musician and mathematician Valery Vermeulen.I knew the record label Raster-Noton and listened to the collaboration album by Alva Noto and Ryuichi Sakamoto. So I really wanted to see his performance since he was combining it with coded visual. It was held in Muziekgebouw aan ‘t IJ, in Amsterdam.

WORM Instruments Make Play

https://worm.org/production/instruments-make-play-2018/
image source: https://www.facebook.com/events/471290653362501/?active_tab=discussion
  
This event was very inspiring. There were many artists/musicians/hobbyists who made there own instrument and they were showcased as an exhibition form. There were performances of them playing their own stuffs. I got inspired to develop my work to my own DIY instrument/spatial installation.

https://worm.org/production/the-sound-of-ketelmuziek-10/

Experiments

I tried several things to be ended up with my final process. First I tried to make an analogue sound filter for the space. Second, I wanted to explore algorithmic composition with coding and combined with coded visual such as Processing. In the end, I decided to make an interface/installation.

Contact Microphone

As an extension from the second project, I wanted to try out if the contact underwater microphone would work as a water filter in our surroundings. I decided to make a hydrophone myself with two piezo elements and xlr mic cable. I followed the instruction of this. The microphone worked well.

For the trial, I put it under the bucket as the former exercise, connected it to audio interface as input connected loud speakers for the output. However, it did not have the effect I wanted when I put it underwater. Because it only captured the sound from the physical contact, it was to clean and crisp for the atmospheric usage which I was imagining. I expected it to have delay, echo effects from the feedback loop of realtime sound input and the filtered output.
  
But then, I still wanted to try with some other audio effect using Ableton. I found that Resonator can work interesting with contact microphone. It made some cool sound but I could not really come up with anything I can go further with this. I also found a very interesting project from an artist Joo Won Park, called Toccata using contact microphone and SuperCollider. From this, I wanted to research what SuperCollider is.

Synthesis, Algorithmic Composition

SuperCollider https://theproaudiofiles.com/sound-synthesis-basics/

How does a synthesiser works?

Algorave

Paul Prudence

João Martinho Moura

Insights from Experimentation

The two experiments has been completely different approach. First idea was about using the existing sound around us and transforming it with water as an analogue filter, more natural, acoustic way without using programming. The second idea was about sound synthesis itself, especially in digital way with programming. I was really exited when I found the programming language SuperCollider, from the idea that I can code the sound and even composition. There were many interesting way to use this combining with other interactive elements and programmes. However, since I am fairly novice to programming and unintuitive interface for making sound and music, I gave up to use this programme for this project for efficiency issue. It is also because I didn't know that much about audio synthesis terms and elements. So I decided to move on to use Ableton again which I am bit more familiar with. But I really want to get back to learn SuperCollider at some point. The idea of Algorave, live coding party, is also very exciting. I missed one party at Utrecht but would really want to experience when it is held around me.

Process

Ableton Live

At the beginning I decided to make a composition with Ableton. Ableton is really good with sampling playing around with musical ideas. I like when found sound was used as the musical sources, so I tried out some found sound beats.

Perception of Sound, pattern, repetition

sound perception can be divided into six cognitive processes. They are: pitch, duration, loudness, timbre, sonic texture and spatial location (Burton 2015, 22–28). Nowadays we are living in the age of mechanical reproduction and it somehow changed our perspective on beauty. Whenever I walk down the streets, I hear the noise of construction works such as drills, banging hammers that become repetitive in my head and suddenly become a rhythm and then music. I decided to make a drum machine interface where audiences can create their own beat from the found sounds. Then I wanted to visualise the patterns and repetition of the beats. I wanted to visualise it as a movement using motors. I thought this way of creating optical kinetic installation would be effective to show the mesmerising effect of the repetition of the patterns.

Found sound, audio edit, effect

First I did some field recordings and manipulated them in DAW Ableton. Some of the found sound, I used it from freesound.org. I warped, cropped the sound and put audio effects to recreate the sound. I put the recorded found sounds in the drum rack to use each elements as a percussion sound.

Mono Sequencer for Synth

500px 500px I wanted to add some synth patterns as well.

Technical aspects

Electronics

By using Max for Live connection kit, I could directly connect it to Arduino for inputs and outputs. I decided to use Servo motors because they are easy to control with Ableton and I thought the movements of motors and sound wave vibration matches to create the pattern and repetition in different forms. I was planning to use 12 to 16 servo motors to control each sounds. But I didn't know that I need extra power source to use more that several motors by using Arduino since it only provides 5V and they take a lot of electricity. The easy way to use many servo motors with Arduino is using a shield. A shield for Arduino is sort of extension plug-in, which gives Arduino more functionalities. By googling, I decided to use this Adafruit 16 servo shield https://learn.adafruit.com/adafruit-16-channel-pwm-slash-servo-shield/overview
  

OSC

Open Sound Control (OSC) is a protocol for networking sound synthesisers, computers, and other multimedia devices for purposes such as musical performance or show control. OSC's advantages include interoperability, accuracy, flexibility and enhanced organisation and documentation. With the Ableton Connection Kit (https://www.ableton.com/en/packs/connection-kit/), it is easy to control motors, LED, etc via Arduino. However, since I am planning to use 12 servo motors and using a shield for Arduino, the digital outputs have been changed. So Ableton could not read the shield. I asked Mike in interaction station for a help, and he suggested me to send OSC to Processing and connect it to Arduino.

Input

I used TouchOSC software to control MIDI for Ableton Live. You can use mobiles and pads for this programme. It works with Wifi connection so I need to know the host IP of my laptop and connect it from the software with iPad. So when people touches certain button in iPad it sends the signal to Ableton, to the set sound.

Material exploration/Visual aesthetics

Using metal was pretty intuitive decision for me since I have worked with metal for a while to make sculptures. I used to make bent, plasma cut, welded geometric steel sheet sculpture to demonstrate my notion of object and space and their relationship. I like the aesthetics of cold, sharp steel sheet. Therefore it was reasonable process for me to further work with it. This time it was used to as a base panel to support the motors, Arudino and all the electronics.

    

  

Metal Sheet (Steel), Wire, Found/Scrap object, colour points, Moving creature

Realised work

Patterns of Consciousness

In the end, I made this phono-kinetic object which is controlled by input from an iPad. I made an interface as a whole, combining sound, touch and movement. The object is attached close to the wall, the typical way to display artwork. I wanted to experiment with the way of displaying, like just hang it somewhere in the air attached from the ceiling. However, since my work is in the transition of coming out of the wall as an art object, it made sense to hang it just on the wall and then implement the interaction via iPad, with people controlling in front of it. I put the iPad on the plinth in front of the object, and put a headphone next to it. So audiences control the iPad, listening and manipulating the sound.

Found sound (surrounding, environment), pattern, repetition, time-based

being in the moment, consciousness

indeterminacy

phono-kinetic object, acoustic

interaction/interface installation

movement (creating awareness, attention catching)

Reflection

From the exhibition, I figured out that my work was quite self-explaining which worked in a good way. When people wore the headphone and touched the iPad, they realised the interaction between their touch, sound and the motor. And then they played with it for a while.

However, I think there should be some ways to enhance the whole experience with each elements as a whole. The sound from the headphone was of course my main purposed sound, but there was another sound coming from the movement of the motors and their scratches on the metal sheet. This was my intention to have as well but when the audiences wear the headphone, the two sounds are disconnected.

The interface for the audience input is quite minimally intuitive. However, I could have used each colours, shapes more relevant to the physical objects. Apart from that, I actually want to make another physical input device or environment, not with a touch pad.

Besides, this time I wanted to make a spatial installation, providing scenery rather than presenting an art object. Because it make more sense to create a spatial scenery merged into the exhibition space to make people aware/conscious of their surroundings. However, I feel like I couldn't break my pattern of presenting art objects in the end. I made an interface so people could be engaged in the installation, but it could be seen as controlling with a machine/object. I want to develop my practice into rather immersive larger scale to make a large scale spatial installation as performative set up or even doing performance myself.

Final Conclusion

How I redefined my position within my art practice. Interaction became important element to me. Conversation with the moment/surrounding. To make oneself more conscious of the surrounding, to be in the moment. how to make an interactive installation using new media technology to make people aware of their surroundings by means of sound

Next step? Similar thing? mindfulness, meditation (how to be immersed in the surrounding) conversation with the environment (how we are conscious of it) transformation of the moment/surrounding spacetime effect of sound spatiality location

context of digital craft immersive effects of new media technology interaction, intriguing, invitation

pattern, repetition, time-based, process music in real time

There are several many ways I want to take from now on. Mostly, I want to develop the interactive sound installation with more spatial merged quality and various elements. I would like to study circuit bending and sound generation technique with electronics in depth. By doing so, I could make my own instrument as well as an extension of the interactive sound installation. I want to focus on the interface to make it as a whole, providing it as a performative scene. At the same time I would like to try performance with my potential sound installation/instrument as well.