User:Noemiino/year4

Info

NOEMI BIRO 

 0919394@hr.nl 

 Graphic Design

Introduction

As a graphic designer I am trained to look at small visual details and make adjustments to them. I am interested in these details not just from a human perspective but how our new technologies enable us new ways of exploration. I want to include technology in my work as much as possible in form of interactions, new layers or as research experiments.

I look at digital craft and see the opportunity to work with my combined interests: analog + digital in one hybrid and that excited me. I am curious about how technology enables machines to recognize, think, design (?) and how humans can create conditions for these interactions to happen. In my opinion when technology is used interaction is already created, left for the designer is to make sure the conditions are the framework in which they happen.

Project 1_Critical Making exercise_Treeincarnate

Reimagine an existing technology or platform using the provided sets of cards. In the first lesson of this project, our group used the cards provided by Shailoh to choose a theme, method and presentation technique. Picking the cards randomly gave us a combination of cards that spelled 'Make an object designed for a tree to use Youtube Comments and use the format of a company and a business model to present your idea'.
Group project realised by Sjoerd Legue, Tom Schouw, Emiel Gilijamse, Manou, Iris and Noemi Biro

RESEARCH

We used this randomly picked cards to set up our project and brainstorm about the potential theme. Almost immediately several ideas popped up in our mind, the one becoming the base of our concept being the writings of Peter Wohlleben, a German forester who is interested in the scientific side of trees communication with each other. The book he wrote, The Hidden Life of Trees became our main source of information. Basically what Wohlleben is researching is the connection between trees to trade nutrients and minerals in order for their fellow trees, mostly family, to survive. For example the fact that older, bigger trees send down nutrients to the smaller trees closer to the surface which have less ability to generate energy through photosynthesis.

Examples

Using their root network to send and receive nutrients. But not completely by themselves, the communication system, also known as the 'Wood Wide Web', is a symbiosis of the trees root network and the mycelium networks that grow between the roots, connecting one tree to another. The mycelium network, also known as Mycorysol, is responsible for the successful communication between trees.

Other scientists are also working on this subject. Suzanne Simard from Canada is also researching the communicative networks in forests. She is mapping the communication taking place between natural forests. Proving the nurturing abilities of trees, working together to create a sustainable living environment. A network where the so-called 'mother-trees' take extra care for their offspring but also other species, by sharing her nutrition for those in need.

Artist, scientists and designers are also intrigued by this phenomenon. For example Barbara Mazzolai from the University in Pisa has had her work published in the book 'Biomimicry for Designers" by Veronika Kapsali. She developed a robot inspired by the communicative abilities of trees and mimicking their movements in the search for nutrition in the soil.

• 

  The Hidden Life of Trees by Peter Wohlleben

• 

  Illustration from the book

• 

  Suzanne Simard's explanation of the tree root system

• 

  A diagram from Suzanne Simard

• 

  Barbara Mazzolai's Robot Plant

• 

  Technical overview of the inner parts

The Bios pot with the application in use

Bios pot

The idea of presenting this project within a business model introduced us the company Bios. A Californian based company which produces biodegradable urn which can become a tree after planting the urn in the soil. We wanted to use this concept and embed this in our project. The promotional video could provide us with interesting video material for our own video. Besides the material, we were inspired by their application that was part of their product.

CONCEPT

A visualization of trees communicating through nutricices

Error creating thumbnail: File with dimensions greater than 12.5 MP

Using the research about the Wood Wide Web and the ability of trees to communicate with each other. We wanted to make the tree able to communicate with us. Using the same principle of sending different kinds of nutrition depending on what the tree wants to communicate, such as 'danger' or 'help me out'. We wanted to let the trees talk to us. Using a digital interface logged into the root network of the tree and communicate with each other.

At first, we wanted to give the tree a voice with giving it the ability to post likes using its youtube account. Where in- and output would take place in the same root network. But is a tree able to receive information we can? And if so, what will it do with it? We wanted to stay in touch with the scientific evidence of the talking trees and decided to focus on the other application within the field of human-tree communication.

An other desire of our team was the fact that we wanted to present a consumer product taking the role as a company trying to sell our product for the global market. After researching other products concerning plants and trees, we found the biodegradable Bios urn. An urn which use as a pot to plant a tree or plant, which can later be buried in the ground somewhere. This product inspired us to use this example as our physical part of the project. So we wanted to construct a smart vase which had the technical ability to sense the chemical secretion from the roots and convert these to a positive or negative output. The input would also take place using the built-in sensors in the vase using a wireless internet connection.

PROCESS

Project 2 _ Cybernetic Prosthetics

In small groups, you will present a cluster of self-directed works as a prototype of a new relationships between a biological organism- and a machine, relating to our explorations on reimagining technology in the posthuman age. The prototypes should be materialized in 3D form, and simulate interactive feedback loops that generate emergent forms.
Group project realised by Sjoerd Legue, Tom Schouw, Emiel Gilijamse and Noemi Biro

RESEARCH

We started this project with a big brainstorm around different human senses. Looking at researches and recent publications, this article from the guardian << No hugging: are we living through a crisis of touch? >> raised the question of touching in our current state of society. What we found intriguing about this sense is how it is becoming more and more repressed to the technological interfaces of our daily life. It is becoming a taboo to touch a stranger but it is considered normal to walk around the streets holding our idle phone. Institutions are also putting regulations on what is considered appropriate contact between professionals and patients. For example, if a reaction to a bad news used to be a hug, nowadays it is more likely to pat somebody on the shoulders than to have such a large area connection with each other.

Based on the above-mentioned article we started to search for more scientific research and projects around touch and technological surfaces to gain insight into how we treat our closest gadgets. This recent article from Forbes magazine relates the research of Alicia Heraz from the Brain Mining Lab in Montreal << This AI Can Recognize Anger, Awe, Desire, Fear, Hate, Grief, Love ... By How You Touch Your Phone >> who trained an algorithm to recognize humans emotional pattern from the way we touch our phone. In her official research document << Recognition of Emotions Conveyed by Touch Through Force-Sensitive Screens >> Alicia reaches the conclusion:

"Emotions modulate our touches on force-sensitive screens, and humans have a natural ability to recognize other people’s emotions by watching prerecorded videos of their expressive touches. Machines can learn the same emotion recognition ability and do better than humans if they are allowed to continue learning on new data. It is possible to enable force-sensitive screens to recognize users’ emotions and share this emotional insight with users, increasing users’ emotional awareness and allowing researchers to design better technologies for well-being."

We looked at current artificial intelligence models trained on senses and we recognized the pattern that Alicia also mentioned: there is not enough focus on touch. Most of the emotional processing focuses on facial expressions through computer vision. There is an interesting distinction about how private we are about somebody touching our faces but the same body part has become a public domain through security cameras and shared pictures.
With further research into the state of current artificial intelligence on the market and in our surroundings, this quote from the documentary << More Human than Human >> captured our attention

" We need to make it as human as possible "

Looking into the future of AI technology the documentary imagines a world where in order for human and machine to coexist they need to evolve together under the values of compassion and equality. We, humans, are receptive to our surroundings by touch thus we started to imagine how we could introduce AI into this circle to make the first step towards equality. Even though the project is about the extension of AI on an emotional level we recognized this attempt as a humanity-saving mission. Once AI is capable of autonomous thoughts and it can collect all the information from the internet our superiority as a species is being questioned and many specialists even argue that it will be overthrown. That is why it is essential to think of this new relationship in terms of equality and feed our empathetical information into the robots so they can function under the same ethical codes as we do.

OBJECTIVE

From this premise, we first started to think of a first AID kit for robots from where they could learn about our gestures towards each other expressing different emotions. The best manifestation of this kit we saw as an ever-growing database which by traveling around the world could categorize not only the emotion deducted from the touch but also a cultural background linked to geographical location.

For the first prototype, our objective was to realize a working interface where we could make the process of gathering data feel natural and give real-time feedback to the contributor.

MATERIAL RESEARCH

We decided to focus on the human head as a base for our data collection because, on one hand, it is an intimate surface for touch with an assumption for truthful connection, on the other hand, the nervous system of the face can be a base for the visual circuit reacting to the touch.

The first idea was to buy a mannequin head and cast it ourselves from a softer more skin-like material that has a soft memory foam aspect. Searching on the internet and in stores for a base for the cast was already asking so much time from us that we decided on the alternative of searching for the mannequin head in the right material. We found such a head in the makeup industry, used for practicing makeup and eyelash extensions.

Once we had the head we could get on to experiment with the circuits to be used on the head not only as conductors of touch but also as the visual center points of the project. First based on the nervous system we divided the face into forehead and cheeks as different mapping sites. Then with a white paper, we looked at the curves and what the optimal shape on the face looks like folded out. From a rough outline of the shape, we worked toward a smooth outline and then we used offset to get concentric lines inside the shape.

   

To get to the final circuit shapes we decided on the connection points with the crocodile clips to be on the end of the head and underneath the ears. With 5 touchpoints on the forehead and 3 on each side of the face, the design followed the original concentric sketch but added open endings in form of dots to the face.

Not only the design was challenging for the circuits but also the use of the material. For the first technical prototype in which we used a grid of 3x3 to test the capacitive sensor, we used copper tape. Although this would have been the best material to use in terms of conduciveness and instead sticking surface the price for copper sheets big enough for our designs exceeded our budget and using copper tape would have meant assembling the circuit from multiple parts. The alternative material was gold ($$$$), aluminum ($) or graphite ($). Luckily Tom had two cans of graphite spray and we tried it on paper and it worked. We tested it with an LED - it is blinking by the way.
We cut the designs out with a plotter from a matte white foil and then sprayed the designs with the graphite spray. After we read into how to make the graphite a more efficient conductor we tried the tip to rub the surface with a cloth or cotton buns. The result was a shiny metalic surface that added even more character to the visual of the mannequin head.

       
   

TECHNICAL COMPONENTS

   

Adafruit MPR121 12-Key Capacitive Touch Sensor
       

EXHIBITION

It has been scientifically proven that humans have the ability to recognize and communicate emotions through an expressive touch. A new research proved that force-sensitive screens through machine learning are also able to recognize the users` emotions.

From this starting point, we created Midas. Midas is designed to harvest a global database of human emotions expressed through touch. By giving up our unique emotional movement machines can gain emotional intelligence leading to an equal communicational platform.

Through adding touch as an emotional receptor to Artificial Intelligence we upload our unspoken ethical code into this new lifeform. This action is the starting point for a compassionate cohabitant.    

       

       

Project 3 _ From Devices to systems _ The mouse

sensors and sensitivity training. You will open the 'black box' of a technical device in an anatomical machine learning lesson, and you will also dissect and analyse a concrete instance of a complex system at work. How do devices and networked systems interact? Document and research all of the parts, how they work, where they come from. Redesign and add new circuits. Put it back together with a new function and added sensor feedback loops. The basic electronics should be fully functional.

RESEARCH

An article published on Fri, Apr 28, 2000, 01:00 by The Irish Times claims that << IBM's latest mouse can understand mood swings >> These were the quotes that grabbed my attention:

After sensing the physiological attributes of the user, the information is related to emotions by using a correlation model. Your computer may then be able to use the information to adjust its settings or interface to reduce your frustration.

Human understanding and interpretation of the world around us depends to a great extent on our ability to perceive, integrate and interpret visual, auditory and tactile information.

IBM believes that computers would be much more powerful if they had even a fraction of the perceptual ability of humans or animals.

The incorporation of such abilities into computers would enable humans and machines to work together as partners, in a more complimentary way than ever before, it is argued.

However the field of "affective" technology, of which the emotion mouse is a part, embraces all parts of human expression of feelings and aims to make computers more compatible with the human condition.''

   

<< The Emotion Mouse Research document >> published on for the Conference on Human-Computer Interaction: Ergonomics and User Interfaces, Proceedings of HCI International '99 (the 8th International Conference on Human-Computer Interaction), Munich, Germany, August 22-26, 1999, reaches the conclusion that the theory is valid:

giving the computer a longitudinal understanding of the emotional state of its user, the computer could adapt a working style which fits with its user’s personality. By matching a person’s emotional state and the context of the expressed emotion, over a period of time the person’s personality is being exhibited

Fuzzy Model on Human Emotions Recognition, a more recent << [study] >>(2014) done by KAVEH BAKHTIYARI &HAFIZAH HUSAIN at the Department of Electrical, Electronics and Systems Engineering Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia discusses a fuzzy model for multi-level human emotions recognition by computer systems through keyboard keystrokes, mouse and touch-screen interactions. The system was tested on 130 participants and the conclusion was that this model represents a closer similarity between human brain detection of emotions and computer systems. This model also detects more emotions compared to the non-fuzzy human emotions detection methods, but for some of emotions a lower accuracy was obtained.

Optical mouse learn to hack - how you can handle the optical sensor inside a cheap optical-mouse for your next microcontroller project

Optical mouse cam
Optical Mouse Cam long description    

I had a USB mouse that I disassembled to have a look inside. Since it is a fairly new mouse the technical components are made not to be messed with so there is a lot of components to it that i don`t understand what they do. This mouse hsa 3 buttons - left click, right click and what usually is the scroll button when pressed this one has that button on the left side. I found intriguing the emotional mouse examples where the hack would be attaching an extra sensor to measure the depth of the touch. However, I started to think more about the meaning of the black box for me. Our computer screens are black boxes and we use the mouse to navigate inside of it.


What if I created a new interface for the mouse outside this black box? What if moving the mouse would control something outside the screen?

TECH

FRS SENSOR

info courtesy of adafruit << Force Sensitive Resistor >>
FSRs are sensors that allow you to detect physical pressure, squeezing and weight. They are simple to use and low cost. This is a photo of an FSR, specifically the Interlink 402 model. The 1/2" diameter round part is the sensitive bit. The FSR is made of 2 layers separated by a spacer. The more one presses, the more of those Active Element dots touch the semiconductor and that makes the resistance go down.
FSRs are basically a resistor that changes its resistive value (in ohms Ω) depending on how much it is pressed. These sensors are fairly low cost, and easy to use but they're rarely accurate. They also vary some from sensor to sensor perhaps 10%. So basically when you use FSRs you should only expect to get ranges of response. While FSRs can detect weight, they're a bad choice for detecting exactly how many pounds of weight are on them.
FSRs are often a polymer with conductive material silk-screened on. That means they're plastic and the connection tab is crimped on somewhat delicate material. The best way to connect to these is to simply plug them into a breadboard.
   

http://protolab.pbworks.com/w/page/19403657/TutorialSensors#ForceSensitiveResistorFSRThumbWrestling

MOTORS

To visualize the mouse movement within space I though of using motors to control their speed or angle movement based on the mouseX and mouseY positions.
   

Arduino code used for experiment

//Arduino code:
#include <Servo.h>
Servo yservo;  Servo xservo; // servos for x and y
//set initial values for x and y
int ypos = 0;
int xpos= 0;
void setup(){
 xservo.attach(10); //(analog pin 0) for the x servo
 yservo.attach(9);  //(analog pin 1) for the y server

 Serial.begin(19200); // 19200 is the rate of communication
 Serial.println("Rolling"); // some output for debug purposes.

}

void loop() {
 static int v = 0; // value to be sent to the servo (0-180)
 if ( Serial.available()) {
   char ch = Serial.read(); // read in a character from the serial port and assign to ch
   switch(ch) { // switch based on the value of ch
     case '0'...'9': // if it's numeric
       v = v * 10 + ch - '0';
       /*
          so if the chars sent are 45x (turn x servo to 45 degs)..
          v is the value we want to send to the servo and it is currently 0
          The first char (ch) is 4 so
          0*10 = 0 + 4 - 0 = 4;
          Second char is 4;
          4*10 = 40 + 5 = 45 - 0 = 45;
          Third char is not a number(0-9) so we  drop through...
       */
       break;
     case 'x': // if it's x
     /*
      ....and land here
      where we send the value of v which is now 45 to the x servo
      and then reset v to 0
     */
       xservo.write(v);
       v = 0;
       break;
     case 'y':
       yservo.write(v);
       v = 0;
       break;
   }
 }
}

Processing code used for experiment

//Processing code:
import processing.serial.*;       
int xpos=90; // set x servo's value to mid point (0-180);
int ypos=90; // and the same here
Serial port; // The serial port we will be using
void setup()
{
 size(360, 360);
 frameRate(100);
 println(Serial.list()); // List COM-ports
 //select second com-port from the list (COM3 for my device)
 // You will want to change the [1] to select the correct device
 // Remember the list starts at [0] for the first option.
 port = new Serial(this, Serial.list()[1], 19200);
}
void draw()
{
 fill(175);
 rect(0,0,360,360);
 fill(255,0,0); //rgb value so RED
 rect(180, 175, mouseX-180, 10); //xpos, ypos, width, height
 fill(0,255,0); // and GREEN
 rect(175, 180, 10, mouseY-180);
 update(mouseX, mouseY);
}
void update(int x, int y)
{
 //Calculate servo postion from mouseX
 xpos= x/2;
 ypos = y/2;
 //Output the servo position ( from 0 to 180)
 port.write(xpos+"x");
 port.write(ypos+"y");
}

MOTORS USED BY OTHERS

Servo motors and DC motors moving arm joints
       
link to working animation example in middle
Robotic arm walk through here

Zimoun : Compilation --> [1]

mechanical joint reaction, gear reaction --> instructables
        [
Kinetic gear arm - video here - [2]
Tools: 10x gear wheels(hole in middle), 5x gear joint (2 holes), min 10 joint holders - round what?

        [
Kinetic log psiral sculture --> video here more info here --> kinetic paper sculptures

Machine learning for musicians and artists _ Online course

Because of my interest in harvesting data sets, artificial intelligence and computer collaboration I decided to follow the course given online by the Goldsmith University of London << Machine learning for musicians and artists >>
I am hoping to get out of this course a better understanding of machine learning and its application for interactive exhibitions and graphical interfaces. I will be using this space to document my notes and interests found in the course.

Mentioned computer creative programming languages one might want to check:

• ChucK - A computer music programming language for real-time sound synthesis and music creation
• openFrameworks - an open source C++ toolkit designed to assist the creative process by providing a simple and intuitive framework for experimentation. openFrameworks is designed to work as a general purpose glue, and wraps together several commonly used libraries, including:
  • OpenGL, GLEW, GLUT, libtess2 and cairo for graphics
  • rtAudio, PortAudio, OpenAL and Kiss FFT or FMOD for audio input, output and analysis
  • FreeType for fonts
  • FreeImage for image saving and loading
  • Quicktime, GStreamer and videoInput for video playback and grabbing
  • Poco for a variety of utilities
  • OpenCV for computer vision
  • Assimp for 3D model loading

3 aproaches to machine learning:

• CLASSIFICATION - position controlled drum machine
• REGRESSION - hand cntrolled syntehsis
• TEMPORAL ANALYSIS - speech controlled game

   
Supervised learning algorithm as a tool that computes output values from input values. Algorithm to build it from data instead of human writing the code --> MODEL--> compute output values from input values

Interesting projects, Inpsiration

Arduino powered exhibition