Here's Blast Theory's video of us talking about what happened. I've updated a stage by stage blog below.
We skyped with Simon de Bakker from Commonplace about how to make the camera viewfinder idea into a bespoke device. Coming across their piece Bioscope, a device that allows you to hand crank digital moving-image, encouraged me to look again at the possibility of using digital moving-image rather than paper films. I also like the way their piece Lumiere, a chandelier of light bulbs containing videos of weather systems, makes the moving-image more tangible and in-the-world.
We met a dog walker who told us that fighter pilots have visual input in one eye and one eye free.
A constant conversation between Neil and I, is how the development process works. I'm all about the User Experience (the UX in developer speak) and Neil likes to think about how we can get there in achievable stages, balancing up requirements with resources.
Neil’s plan for the heartbeat pointer:
Start with core idea –
1) the heartbox takes you to a single destination while the hunter’s headphones enhance the ambient sounds (no playback of recorded sound)
2) add multiple waypoints if a specific route is important to the story
3) add a sound playback device that can be triggered by arriving at waypoints and routed through headphones. This will give a full enough experience to apply to a site and test.
4) Once the tech is stable, experiment with different qualities of haptic feedback as well as the question of having direction relate to different points of the body (like the feelspace belt).
5) Try picking up the sound of solenoid heartbeat to add to the audio
6) Try using other sounds using a waveshield to add other sounds as payoff at destination
Over lunch at the Omelette Café we talked about which medium might be best as the core indicator of direction. Dan told us about his experience of Janet Cardiff’s walk through a theatre where the participant uses the moving image in the lcd screen of the camera to navigate by by lining up the recorded visuals with the live environment, in much the same way as my flickbooks do. Dan has had some experience of trying to make sound clues for navigation, both text and non-text based (ie footsteps) and he was struck by the elegance of point-of-view visual as a method of navigation (think google street view). That made me happy as my natural medium is very visual and confirmed my first instinct with the flick books was a good one. So perhaps we could use sound to deepen the immersion and give narrative support, rather than being solely responsible for giving the directional cues.
Finally with Dan, we recorded sound sequences to support some of the visual sequences I have in mind. These included the sound of someone following a walking listener from behind at southeast and southwest and the sound of someone walking from North to South, and from South to North passing a stationary listener close by. I want to try a sequence where you hear someone coming up behind you and then see them walking past and away from you or vice versa. I’d also like to try the idea of lining up sounds at southeast and southwest as navigational cues.
Sound and navigation is a big subject and we have hardly scratched the surface.
For Day Five and Six we worked with Daniel Belasco New. We set out to see if binaurally recorded sounds could act as directional cues, soon discovering the true complexity of that question, but from this process came up with a simple way of layering live ambient sound with site-specific binaural recordings.
We did some simple experiments. Not sure they are highly reliable, but they gave us some info.
Q: Can binaural sound be used as an accurate sound beacon?
A: The jury's still out.
We recorded different sounds (clapping, clicking and rustling) at eight points of the compass in a quietish room, thinking that this would be a relatively neutral space and the sounds could be isolated more easily. We recorded the sounds at eight points of the compass at a distance of around three metres from 'listener' who stood in the centre. We played the sounds back at random to test whether we could accurately identify the direction they were recorded from. We observed:
Next we recorded the sound of falling water played on a smartphone in an outdoor site at eight points of the compass.The site was a park at school pick-up time, a place rich in ambient sound. We also recorded the sound moving from northwest to northeast, southwest to southeast, southwest to northwest and southeast to northeast as well as from North to South and from East to West, passing very close to the listener. Then we played the recordings randomly to each other using the hunter’s headphones and asked the listener to identify the direction.
From these very simple and extremely unrigorous experiments we came up with a few hypotheses and remarks:
We looked at door viewers but the angle is too wide to merge with how our eyes see.
We did mirror signalling from dusk to dark. Interesting, but not for this project as it somehow implies presence of another person and I want the person to be in solitude.
Day Four and Five were with Clara Garcia Fraile of Me and the Machine
She reminded me about content and narrative. We talked about:
the dynamics of following/being followed
enjoying the brink/jumping into the river
What part is the participant playing?
Why do they want to get to the destination?
What is the fiction around the action the participant is asked to do?
We talked about binaural sound. She said:
Binaural sound is better when linked to a visual.
Blocking or obstructing senses makes you scared/stimulates adrenalin/heightens things.
We went out with the Hunter’s headphones which amplify ambient sound and played a game where one of us led and the other followed while trying not to be seen by the leader. I wasn’t interested by the dramatic relationship between me and the performer but in the way the edges of walls and railings framed the distant person. I began looking through the view finder of my camera with both eyes open and keeping Clara in the viewfinder eye.
Hunter's headphones make things cinematic.
Q: Could you hide sounds at low volumes that only someone with the headphones can hear?
Testing the idea of a walker triggering hidden receivers ahead on the path for way-finding and layering sound or even visual content (puffs of smoke?!). Nikki called this 'breadcrumbs.'
We tested how it felt as an experience with key finders triggered by whistling.
Q: How does it feel whistling for a signal?
Q: How does it work outside?
A: I was focused on the ground, looking for the thing giving the signal. I wasn't looking out and long. Now I can let go of this idea. It's not right for the experience I'm after. Simple prototyping for no money. Result!
I am always happy to be pointed back to the visual as a solution.
Notes at the end of the day:
So how could I place a walking figure constantly at vanishing point? Through an optic? pepper’s ghost? (try it with an old cd case). It's important where we place the moving-image in our field of vision.
When we arrived at the space, Nikki had wired up her solenoid heartbeat for us to try as a feedback. We were sold immediately. It felt warm and alive against my hand.
Mission: Try heartbeat with compass and gps
Steps: Try the led with compass and gps then connect to the solenoid and learn all practical steps to make this with our components.
Here's Neil's first try at a solanoid heartbeat you can wear against your chest.
Nikki gave us a head start, teaching us not just the basics of physical computing but specifically how to make direction finding devices and animatronic/felt-in-the-body feedbacks. On the first day we made a pointer with an led light as indicator from scratch and tested it.
First lesson: soldering
Testing in the wind and rain.
It works, we got there! Northeast corner of Vale Park.
Tuning: Make the feedback more instant and clearer for the participant. Try three signals: warm, on target and destination
The residency at Blast Theory is to develop non-screen based navigation devices which draw our attention outward so that we become more alert to our surroundings, perhaps working with what we see, hear or feel on our skin as a way of indicating direction of travel. Then to add in tactile, sound and/or moving-image content, which feeds into the narrative journey.
We invited three artists help us explore solutions: Nikki Pugh, Clara Garcia Fraile and Dan Belasco Rogers.