Tuesday, December 1, 2015

A Day in the Life...

This has been an amazing autumn. I am doing some of the most creative work I have ever done in my career, and at the same time it has been a highly productive period, generating six full peer-reviewed papers over the past year, with three-plus more in the works. So, a typical day :

1. Designing erotic accessories for people with disabilities : This is a two-year project funded by Quebec's Fonds de recherche québécois en société et culture (FQRSC), led by my collaborator Dr Ernesto Morales and involving, in addition to myself, our sexologist collaborator Dr Frédérique Courtois. We organized a series with people with a range of disabilities, both men and women, to discuss their masturbation practices and the difficulties and challenges they face. Based on these interviews, we have elaborated some preliminary designs. Most of these are of the "handle" type, that is, extensions that connect to existing sex toys that allow people with a range of disabilities to use these more effectively. In addition, some actual sex toys are also under development where there don't appear to be any commercial ones available. The work is inspiring in the sense that we are clearly serving a need that exists and for which little work has been done in the past. Our work in this area will probably provide a kind of basis for encouraging other projects in this area over the coming years. Furthermore, the designs we are producing can be either 3d printed directly, or one can 3d print molds that are used to construct the object from non 3d-printed materials such as silicone gels.

2. Understanding how children with and without disability move together : This is a three-year project funded by the Canadian Institutes of Health Research under the leadership of my collaborator Dr Coralee McLaren and involving, in addition to myself, Dr Tom Chau and Dr Barbara Gibson from Holland-Blooview Kids Rehab and Dr Cheryl Missiuna from the University of Waterloo. The project involves bringing a group of children, both with and without disabilities, together within a dance exercise that encompasses both structured and unstructured movement exercises. We will be monitoring the children's neurological responses via their own movements, but especially as they observe and respond to the movements of other children. Although we have not yet begin the full data analysis phase, we have used the EEG cap to study one child's responses to the movements of others, and the results, even for this one individual, are fascinating. Certain observed movements trigger "motor imagery" related to those movements, but not all movements have this effect. It seems that children "feel" certain movements of their peers more than others, and these "felt movements" trigger them to move in similar ways (a kind of mimicking). These observations are changing how we understand the way children play together. There will be much more to come, but even these preliminary results are intriguing.

3. Designing interactive and immersive installations for people with disabilities : Artists across the world are designing and implementing immersive and interactive installations that provide unusual experiences to participants. These installations are often not accessible to people with disabilities. In addition, the experience of disability can itself generate new possibilities for unusual experiences for those without disability, and unusual experiences of the body can lead to new insights for people with disabilities about their own interactions with the environment. A Ph.D. student, Ms. Afnen Arfaoui, has taken on the challenge of developing a design methodology for creating interactive and immersive installations that address issues of disability. She is drawing on Alfred North Whitehead's "process philosophy" in order to structure this design process. This is leading to a novel approach to design of these unusual installations, an approach which integrates both scientific preoccupations and artistic sensitivities.

4. Designing a participative, interactive opera : Given today's technology and the possibilities offered by social networking, it seems obvious that the traditional presentation formats of performance art - theatre, music, opera, dance - should evolve. Modifying the formats of public presentations of these art forms is not easy, however - these have settled into "ways of doing" that are normative and generally accepted by the public. We propose to open up the opera form to new forms of interaction with the audience. To do this, we are working with an unpublished manuscript (see the discussion of the opera project below), a science fiction story still under development. The manuscript is part of a "vast narrative" type project, that is to say, the story is told across 15 books in over 2500 pages, and parts of the story are finding their way into a variety of formats, including game environments and visual art (paintings). Furthermore, the story draws inspiration from the Greek myths and legends of gods and heroes, in particular the stories of Agamemnon Atreus and his family (the Oresteia), of Jason and Medea, of Odysseus and his compatriots and family, and of Orpheus and his family. The Greeks "invented" dramatic presentations, but in their day the public played an active role in the presentation of theatre. We propose to restore this participation. Right now, the issue we are struggling with is how to "segment" the story (in essence, the libretto) into short pieces which can be recombined in different ways through audience choices. The segments have to work not just as text, but also musically and choreographically. You can't just take an arbitrary piece of music and move it around, the result will be incoherent. Choreography also imposes constraints on the process. So as a team we are working on finding a kind of "basic unit" that incorporates the constraints on music, dance and text recombination.

5. Ongoing projects in other areas : Beyond these main challenges, which tend to be present every day in one form or another, several other projects are also ongoing. As part of the effort of developing smart garments and intelligent environments, I am drawing on ecoscience theory to develop a new understanding of complex sensor systems to assist the design process, and to recognize the need to address issues of personal vulnerability as these systems encroach on the highly personal spaces of the body. Another project under development focusses on designing techniques to test and enhance the combined effect of viewing and feeling the body, as a means to improve movement retraining following stroke, for example. In a third area we are looking at how to recruit adolescents with disabilities for a study of virtual reality as a tool for skill development for eventual employment. And this work excludes my ongoing writing initiatives and my work on fashion design...

Tuesday, August 25, 2015

Pinnacle - A One Act, Massively Participatory Opera

The participatory opera project is such an interesting effort it deserves a blog entry of its own. The convergence of several completely separate initiatives, three under my direct purview and one from another team, the project integrates these into one unifying initiative. Hence, over the past five years, in addition to my ongoing research at the intersection of the geomatics and rehabilitation sciences, on the one hand, and performance and media arts on the other, I have undertaken an ambitious writing effort as well as developed and brought to market two fashion collections based on my own designs.

The writing project consists of developing a "vast narrative" of some 2500 pages, involving hundreds of characters, planets, and habitats and more than 250 years of "future history) - about 65% of this project has been completed. The story is inspired by several of the Greek myths - the stories of Orestes and Elektra (immortalized in Euripedes' Orestia), of Orpheus and Eurydice, of Ulysses and Penelope, and of Jason, Medea and the Argonauts are the main narratives involved. The opera focusses on the first 100 pages of this story, essentially the youth of its central character, Oreph Sodenheim (based on both Orestes and Orpheus).

In addition to the scenario (from which the opera's libretto is derived), we propose to provide opportunities for large sectors of the public to participate both in the development of the opera and in its final delivery on stage, via an online web access we will create for the project. Opportunities to influence the scenario, the music, the choreography and parts of the staging will be provided. In addition, we are developing a smart garment that will incorporate movement sensors and which will serve as a specialized interface for the opera development - the garment will be provided to targeted audience members. The project involves several researcher-artists from Université Laval's Department of Music (Jocelyne Kiss, Serge Lacasse, Sophie Stévance) as well as collaborators from outside the university. An innovate project which will act as a showcase for innovation for related research at Laval University.

Friday, July 3, 2015

Research Nirvana?

Have I died and gone to researcher heaven? The set of projects on which I and my collaborators are currently working feels very much like that could be the case. Here is a brief overview :

1. The Winter Project : This project is the brain child of my colleague, Dr. Ernesto Morales. Essentially, we are working in collaboration with the city council for Quebec City and a range of organisations for people with disability, to redesign city policies, including snow removal practices, to make the city more "disability friendly" during winter conditions. Early results from the project include a redesign of pavement ramps to include a drainage system allowing snow to be melted and removed - pavement ramps are of key importance for people in wheelchairs and are often the site of an accumulation of snow and ice ; the development of a wheelchair-adapted snow-mobile device ; and discussions concerning changes to snow removal practices.

2. Pro(x)thèse : This project, under my oversight, seeks to create an interactive smart garment that is embedded within an active, immersive environment, to facilitate the exploration of self image in relation to sensuality and sexuality and disability. A prototype poncho has been developed which incorporates 12 pressure sensors, interface software has been completed, and photography of people with disability has been acquired. The project includes collaborators Dr. Frédérique Courtois, Dr. Ernesto Morales and Ms. Nancy Dubé.

3. Sex toys : Another collaborative project with Dr. Ernesto Morales, in this project we are designing aids that can be combined with existing sex toys, as well as developing new ones, that provide masturbation ability to people with disability who are often unable to use existing sex aids. The designs can be printed on a 3d printer, ensuring privacy and customizability.

4. ProBE : ProBE (Proximal Body Environments) seeks to develop a diagnostic tool for clinics to assess the presence of brain trauma, both mild and more severe, in patients. The project is predicated on the observation, tested across decades of research on obstacle avoidance strategies, that people who have even mild brain trauma move differently in space and in proximity to other objects (or people) than do healthy individuals. ProBE involves researchers from Quebec City (Drs Brad McFadyen, the leader of the project and Denis Laurendeau) and Toronto (Dr Karl Zabjek) as well as clinicians from both cities.

5. Participatory Opera : The crown of my ongoing research program, developing a massively participative one-act opera based on my own science fiction opus. The concept is to develop the opera itself in parallel with a range of tools that allow active participation by diverse publics in all stages of development as well as the final real-time performance of the opera - input concerning the music, the choreography and the story line itself. These tools include virtual and social environments and specially designed smart garments. The opera project, led by myself, involves musicians Jocelyn Kiss, Serge Lacasse and Sophie Stévance from the Department of Music at Laval University, Marie Louise Bourbeau, a lyrical singer in private practice, Dr Adel Elmaghraby, a computer scientist and dancer-researchers Erin Manning and Cora McLaren. Both myself and Dr. Manning are also fashion designers - we will also be developing the costumes for the opera.

Monday, May 19, 2014

On Designing Immersive Spaces for Rehabilitation Research

Over the course of the past ten years, my team and I have successfully designed and implemented (produced) more than a dozen immersive installations within a research focus. It is with the emergence of technologies for creating and working with immersive, interactive environments (virtual reality, augmented reality, sensor networks, wearable computing, smart environments, etc.), that the design of immersive spaces has become a viable process. Hence working with such spaces and installations is still very new. Artists have been creating interactive experiences using these technologies for a little more than a decade, while the use of these environments in research has been mostly confined to virtual reality and related technologies. The convergence of immersive technologies from disparate fields into a coherent and powerful set of tools for immersive design is no older than a few years, and, of course, just as the technologies are still evolving rapidly, research based on their use is only beginning to develop.

Research that embraces immersive design requires new methodologies and these, too, are still very experimental. Immersive spaces are different from conventional experimental designs. Immersive environments are increasingly multimodal - that is, visual and auditory and tactile or haptic. They are "ecological" in the sense of being global, holistic and systemic - they engage all of a person, not just targeted aspects. Furthermore, they often generate surprises - this is a consequence of their ecological nature.

Our experience has demonstrated that the creation of these environments demands a three phase process :








This might look like the classic life cycle of any project, but each of these phases is substantial - these are complex projects to organize. Let us examine just one of our projects to get a better sense of what is involved. I like to call this project "Virtuoso" - all of our installation projects have a distinct title. This project draws upon another key principle we have identified over the course of our development work - immersive installations that have a research (scientific) focus should serve a specific and well-identified need. This is distinct from artistic installations which may or may not serve such a need. "Virtuoso" is being developed to provide an engaging experience to adolescents with motor impairments, who experience both growing isolation and a tendency towards depression as they leave behind their childhood friends. By providing these adolescents with a virtual, online, and shared experience of architectural design within a virtual world environment, we hope not only to break the cycle of isolation and depression but potentially offer also an opportunity to develop an interest that might lead to employment.

The Design Phase of this project lasted over four years. Although this was longer than many, it is by no means atypical of the collection of installations we developed. The Design work actually consisted of two more intensive periods, each lasting from 4 to 6 months, separated by an interval during which the project was shelved. The reasons for the break were multiple, but experience shows that such breaks may be a useful part of the project development cycle (although they need not be as long as in this particular case!). Lack of resources, funding and difficulties developing a mature concept were among the reasons, and possibly also our lack of experience developing and financing complex projects. During the first intensive planning period, we organized a series of meetings with both scientists and clinicians, and agreed to develop a kind of video game experience that would include a social networking dimension as well as some form of cooperative manipulation. However, the exact nature of the video game itself never became clear.

Over the fallow period, the team pursued other projects, including several projects that harnessed the virtual world environment called "Second Life". During this work, the existence of several simulators inspired by the "Second Life" environment were identified and explored. One of these, a simulator called "OpenSim", seemed particularly interesting because it shares with "Second Life" the same simplicity of operation. The arrival of a new researcher led to renewed interest in the "Virtuoso" project, and this time the possibility of using OpenSim as a framing environment for the project gave a clear direction for development. The second period of intensive work on design was therefore able to finalize a design concept - we would lead the adolescents through the process of designing and constructing architectural projects using the virtual world environment. Furthermore, the design concept included not just the design of the immersive environment itself, but also details about the experimental protocols that would need to be followed in the third phase of the project. Indeed, we would evaluate the effect on users of the environment of the design process, compared to a control group that would play a video game.

The Production Phase involved the hiring of a summer student to do the development work. The OpenSim simulation was acquired and the necessary procedures for creating, storing and running a variety of virtual landscapes were studied and adopted. A thematic context for the project was created, pedagogical tools for guiding the young people through the process of virtual construction, and different ways for encouraging collaboration and sharing results were discussed and adopted. The summer student hired to develop the environment is an architectural student with an interest in virtual environments. The development phase is expected to last 3 to 4 months.

The Evaluation Phase is expected to take place over the course of another 4 to 5 months. This phase is, of course, completely different than the development phase - it requires very different kinds of expertise. Indeed, each of the three phases requires different mixes of expertise - this is one of the challenges posed by projects of this kind. In Phase Three, what is required are experts on experimental protocols and data analysis. Technical support for the environment itself must also be in place, and it may be necessary to make adjustments to the design of the immersive environment as a result of challenges in addressing the evaluation phase.

A brief analysis of the dozen or so projects that have been developed using this approach shows that the average duration of the Design Phase was 24 months, of the Production Phase was 10 months, and of the Testing Phase was 9 months. At least 5 of the projects were characterized by a two-step design phase with a fallow period between the two more intense design periods. From initial steps to completion took, on average, 4 years. Some of this duration is almost certainly due to working with limited monetary and human resources, but other factors included the fact that we had to invent research methodology as we progressed, that the projects require different mixes of expertise at different phases, and that creation, implementation and testing of these environments is necessarily complex and demanding work. Perhaps with the benefit of experience, it will be possible to limit the time span to less than 3 years. This is important, since most grants are awarded for three year periods - projects which extend significantly beyond three years are therefore extremely difficult to bring to term in an academic environment. Indeed the early projects in our slate of initiatives were actually taken through only to the end of Phase Two, a result of the need to provide some constraints before procedures and research methodologies became clearer. (Nine of the dozen projects completed or underway are shown in the table below. The projects which are excluded from this list are those which have not yet progressed beyond the Design Phase.)

# Project Phase Duration (mo.) Expertise
Ariadne Emerging Design
Performance designer, choreographer, researcher
Singer, dancer/choreographer, video production team
Video production team, researcher, designer
Incarnatus Design
Performance designer, singer, theatre technician, programmer, researcher
Singer, theatre technician, programmer, researcher
Virtualities in Dusseldorf Design
Designer, researcher
Researcher, designers, programmer/technician, virtual content creators, museum staff, translator
Augmented Reality for Bloorview Kids Rehab Design
Researchers, designers, programmer/technician, clinicians
Researcher, designer, programmer/technician, hospital patients
Ulysses : A Sound Geography Design
Researchers, designer, composer
Researcher, designer, composer, programmer/technician
EcoOracle Design
Researchers, designer, programmer
Researcher, designer, programmer/technician
EMIR Demos Design
Researcher, designers/programmers
Researcher, programmers/technicians
Pro(x)thèse Design
Researchers, designer, composer
Researchers, sexologist, composer, programmer/technician, fashion engineer, photographers, artists, clinicians
Researchers, programmer/technician, clinicians, sexologist
Virtuoso Design
Researchers, designers
Researchers, designers, architect, programmer/technician
Researchers, designers, programmer/technician, data analyst

It is worth noting that work of this nature may have been possible only within the context of extended funding such as within a Research Chair. The availability of significant funding levels every year over a seven year grant, obviating the need to reapply for funding every two to three years, made this effort viable. Now that procedures and methods are more well defined, such projects can be managed within shorter term funding arrangements, but the initial development work would have been very difficult to undertake within a standard academic grant environment.

Friday, May 16, 2014

EMIR Laboratory Now Functional

It has taken longer to get the lab completed and operational than expected, but this has finally been achieved. In conjunction with the efforts to complete the installation of the laboratory, we have developed five demos that showcase the potential of the laboratory to support diverse projects in rehabilitation and disability studies. These are as follows :

  1. Fragment de vie (Fragmented Life) : Demo that provides different viewpoints of disability by offering different soundtracks for the same film delivered to wireless headphones (collaboration with CinéScène Inc.)
  2. Vertiges (Vertigo) : Demo that provides an experience of vertigo by simulating walking across a narrow pathway suspended above Quebec City (collaboration with CinéScène Inc.)
  3. Deuxième peau (Second Skin) : Demo that provides an observer a vicarious experience of disability. The movements of a person in a motion capture suit are transfered in real time to an avatar in a virtual kitchen, but attempts to interact with the kitchen and manipulate objects are restricted because the avatar experiences various types of motor impairment (collaboration with CinéScène Inc.)
  4. Viscères (Visceres) : Demonstration of a body-based interface for virtual navigation of spaces (Google Street Map)
  5. Meta-laboratoire (Meta-laboratory) : Demonstration of the use of the lab to support immersive experiments, in this case, a study of the effects of heminegligence on spatial orientation and judgement in the far field (collaboration with CinéScène Inc. and Dr. Julien Voisin)
In addition to the demos, a growing number of research projects are taking place that harness the possibilities that the lab offers. I have subdivided these projects into six categories : Bimodal Environments, Trimodal Environments, Movement-based Environments, Visceral Environments, Virtual Environments and Geographic Environments.

  1. Bimodal Environments : Immersive environments that engage two major sense modalities

    Hemispheres (Hémisphères)
    Experimental study in planning stages of far-field effects of heminegligence (collaboration with Dr. Julien Voisin)
    Co-breather (Co-respirateur)
    Design and validation study underway to build and test four co-breathers for possible clinical applications. Co-breathers provide auditory-tactile immersion (collaboration with Ms. M.L. Bourbeau and C. Légaré)
  2. Trimodal Environments : Immersive environments that engage three major sense modalities

    Living Wall (Mur vivant)
    Project in preparation, aimed at developing a playful immersive installation for waiting areas in clinics for children and adolescents that offers an engaging and absorbing environment that provides motor training opportunities (dexterity exercises) for individuals with fine motor impairments (collaboration with Dr. Ernesto Morales; projected Ph.D. thesis of Walid Baccari)
    Pro(x)thèse (Pro(x)thesis)
    Project underway in design phase, aimed at developing a clinical tool for allowing people with disability to explore sexual/sensual imagery and providing the means to track image choices over time. The tool involves the use of a touch-sensitive smart garment and an immersive visual environment, and we are commissioning photos by a professional photographer (collaboration with Dr. Ernesto Morales and Dr Frédérique Courtois).
    Auric Space (Espace aurique)
    Current in the planning stage, this project seeks to provide a training environment for people who have difficulties locating sounds in their immediate environment. We will use a bimodal environment (visual and auditory and haptic) to provide cues to and test for sound location (projected Ph.D. thesis of Afnen Arfaoui)
  3. Movement-based Environments : Immersive environments that explore movement modalities

    Third Skin (Troisième peau)
    Project in planning stages that seeks to extend the work initiated in the project "Second Skin" to provide a variety of vicarious experiences of disability and ability
    Choreographic maps (Cartes chorégraphiques)
    Planned project that seeks to study how dance may contribute to emerging ideas about how children play (collaboration with Dr. Cora McLaren)
  4. Visceral Environments : Immersive environments that explore actions rooted in the engagement of the body's visceral organs

    Visceres II (Viscères)
    Project in planning stages that aims to test the hypothesis that viscerally learned spaces are more fully understood and remembered than spaces learned by more traditional means
    OrienT (OrienT)
    Planned project that seeks to use a smart garment to help people who get easily disoriented to resituate themselves in their environments (collaboration with Dr. Claude Vincent)
  5. Virtual Environments : Immersive virtual environments

    Virtuarch (Virtuarch)
    Project in design development stage that seeks to provide adolescents with disabilities who feel isolated and have a tendency towards depression, access to an environment and situation that engages them in the creative design of architectural spaces (collaboration with Dr. Ernesto Morales)
  6. Geographic Environments : Immersive environments that encourage an appropriation of geographic space

    Multimodal Online Mapping Interface (Interface multimodal pour la cartographie en ligne)
    Project in planning stage that seeks to design and implement an on-line multimodal interface for a mapping application that draws on cognitive design principles (collaboration with Dr. Mir Mostafavi; projected Ph.D. thesis of Bilel Saadani)
These different projects seek to serve a variety of populations of people with impairments, including the deaf, the deafblind, the blind, those with low vision, people with either gross or fine motor impairments, people with attention deficits and people with intellectual impairments. Furthermore, different projects have different scientific goals and involve distinct methodologies. These include projects that are experimental or involve evaluation and assessment (Hemispheres, Visceres, etc.), projects that have more educational or pedagogical objectives (Second/Third Skin, etc.), projects that seek the development of assistive technologies or that are focused on design issues (OrienT, etc.), projects aimed at developing training environments (Auric Space, etc.), projects aimed at enhancing personal development among those struggling with issues of disability and impairment (Virtuarch, Pro(x)thesis, etc.), and projects that are far more exploratory in nature (Choreographic Maps, etc.).

Friday, August 17, 2012

The EMIR Laboratory Nears Completion

The Exploration of Media Immersion for Rehabilitation (EMIR) Laboratory is in its final stages of implementation before going "live" over the coming fall. The project received a significant new impetus in 2012 with the engagement of the company Cine-Scène, which has helped us line up a series of "showcase demos" that explore the use of the lab for physical rehabilitation in a variety of areas and ways.

The lab is now organized into seven distinct "stations", each providing different immersive ensembles for the user :

  1. Interactive Floor
  2. Stereo Panorama with Motion Capture
  3. Tactile Interface
  4. Co-breathers
  5. Forward Spatialized Sound System
  6. Rear Spatialized Sound System
  7. Physiological Interface

In addition to these, 15 wireless headsets and 3 HD video cameras are available for additional initiatives. We expect to extend these supplementary systems over the come year to include other elements.

Typically, stations might be operated jointly in pairs - for example, the Stereo Panorama will be operated with the Forward Sound System, or with the 15 wireless headsets.

A coordinated controller is being incorporated into the Tactile Interface to allow a user to select the stations to use for any particular application.

With Cine-Scène, we have been constructing six "showcase demos" that will be used to promote the lab to other researchers and to clinical staff at the hospital. These include the development of a floor that perturbs balance, the development of a real-time avatar with reduced or enhanced mobility compared to the user, the exploration of different perceptions of the same immersive experience, the development of a laboratory experiment, the use of the co-breathers and the use of a physiological interface. I should have photos to show shortly!

Sunday, July 18, 2010

Designing Sound Spaces

Much of the work over the past several years undertaken by this chair has been concentrated towards visual and haptic experiences, but not so much towards sound experiences. Since the EMIR laboratory (see previous blog) is being equipped with a modern spatialized sound system (albeit not the most sophisticated version of such a system), and since some of the clientele of the lab have either visual or auditory challenges, it seemed and seems appropriate to investigate more actively the opportunities for designing useful and interesting sound spaces.

This work was at first hampered by the fact that we had no reliable tools for understanding auditory spaces. Over the past several years we have used several theoretical concepts to structure visual and haptic spaces (including Voronoi diagrams, a mathematical concept called a "panorama", image schemata, etc.), but these could not be readily applied to understanding how sound inhabits spaces. Sound, unlike light, moves around corners and through many quite substantial barriers. However, sound is usually limited to so-called "point sources" and we do not work with "sound images" the same way we work with visual images - that is, sounds organized spatially in matrices.

This past year I (G. Edwards) have been called upon to teach undergraduates in the geomatics program some basic physics of wave propagation and satellite orbits. While thinking over the challenges of developing a tool for understanding sound spaces, an idea emerged. This idea, after some development work, was shown to successfully resolve the problem of modeling the space for its sounds.

Our model uses a variation of the principle called "Huygen's Principle". Huygen's principle states that when propagating waves encounter an aperture, their movement through the aperture can be simulated by supposing that at each point within the aperture a new (circular) wave is generated with the same phase and intensity as the incident wave. This principle allows one to model the movement of sound waves around corners (actually light also moves around corners using a similar mechanism, but the size of the deviation is small in the case of light).

To develop a model that can be used to simplify how we understand sound spaces, we look for a "partitioning schema", that is, a way to partition space (a room, a theatre, a park, etc.) into spaces that are auditorially "homogeneous" (invariant), in the sense that the sound experience within a given region of the tiled space is similar throughout that region, but different from one region to the next. This idea is commensurate with our earlier visual and haptic models of space.

Figure 1 : Spatial partition for sounds previous to their encounter with absorbing barriers

In the model, sound experiences are generated by point sources (A,B,C and D) that propagate into circular regions. Sound sources with low overall gain generate small circles (C and D) - sources with large gain generate large circles (A and B). Barriers that the sounds may cross are shown as lines 1 and 2. A path through the space is then introduced (dashed line).

This kind of model is quite different from the results of a numerical simulation of sound propagation within a space, as the latter will produce a map of continuously changing values for the sound field. Numerical simulations can be very useful, but that take large chunks of processor time and/or high end computers to "do the job", and usually they also have to make a variety of simplifying assumptions to converge on a result within a reasonable time. The use of qualitative models, such as ours, which segment space into regions, can provide a useful, even powerful alternative to numerical simulation.

Our qualitative model of sound space, however, differs from our earlier models in that we are required to dynamically update the partitioning each time the sound passes across a barrier that dampens the signal. Hence, as long as the observer is located on the path previous to the location labeled "alpha", the sound sources as shown in the above diagram hold true. Once the observer passes alpha, however, the sound sources "behind" the barrier made by lines 1 and 2 and their extensions must be modified (see Figure below).

Figure 2 : Spatial partition for sounds posterior to their encounter with absorbing barriers 1 and 2

At this point in time, the sound B must be "re-sourced" at the location B2, generating a new, much smaller circle. Furthermore, as the observer moves beyond the zone of influence B2, along a path parallel with barrier #1, and within the initial zone of influence of B, the sound source B must be "re-sourced" to a point (B3) defined by the orthogonal to the barrier #1 that passes through the location of the observer. Usually, the size of the circle for B3 will be much smaller due to the absorptive properties of the barrier #1.

Thus by representing the movement of the observer along a path as a series of partitions that change every time the observer crosses a barrier, we can construct a model that predicts the set of sound experiences an arbitrary observer will receive.

Using this model, we developed software that can compute in real time the relative intensity and the direction from which each sound is heard (that is, the sounds are "re-sourced" at a new location) for one or more arbitrary observers. Using this software, we are able to design a virtual sound space and hence generate a realistic sound experience for a fictional space.

We did exactly this as part of a conference presentation of the new theory at a recent meeting in the small town of Las Navas del Marques, in Spain. The conference was organized around the theme "Cognitive and Linguistic Aspects of Geographic Space", actually, a review of 20 years of research in this area since an earlier meeting of the same group at the same location in 1990. For this event, in collaboration with Ms. Marie Louise Bourbeau (longtime collaborator for the Chair) and also Mr. René Dupéré, the talented composer who reinvented circus music for the Cirque du Soleil in the 1980s, we developed and implemented a transposition of the Ulysses Voyages into a fictional and virtual sound space that was updated in real time using our software.

The result is a 40 minute presentation including a 20-minute "show", an interactive component that demonstrates the real time nature of the experience and an explanation of the scientific theory leading to this work.