First Contact
Christoph: Early this spring I walked across the aisle at the Medical Faculty and spotted an unusual event announced on the University’s information screens, «Creative Computing Competition 2023…C2HO…open to everyone…audiovisual artwork…text-based programming…award…». The combination of these keywords sounded interesting and puzzling at the same time. Creating art with text-based programming? Yes, I use text-based programming in my daily work and yes, I have been to art exhibitions, but I have not heard about this «wedding» before. Sounds like fun, I thought, but I probably have no time to participate in this event besides work and daily family duties. I continued to my office and did not think about the event further. For some reason however, the idea persisted, and the relentless screen reminded me every day about the upcoming event on the way to my office. I turned to my colleague André, a fellow consciousness researcher, haiku connoisseur and friend of virtual synthesizers.
André: Going for one of my many daily coffee runs I heard Christoph say, «Hey André, there is a creative computing competition at UiO, should we join?». I didn’t know what he was referring to, but creative + computing sounded fun. It was something about creating an audio-visual artwork through programming. I thought about my current synthesizer obsession, some data I was working on, and as surely as 1+2=4 the idea to marry the two was born. Then I read the rules. The result had to be generated by a single script. No outside tools allowed. Well, rules are made to be broken.
Motivation
After we registered for the event there was no backing down and we were excited to present our lab data and show the «world» how consciousness sounds and looks like. Certainly, there was an inner wish to disseminate our research to the public and create something that we can share and discuss with others without relying on static graphs, and descriptive statistics commonly seen in research articles. The idea that science could have an aesthetic appeal, thereby conveying knowledge through an artistic approach fueled our motivation to participate in the creative computing competition. However, really, it was about making “bleeps” and “bloops” instead of rigorous box-plots.
The sound of consciousness
Having an experience is at the core of who we are and how we see and interact with the world. In our research, we investigate brain processes, which separate the awake, conscious state from periods of assumed unconsciousness, such as dreamless sleep and anesthesia. After a dreamless sleep we are usually unable to report what has happened during the night; our experience has miraculously faded for a couple of hours with no sense of being aware about our surroundings or any dreams. One way to probe consciousness is to simply ask a question and check for a reply. For example, a doctor asks a coma patient «Can you hear me?». What if the person is conscious but unable to communicate or hear the question? How should this patient be treated? This dilemma highlights the importance of finding objective measures of consciousness that do not rely on sensory input (hearing, sight,..) and motor output (verbal/non-verbal communication). Luckily, instead of relying only on traditional forms of communication, we can nowadays directly assess consciousness by «listening» to the brain and recording electrical impulses from the brain surface. The mouse brain is in some respects similar to the human brain and gives us the opportunity to study consciousness and underlying brain areas and networks in depth. Similar to digital signals in a computer the brain uses a binary code in the form of electrical signals (spikes [1] vs no spikes [0]) to integrate and process information. In our artwork, we sonified and visualized this binary code, which we previously recorded in the brain of an awake, and anesthetized mouse. Christoph transformed this to audible frequencies, accompanied with a visual representation, while André sent the whole sequence to a virtual modular synthesizer to further manipulate the sound.
While interpretation of brain signals is not intuitive for the general public and even cumbersome for trained neuroscientists, the creative computing approach yielded a more accessible, understandable, and fun summary of our research. For us, creating an artwork and participating in the C2HO competition was an unique opportunity to meet new people, and socialize around our research with family, friends, and other contestants - an activity which is otherwise more isolated from the rest of the world.