these are all the posts under generative art...

Harping on Humans

Since I began exploring the science + art connection, I have become increasingly fascinated with the body’s relationship to music–both in its creation and in the actual experience of it. But what would happen if our bodies themselves were transformed into music-making instruments? London-based artist Di Mainstone seeks to answer that very question in her latest artistic endeavor–the Human Harp. Inspired by the iconic harp-like cables supporting the Brooklyn Bridge, Mainstone began to wonder what it would be like if people could “play the bridge”:

As I listened to the hum of the steel suspension cables, the chatter of visitors and the musical ‘clonks’ of their footsteps along the bridge’s wooden walkway, I wondered if these sounds could be recorded, remixed and replayed through a collaborative digital interface? Mirroring the steel suspension cables of the bridge, I decided that this clip-on device could be harp-like, with retractable strings that physically attach the user or Movician’s body to the bridge, literally turning them into a human harp.

Human Harp Module

The Human Harp module.

The Human Harp places the user at the center of a network of strings, with each string responsible for a particular sound. To create this faux harp, the participant is connected to a series of retractable strings that are housed within a small clip-on module that detects how far the string is being pulled + at what angle the string is being drawn, and records this information on an Arduino microcontroller. These measurements are then integrated to control the mood, volume, pitch and intensity of each string’s given sound by first translating the data into Open Sound Control messages, which are then read by sound-synthesizing software. Consequently, the resulting sounds come to reflect the quality of the user’s movements, effectively engaging the participant in a collaboration with the strings to create music.

Last fall, Mainstone began assembling a team of artists + engineers to workshop and create the Human Harp interface. In May, Mainstone was joined by dancer Hollie Miller in New York to debut the first version of the Human Harp for The Creator’s Project. At the time, the prototype was too sensitive to be showcased outside of a controlled studio environment, so Mainstone + company demonstrated the concept through a silent performance on the Brooklyn Bridge. The sounds were then reverse engineered to match Miller’s movements, illustrating how the interface will ultimately look + sound. In its final iteration, the Human Harp will tour the globe, where it will be installed across a range of suspension bridges. Though participants won’t literally “play the bridge,” the sounds of the harp will be sourced from local musicians + live recordings of urban spaces to synthesize a site-specific soundscape composed by human movement, creating a wholly unique sonic // visual // interactive experience.

Generative Storytelling

Last month, I had the great honor of putting all my art + science talk into practice, joining twelve up-and-coming artists in a multimedia art show curated by artists Gina Pollack and Anu Valia. Entitled You Are Here, the show–which was on exhibit at 3rd Ward from July 18th to the 21st–featured original work designed to spark a dialogue about our relationship with technology and the consequences of its pervasiveness–the havoc it can wreak on our attention by creating a series of parallel + disjointed virtual realities // the beauty of how our brains have adapted to sift through + process this information overload // the quiet reminders of what it means to be fully present in this hectic age.

In particular, my interest in the show stemmed from my growing preoccupation with how our intimate relationship with technology has reshaped the way we interact with one another. Since starting this blog, I’ve become increasingly conscious of how technology has added a new + rather strange dimension to my social interactions. The more I’ve embraced social media, the more comfortable I’ve become sharing my thoughts with anyone plugged into the Internet without even a moment’s thought. It’s incredible how many interactions I’ve had over the last year with people I’ve never once seen in person–and with a degree of familiarity I find rather hard to muster with acquaintances I’ve made in real life–likely because I’ve found some strange comfort behind the veil of quasi-anonymity that comes with interacting via social media.

With all that in mind, I approached Gina and Anu with an idea to create an installation for the show that would use technology to facilitate face-to-face human interaction–an experiment in what would happen if a computer program forced you to engage with a complete stranger in person. With their blessing, I started coding an interactive computer program [written in Python and executed in Terminal], which I came to call The Generative Storytelling Project. The program was designed to motivate gallery-goers into engaging with one another to generate some collaborative narrative–a technology-mediated effort to inspire communal storybuilding. To do so, the program first randomly selects a photograph and then prompts participants to share a sentence based on the image before them. Once someone has contributed a sentence, the computer instructs them to find a stranger, introduce themselves, and bring said stranger over to the computer to make their own contribution to the unfolding story. At some point, a user would decide that the narrative was satisfactorily complete, at which point a new image would then be selected so the cycle could begin once more.

By the end of the You Are Here opening night, participants in “The Generative Storytelling Project” had crafted over twenty stories. Below, I’ve shared some of my favorites along with the images that inspired their collaborative creation!


25

The imperalist statue looked upon the harsh cold winter and stared avidly upon threes with poignant gaze. He wondered how did I get here who made me and what is my prupose. His existential thoughts continued to flow coursing through his veins as the harsh reality begun to sink in that he will never be a mortal only a statue. Just as he was begining to panic, he woke up suddenly from the nightmare that seemed so real only a moment ago. I think I’ve had similar feelings before. All my life I have wondered who I truly am.


15

My fallen soldier! I reach but cannot reach. As my brothers call out for your brothers, I try to join our families; we the stable and you afloat. Alas, we will forever be apart. But, we are only a message in a bottle away. I will pluck a feather and write to you as soon as I may. Peace be with you. As I finished reading the message my soul sank into my body. I knew I would never be the same. O lord how will this message transform the relationship that soon beseeches us all. These soldiers look like modern day Roman warriors. Why are they so close to Ellis Island?


24

I run track, so maybe that makes this all a little easier. My breathing is even and my eyes haven’t glazed yet. Tim is home. He’s home, and he’s waiting for me. Damn this field for stretching under the sun. Run Ani! That’s an interesting cloud. Is this whole thing a test of my attention to detail? Because I’m really focused on the shape of that cloud. It reminds me of the space invaders from Independence Day. I love Jerry Bruckheimer movies. I run track, so maybe that makes this all a little easier. My breathing is even and my eyes haven’t glazed yet. Tim is home. I want to go home, I want to go home. I want to go home.


7

My last reincarnation… what to do… I could light it on fire, send the other riders running and screaming into the East River. I could try to solicit the younger man sitting across from me, because YOLO. But honestly, I’ll probably just sit here and think about what I should’ve done all the previous times. I’ll break the glass, and liberate the soul. This is the night of our lives.

Fractaled Atlas

No matter where we look in the natural world, we are sure to find recurring patterns everywhere. As a result, natural scientists devote their careers to [humbly] attempting to find and define these very patterns. The most abundant of these natural motifs is arguably the fractal—a geometric structure that can be subdivided into smaller parts that look roughly similar to the whole. Take the branching pattern of the veins on a leaf as an example: zoom into one of those branches, and you’ll find that it’s reminiscent of the overarching branching structure // zoom into one of those branches’ branches and you’ll find the same thing… over and over again!

snail shell // milky way // leaf veins // motor neuron

fractals in nature. snail shell // milky way // leaf veins // motor neuron

At their core, fractals are simply the geometric result of repeating the same pattern over and over at a smaller and smaller scale—increasingly tiny patterns within a greater overarching motif. But fractals are the ultimate paradox. Though they are built on simple repetitions, they are infinitely complex. You can subdivide // zoom in // subdivide // zoom in and you’ll still see the same [or similar] patterns emerging and repeating with detail at all scales. Nature is built on these repetitions, all the way down to the subatomic level—the quarks // leptons // bosons.1

Philosophy is … written in the language of mathematics, and its characters are triangles, circles, and other geometric figures … without these, one is wandering about in a dark labyrinth. ∇Δ Galileo Galilei

The ultimate quest for a mathematician is to define simple equations with far-reaching consequences: a2 + b2 = c2 // e + 1 = 0 // E=mc2. To distill limitless complexity down into elegant + powerful formulas. Surprisingly, despite the natural abundance of the fractal form, it was not until the 20th century that mathematicians even really began investigating fractal structures and their geometry. In fact, it was not even until 1975 that mathematician Benoît Mandelbrot even gave a name to these forms! Mandelbrot is largely credited with elevating the study of fractals to its prominence today, beginning with his discovery of the Mandelbrot set.2

mandelbrot set. on loop? or forever zooming in?

Mandelbrot created his revolutionary // revelationary set essentially by assigning every point on a screen with a unique number. He then plugged each number into a formula, got a result, and plugged the result back into the formula.3 Over and over. Millions and billions of times. In this manner, Mandelbrot tracked the fate of each point on the screen; either it grew to infinity or shrank to zero through these repetitions. If the initial number shrank—was bounded— he colored the point black. If, however, it grew—or escaped—to infinity, he assigned the point a particular color based on how many times he could repeat the formula before the result became exceedingly large.

I never had the feeling that my imagination was rich enough to invent all those extraordinary things … They were there, even though nobody had seen them before. It’s marvelous … the goal of science is starting with a mess, and explaining it with a simple formula, a kind of dream of science. ∇Δ Benoît Mandelbrot

Point by point, Mandelbrot transformed a mathematical repetition into a visual, fractal form. Amazingly, you can zoom in and in on the result and find limitless repetitions of the same basic pattern so that there is complexity at all levels—infinite resolution. The Mandelbrot set is often called “the thumbprint of God” not only because it is a visual representation of the infinite + eternal, but also because it is the mathematical embodiment of so many [if not all] of nature’s forms. [Fingerprints are also yet another great example of fractals!] If beauty really is in the details, then the Mandelbrot set—and fractals more broadly—provides an infinite source of beauty.

electric sheep.

“Do androids dream of electric sheep?” One of the flock.

With the rise of computing capabilities, fractal art has gained a major foothold in the art world, as artists have begun applying this mathematical elegance to their work. Electric Sheep, the brainchild of Scott Draves, is an iterative screensaver that is continually evolving based on the aesthetic selective pressures of over 450,000 participants: “It’s all about how pattern emerges from chaos and random stuff.” The project has taken fractal art to the next level by incorporating animation into the fractal framework to generate abstract animations using the fractal Flame algorithm. This algorithm treats each pixel on a screen like a particle and moves it based on an equation that is reiterated. The animations are the result of the interactions of billions of interweaving pixels that move according to the algorithm’s instructions. These animations are then collectively ranked by participants using the Electric Sheep screen savers, and the most highly ranked—the most fit—mate to produce new animations for the next generation [an iterative process of evolution by aesthetic selection].

Given that the algorithms that code for the emergence of the Electric Sheep are based in natural forms and phenomena, it is no surprise that our computers’ dreams flow organically into and out of each other, endlessly looping into new and beautiful forms. For me, Electric Sheep is a prime example of the art inherent in these most fundamental models for the natural world. Of how seemingly abstract numbers and figures have the potential to inform and add a new dimension to art.


________________________________________________

1 Unlike mathematically generated fractals like the Mandelbrot set, fractals found in nature are not infinitely repetitive. The repetition ends at the subatomic level because, at least to our current knowledge, there is no unit smaller than a subatomic particle.

2 The Mandelbrot set was first published in Scientific American for a broad audience in 1985: “Computer Recreations: A computer microscope zooms in for a look at the most complex object in mathematics.”

3 If you’re interested in some more mathematical detail, he called each unique point c. He then plugged each value for c into the formula f(z) = z2 + c using z = 0 and calculated what number came out. He took the result, plugged it back in as the new z value and saw what number came out again. The output of the equation becomes the input for the next iteration of that operation. This mathematical process is called recursion.

Evolution by Aesthetic Design

evolution analogiesAt its most basic, evolution is simply change over time. Since life is not stagnant, but perpetually moving forward, we can make analogies to evolution for just about anything we experience. But how can we use these analogies to glean something meaningful about our experiences?

In an experiment called DarwinTunes, bioinformatician Robert MacCallum at Imperial College London put the analogy into practice in an attempt to evolve music from noise by applying basic evolutionary principles. In so doing, he hoped to gain some insight into what aural // aesthetic forces underlie audience experience of music.

For musical evolution to proceed, MacCullum and his team first generated a population of noises—the origin for [Darwinian] musicality to come. Because the origin of life was devoid of any human intervention, they used an algorithm to generate a series of computer programs, or “digital genomes,” thereby limiting their influence on the generative process. Just as our DNA genomes hold all the information needed to build us, each program specifies how to build a particular short sound loop by determining where and when particular notes are played and what instruments should play them.

But, for evolution to occur—for these loops to meld and morph into something new and different—this aural system needed some selective pressure acting on it. Instead of evolution by natural selection, MacCullum’s group recruited almost 7,000 volunteers to lend their ears for an experiment in evolution by audience selection. Volunteers listened to and rated sound clips for their musical appeal on a scale of 1 [can’t stand it] to 5 [love it!] and the top 50% were chosen to pair off and “mate” with each other—”survival of the funkiest.”

During these sonic sexual encounters, bits of the computer programs that contain the information needed to make the sounds get exchanged, shuffled around, and even mutated to make the next generation of sound loops. Remarkably, the quality of the loops increases rather rapidly in the first 900 generations, transitioning from nonsensical noise to what could qualify as music. And all without a composer! Of course, the results don’t hold a candle to Beethoven or the Beatles. [In music, intelligent design // divine intervention still prevails!]

survival of the funkiest.

Nevertheless, what emerges resembles musical sounds we’re more accustomed to. In fact, these loops amazingly exhibit a rhythmic complexity and share chords commonly used in popular music. Unconsciously, audiences were picking sounds that converged on music that we hear all the time!1

Of course, at some point music [and art in general] becomes a subjective experience so that enjoyment relies on individual aesthetic tastes. Consequently, after those first 900 generations, the average musical appeal of the sound loops level off at an average rating of about 3 and not the max rating of 5.2 MacCullum poses a few detailed explanations for this leveling off, but to me [jargon aside], this aural plateau is the result of a sort of mass consensus as to what the baseline is for musical enjoyment. A sort of stabilizing selection by consensus. Because the sounds are generally more pleasant and familiar to the listener, the individual’s taste begins to play more of a role in selection. As a result, the average musical appeal remains the same, while individual musical appeal is all over the place.3

That great, mighty current of evolution which is advancing the life of everything in creation is simply invincible—no one can resist it. ∇Δ Maharishi Mahesh Yogi

The truly amazing part about MacCullum’s experiment in music synthesis is really not that he was able to evolve music from randomly generated and rather unpleasant noises. Instead, it’s that his team was able to apply the most pervasive natural phenomenon—that thing which is responsible for life on Earth as we know it—to replicate and study a musical phenomenon. To use patterns found in nature to observe the emergence of patterns [however much we may take them for granted] found in music: the emergence of certain common chords, rhythms, musical structures.


________________________________________________

1 Of course, this is not wholly surprising considering that our ears have been conditioned to be more receptive to certain kinds of musical sounds; however, for these sounds to be so pervasive they must have some intrinsic appeal to us [sometimes even causing us to release the feel good neurotransmitter dopamine on listening].

2 Results from this experiment in music synthesis are detailed in: Robert MacCullum et al. “Evolution of music by public choice.” PNAS: 2012.

3 To create your own individual evolutionary music, check out digital artist // programmer // researcher Jeffrey Ventrella’s musical gene pool!