A presentation at I T.A.K.E Unconference in in Bucharest, Romania by D. Schmudde
Aesthe cs and Narra ve: Programming What Cannot Be Programmed @dschmudde h p ://schmud.de
Lambda Calculus Lisp Scheme Clojure Haskell
λ REPL
λ Symbolic Expressions
λ Composable g(f(x))
λ Pure
λ Reason About Code Finance (Quan ta ve) Data (Quan ta ve)
Quan ta ve Reasoning Mathema cally Provable Correctness Performance Cogita ng Behavior Improvements
λ Long History in Abstract Domains Aesthe cs Story
Art
Process Chaos and Surrender Decisiveness and Agency
The Decisive Ar st Know the Rules Before You Can Break Them
The Decisive Programmer Know the Rules Before You Can Write Them
We Don’t Always Know the Rules
AARON Development: 1968 ‐ 21st Century FORTRAN → C→ Lisp (CLOS)
1983
1992
Color Composing a Small Number of Primi ves
Process AARON Consider All Possibili es and Decide
Process Ar st Manifest Intent and Respond
Intelligence? Crea ve?
Autonomy “ I want to make it clear that in trying to model Harold Cohen “ cognitive behavior in a computer program […] I was not a cognitive scientist doing research, I was an artist attempting to give form to what my own cognitive experience had told me.
Autonomy Harold Cohen (1 May 1928 – 27 April 2016) Pain ngs by AARON (1968 ‐ ?)
Zork Development: 1975 ‐ 1982 FORTRAN (Adventure) → MDL (PDP‐10) → ZIL (Z‐Machine running Zork I‐III)
1975 Colossal Cave Adventure Verb‐Noun Commands go west
1977 Zork Preposi ons and Conjunc ons Direct and Indirect Objects fill the bo le with water
ZIL <OBJECT LANTERN (LOC LIVING-ROOM) (SYNONYM LAMP LANTERN LIGHT) (ADJECTIVE BRASS) (DESC “brass lantern”) (FLAGS TAKEBIT LIGHTBIT) (ACTION LANTERN-F) (FDESC “A battery-powered lantern is on the trophy case.”) (LDESC “There is a brass lantern (battery-powered) here.”) (SIZE 15)>
Parsing and Language Small Number of Primi ves
Autonomy Narra ve Guide Natural Language
Making a Movie
Film Script: A Technical Document Clarity and precision for interpretation: Production breakdowns Camera shooting scripts Direction for actors and directors Basis for novelizations
Autonomy script ≠ film code ≠ execu on
Quan ta ve Acts of Crea vity
Every Icon (1997‐present) John F. Simon, Jr.
Super Mario Bros. (1985) Shigeru Miyamoto
Qualita ve Reasoning Unknown Quan es Explora on of the Idea
Borderless Kinect Overtone
Abstract Domain
Transducers → Input personEntered personUpdated personWillLeave
→ Input 0: pid; 1: oid; 2: age; 3: centroid.x; 4: centroid.y; 5: velocity.x; 6: velocity.y; 7: depth; 8: boundingRect.x;
→ Input 9: boundingRect.y; 10: boundingRect.width; 11: boundingRect.height; 12: highest.x 13: highest.y 14: haarRect.x; 15: haarRect.y; 16: haarRect.width;
Output → vca (loudness) reverb (timbre) vco (pitch) attack (timbre) sustain (timbre) release (timbre) gate (loudness) silence (loudness)
A/V Flow → Input: Kinect Vision Δ Internally: Clojure: Operate on Primitives → Output: Overtone Sound
Δ Opera ons osc/person-enter ∈ @person-sound{id-1 “vowel-1” id-2 “vowel-2” …} (osc/person-updated ‘(id age)) ∃ @person-sound{id …} × (vowel (map #(* age %) [amp verb osc])
Complexity (definst drone-ae-sus “I make the ‘ae’ vowel sound at a given frequency. I start/stop with the gate set to 1 or 0.” [freq 100 gate (synth-defaults ::gate) amp (synth-defaults ::vca) verb (synth-defaults ::reverb) kr-mul (synth-defaults ::vco)] (let [kr-mul eq-freq hpf-rlpf q synth-unit (:value kr-mul) [270 2290 3010] [600 8000 0.6] 0.1 (synth-unit-layered freq eq-freq q kr-mul)] (synth-filter-chain synth-unit amp verb gate hpf-rlpf)))
Art Qualita ve Reasoning Know the rules before you can break them
Art Qualita ve Reasoning Know the rules before you can break them Constraints breed creativity
clojure.spec Integrated System For Specifica on and Tes ng
spec Scien fic Constraints (s/def ::frequency (s/and number? #((control-range 20 20000) %) ))
spec Aesthe c Constraints (s/def ::vca (s/and number? #((control-range 0.4 1) %))) (s/def ::reverb (s/and integer? #((control-range 0 1000) %))) (s/def ::vco (s/and integer? #((control-range 0 25) %)))
Autonomy testable experiment‐able with spec
Autonomy Unexpected Paths Unexpected Results Expressive Tool
Autonomy Ar ficial Intelligence (Computers)
Autonomy Augmented Intelligence (Humani es)
Autonomy & Authorship
Aesthe cs and Narra ve: Programming What Cannot Can Be Programmed @dschmudde h p ://schmud.de Licensed under a Creative Commons license.
Declarative programming has been the style of choice for implementing countless creative applications, from “Zork” to Harold Cohen’s “AARON.” We’ll explore why it helps to reason about machine creativity in this way and use Clojure’s Overtone toolkit and clojure.spec to illustrate abstract concerns and domain intelligence.
The content of this talk will be based on my successes and missteps building art installations and writing software concerned with aesthetics and narrative.
The following resources were mentioned during the presentation or are useful additional information.
Here’s what was said about this presentation on social media.
for free. You
can too.