Aesthetics and Narrative: Programming What Cannot Be Programmed

A presentation at Clojure/conj in December 2016 in Austin, TX, USA by D. Schmudde

Slide 1

Slide 1

Aesthetics and Narrative: Programming What Cannot Be Programmed Clojure/conj 2016 @dschmudde http ://schmud.de

Slide 2

Slide 2

Functional Programming Reason About Code Finance (Quantitative) Data (Quantitative)

Slide 3

Slide 3

Quantitative Reasoning Soft: Cogitating Behavior Improvements Hard: Mathematically Provable Correctness Performance

Slide 4

Slide 4

Functional Programming Long History in Abstract Domains Aesthetics Story

Slide 5

Slide 5

Art

Slide 6

Slide 6

Artist: Mostly Qualitative Chaos and Surrender Decisiveness and Agency

Slide 7

Slide 7

The Decisive Artist Know the Rules Before You Can Break Them

Slide 8

Slide 8

The Decisive Programmer Know the Rules Before You Can Write Them

Slide 9

Slide 9

We Don’t Always Know the Rules

Slide 10

Slide 10

AARON Development: 1968 - 21st Century Fortran → C→ Lisp (CLOS)

Slide 11

Slide 11

1983

Slide 12

Slide 12

1992

Slide 13

Slide 13

Color Composing a Small Number of Primitives

Slide 14

Slide 14

Programming Autonomy Aesthetic Choices

Slide 15

Slide 15

Harold Cohen (1 May 1928 – 27 April 2016) Paintings by AARON (1968 - ?)

Slide 16

Slide 16

Zork Development: 1975 - 1982 Fortran (Adventure) → MDL (PDP-10) → ZIL (Z-Machine running Zork I-III)

Slide 17

Slide 17

Slide 18

Slide 18

1975 Colossal Cave Adventure Verb-Noun Commands go west

Slide 19

Slide 19

1977 Zork Prepositions and Conjunctions Direct and Indirect Objects fill the bottle with water

Slide 20

Slide 20

MDL <DEFINE AXE-FUNCTION () <COND (<VERB? “TAKE”> <TELL “The troll’s axe seems white-hot. You can’t hold on to it.”> T)>>

Slide 21

Slide 21

ZIL <OBJECT LANTERN (LOC LIVING-ROOM) (SYNONYM LAMP LANTERN LIGHT) (ADJECTIVE BRASS) (DESC “brass lantern”) (FLAGS TAKEBIT LIGHTBIT) (ACTION LANTERN-F) (FDESC “A battery-powered lantern is on the trophy case.”) (LDESC “There is a brass lantern (battery-powered) here.”) (SIZE 15)>

Slide 22

Slide 22

Parsing and Language Small Number of Primitives

Slide 23

Slide 23

Autonomy Narrative Guide Natural Language

Slide 24

Slide 24

Qualitative Reasoning Unknown Quantities Exploration of the Idea

Slide 25

Slide 25

Exploration Agent Autonomy → Intelligence? → Creativity?

Slide 26

Slide 26

Making a Movie

Slide 27

Slide 27

Film Script: A Technical Document Clarity and precision for interpretation: Production breakdowns Camera shooting scripts Direction for actors and directors Basis for novelizations

Slide 28

Slide 28

blueprint ≠ bridge code ≠ execution script ≠ film

Slide 29

Slide 29

Script Pure Narrative

Slide 30

Slide 30

Film Result of Autonomous Agents Making Aesthetic Choices

Slide 31

Slide 31

Creating Agency Pure Functions Composition

Slide 32

Slide 32

Borderless Kinect Overtone

Slide 33

Slide 33

Abstract Domain

Slide 34

Slide 34

Slide 35

Slide 35

Transducers → Input personEntered personUpdated personWillLeave

Slide 36

Slide 36

→ Input 0: pid; 1: oid; 2: age; 3: centroid.x; 4: centroid.y; 5: velocity.x; 6: velocity.y; 7: depth; 8: boundingRect.x;

Slide 37

Slide 37

→ Input 9: boundingRect.y; 10: boundingRect.width; 11: boundingRect.height; 12: highest.x 13: highest.y 14: haarRect.x; - will be 0 if hasHaar == false 15: haarRect.y; - will be 0 if hasHaar == false 16: haarRect.width; - will be 0 if hasHaar == false …

Slide 38

Slide 38

Output → vca (loudness) reverb (timbre) vco (pitch) attack (timbre) sustain (timbre) release (timbre) gate (loudness) silence (loudness)

Slide 39

Slide 39

A/V Flow → Input: Kinect Vision Δ Internally: Clojure: Operate on Primitives → Output: Overtone Sound

Slide 40

Slide 40

Δ Operations osc/person-enter ∈ @person-sound{id-1 “vowel-1” id-2 “vowel-2” …} (osc/person-updated ‘(id age)) ∃ @person-sound{id …} × (vowel (map #(* age %) [amp verb osc])

Slide 41

Slide 41

Complexity (definst drone-ae-sus “I make the ‘ae’ vowel sound at a given frequency. I start/stop with the gate set to 1 or 0.” [freq 100 gate (synth-defaults ::gate) amp (synth-defaults ::vca) verb (synth-defaults ::reverb) kr-mul (synth-defaults ::vco)] (let [kr-mul eq-freq hpf-rlpf q synth-unit (:value kr-mul) [270 2290 3010] [600 8000 0.6] 0.1 (synth-unit-layered freq eq-freq q kr-mul)] (synth-filter-chain synth-unit amp verb gate hpf-rlpf)))

Slide 42

Slide 42

Art Qualitative Reasoning

Slide 43

Slide 43

Know The Rules Before You Can Break Them

Slide 44

Slide 44

Constraints Breed Creativity

Slide 45

Slide 45

Trust Your Instinct

Slide 46

Slide 46

(REPL is God)

Slide 47

Slide 47

spec Scientific Constraints (s/def ::frequency (s/and number? #((control-range 20 20000) %) ))

Slide 48

Slide 48

spec Aesthetic Constraints (s/def ::vca (s/and number? #((control-range 0.4 1) %))) (s/def ::reverb (s/and integer? #((control-range 0 1000) %))) (s/def ::vco (s/and integer? #((control-range 0 25) %)))

Slide 49

Slide 49

Autonomous Decision Making testable experiment-able with spec

Slide 50

Slide 50

Autonomous Decision Making Designing its own timbre over long periods of uptime

Slide 51

Slide 51

Autonomy & Authorship

Slide 52

Slide 52

Objectivity Subjectivity

Slide 53

Slide 53

Art

Slide 54

Slide 54

Aesthetics and Narrative: Programming What Cannot Can Be Programmed @dschmudde http ://schmud.de Licensed under a Creative Commons license.