Aesthetics and Narrative: Programming What Cannot Be Programmed

Aesthe cs and Narra ve: Programming What Cannot Be Programmed @dschmudde h p ://schmud.de

Lambda Calculus Lisp Scheme Clojure Haskell

λ REPL

λ Symbolic Expressions

λ Composable g(f(x))

λ Pure

λ Reason About Code Finance (Quan ta ve) Data (Quan ta ve)

Quan ta ve Reasoning Mathema cally Provable  Correctness  Performance Cogita ng  Behavior  Improvements

λ Long History in Abstract Domains Aesthe cs Story

Art

Process Chaos and Surrender Decisiveness and Agency

The Decisive Ar st Know the Rules Before You Can Break Them

The Decisive Programmer Know the Rules Before You Can Write Them

We Don’t Always Know the Rules

AARON Development: 1968 ‐ 21st Century FORTRAN → C→ Lisp (CLOS)

1983

1992

Color Composing a Small Number of Primi ves

Process AARON Consider All Possibili es and Decide

Process Ar st Manifest Intent and Respond

Intelligence? Crea ve?

Autonomy “ I want to make it clear that in trying to model Harold Cohen “ cognitive behavior in a computer program […] I was not a cognitive scientist doing research, I was an artist attempting to give form to what my own cognitive experience had told me.

Autonomy Harold Cohen (1 May 1928 – 27 April 2016) Pain ngs by AARON (1968 ‐ ?)

Zork Development: 1975 ‐ 1982 FORTRAN (Adventure) → MDL (PDP‐10) → ZIL (Z‐Machine running Zork I‐III)

1975 Colossal Cave Adventure Verb‐Noun Commands go west

1977 Zork Preposi ons and Conjunc ons Direct and Indirect Objects fill the bo le with water

ZIL <OBJECT LANTERN (LOC LIVING-ROOM) (SYNONYM LAMP LANTERN LIGHT) (ADJECTIVE BRASS) (DESC “brass lantern”) (FLAGS TAKEBIT LIGHTBIT) (ACTION LANTERN-F) (FDESC “A battery-powered lantern is on the trophy case.”) (LDESC “There is a brass lantern (battery-powered) here.”) (SIZE 15)>

Parsing and Language Small Number of Primi ves

Autonomy Narra ve Guide Natural Language

Making a Movie

Film Script: A Technical Document Clarity and precision for interpretation: Production breakdowns Camera shooting scripts Direction for actors and directors Basis for novelizations

Autonomy script ≠ film code ≠ execu on

Quan ta ve Acts of Crea vity

Every Icon (1997‐present) John F. Simon, Jr.

Super Mario Bros. (1985) Shigeru Miyamoto

Qualita ve Reasoning Unknown Quan es Explora on of the Idea

Borderless Kinect Overtone

Abstract Domain

Transducers → Input personEntered personUpdated personWillLeave

→ Input 0: pid; 1: oid; 2: age; 3: centroid.x; 4: centroid.y; 5: velocity.x; 6: velocity.y; 7: depth; 8: boundingRect.x;

→ Input 9: boundingRect.y; 10: boundingRect.width; 11: boundingRect.height; 12: highest.x 13: highest.y 14: haarRect.x; 15: haarRect.y; 16: haarRect.width;

Output → vca (loudness) reverb (timbre) vco (pitch) attack (timbre) sustain (timbre) release (timbre) gate (loudness) silence (loudness)

A/V Flow → Input: Kinect Vision Δ Internally: Clojure: Operate on Primitives → Output: Overtone Sound

Δ Opera ons osc/person-enter ∈ @person-sound{id-1 “vowel-1” id-2 “vowel-2” …} (osc/person-updated ‘(id age)) ∃ @person-sound{id …} × (vowel (map #(* age %) [amp verb osc])

Complexity (definst drone-ae-sus “I make the ‘ae’ vowel sound at a given frequency. I start/stop with the gate set to 1 or 0.” [freq 100 gate (synth-defaults ::gate) amp (synth-defaults ::vca) verb (synth-defaults ::reverb) kr-mul (synth-defaults ::vco)] (let [kr-mul eq-freq hpf-rlpf q synth-unit (:value kr-mul) [270 2290 3010] [600 8000 0.6] 0.1 (synth-unit-layered freq eq-freq q kr-mul)] (synth-filter-chain synth-unit amp verb gate hpf-rlpf)))

Art Qualita ve Reasoning Know the rules before you can break them

Art Qualita ve Reasoning Know the rules before you can break them Constraints breed creativity

clojure.spec Integrated System For Specifica on and Tes ng

spec Scien fic Constraints (s/def ::frequency (s/and number? #((control-range 20 20000) %) ))

spec Aesthe c Constraints (s/def ::vca (s/and number? #((control-range 0.4 1) %))) (s/def ::reverb (s/and integer? #((control-range 0 1000) %))) (s/def ::vco (s/and integer? #((control-range 0 25) %)))

Autonomy testable experiment‐able with spec

Autonomy Unexpected Paths Unexpected Results Expressive Tool

Autonomy Ar ficial Intelligence (Computers)

Autonomy Augmented Intelligence (Humani es)

Autonomy & Authorship

Aesthe cs and Narra ve: Programming What Cannot Can Be Programmed @dschmudde h p ://schmud.de Licensed under a Creative Commons license.