Here is my revised lisp symbolic deep learning inferencing. It is dual to (is an implementation of) a feedforward neural network of a single hidden layer (Krotov and Hopfield, 2016). As I was just telling my friend 1A on the Mastodon, my use of ansi common lisp condition handling to bind a grab-bag of small and specific functions is non-optional for me. No other language has Kent Pitman’s style of condition system yet, which is an unexplored facet of existing lisp thought leadership.

Instead of a return per se, there should be a local restart after dl-inference is signalled, which should be restarted to with the new value for the inferenced location. (For example).

My algorithm’s flow is done by binding currently available individually useful object oriented functions as handlers for my dl-inference condition. In particular my inference keeps signalling the same condition, which should be picked up by different handlers, until one transfers control out of the algorithm, i.e. as a sink. This is done by my outer restart-case.

References, explorations, expansions, criticisms, conversation welcome.

Example d l inference

CL-USER> (restart-case
    (handler-bind
	((dl-inference #'handle-new-value)
	 (dl-inference #'handle-winner)
	 (dl-inference #'handle-compute-winner))
      (infer 0 0 0 0
	     :item 'foo
	     :input '(((foo bar)))
	     :memories '((((bar))))))
  (write-new-value (new-value)
    (print 'new-value!)
    (print new-value)))

NEW-VALUE! 
(BAR) 
(BAR)
CL-USER>

Implementation

(defun make-rectified-polynomial (degree)
  (lambda (x) (cond ((plusp x) (expt x degree))
		    ((not (plusp x)) '0))))

(defparameter *keys* (list :rectified-polynomial (make-rectified-polynomial 2)))

(defun infer (start-row start-col target-row target-col
	      &key item input memories (dl-keys *keys*))
  "infer expects a signal to dl-condition to resignal and exit via a restart.
start-row: Row of top left corner of memory inside input
start-col: Col of top left corner of memory inside input
target-row: Row of cell inside memory inside input being inferenced
target-col: Col of cell inside memory inside input being inferenced
Keywords:
item: Item, maybe a symbol (e.g. to MEMBER)
input: A list of rows, rows being a list of cells (columns)
memories: A list of data, each resembling and fitting inside the input
dl-keys: i.e. for (apply 'sum-memories params dl-keys) see SUM-MEMORIES"
  (setf (getf dl-keys :start-row) start-row
	(getf dl-keys :start-col) start-col
	(getf dl-keys :target-row) target-row
	(getf dl-keys :target-col) target-col)
  (prog ((condition nil))
   start
     (restart-case
	 (if condition
	     (signal condition)
	     (signal 'dl-inference
		     :item item :input input
		     :memories memories
		     :keys dl-keys))
       (resignal (c) (setq condition c) (go start)))))

My actual symbolic deep learning / ffnn implementation

(defun sum-one-row
    (item row-in mem-in idx
     &key (predicate 'member) predicate-keys &allow-other-keys)
  (loop :for me :in mem-in
	:for ro :in row-in
	:for count :from 0
	:for m := (if (apply predicate item me predicate-keys) +1 -1)
	:for c := (if (apply predicate item ro predicate-keys) +1 -1)
	:when (and idx (= count idx))
	  :sum (* +1 m)
	    :into pluses
	:when (and idx (= count idx))
	  :sum (* -1 m)
	    :into minuses
	:unless (and idx (= count idx))
	  :sum (* c m)
	    :into pluses
	:unless (and idx (= count idx))
	  :sum (* c m)
	    :into minuses
	:finally
	   (return (list pluses minuses))))

(defun sum-one-memory
    (item input memory
     &rest keys
     &key target-row target-col
       start-row start-col
       rectified-polynomial &allow-other-keys)
  (loop :for mem-row :in memory
	:for count :from 0
	:for whole-input-row
	  :in (nthcdr start-row input)
	:for input-row
	  := (nthcdr start-col whole-input-row)
	:for idx
	  := (when (= count target-row) target-col)
	:for (row-plus row-minus)
	  := (apply 'sum-one-row item input-row mem-row idx keys)
	:summing row-plus :into row-pluses
	:summing row-minus :into row-minuses
	:finally
	   (return
	     (- (funcall rectified-polynomial row-pluses)
		(funcall rectified-polynomial row-minuses)))))

(defun sum-memories (item input memories
		     &rest keys &key &allow-other-keys)
  (loop :with old := (elt (elt input
			       (+ (getf keys :start-row)
				  (getf keys :target-row)))
			  (+ (getf keys :start-col)
			     (getf keys :target-col)))
	:for memory :in memories
	:sum (apply 'sum-one-memory
		    item input memory
		    keys)
	  :into best-memory-yes
	:finally (return (if (minusp best-memory-yes)
			     (values NIL old)
			     (values t old)))))

A dl-inference condition

(define-condition
      dl-inference
    ()
  ((item :initarg :item :accessor dl-item)
   (input :initarg :input :accessor dl-input)
   (memories :initarg :memories :accessor dl-memories)
   (keys :initarg :keys :type t :accessor dl-keys)
   (winner :type t :initarg :winner :accessor dl-winner)
   (old-value :type t :initarg :old-value
	      :accessor dl-old-value)
   (new-value :type t :initarg :new-value
	      :accessor dl-new-value)))

Three step `dl-inference` handling

(defun handle-compute-winner (dl-inference)
  (when (slot-boundp dl-inference 'winner)
    (return-from handle-compute-winner))
  (with-slots
	(winner old-value)
      dl-inference
    (multiple-value-setq
	(winner old-value)
      (apply 'sum-memories
	     (dl-item dl-inference)
	     (dl-input dl-inference)
	     (dl-memories dl-inference)
	     (dl-keys dl-inference))))
  (let ((r (find-restart 'resignal dl-inference)))
    (when r (invoke-restart r dl-inference))))

(defun handle-winner (dl-inference)
  (unless (slot-boundp dl-inference 'winner)
    (return-from handle-winner))
  (setf (dl-new-value dl-inference)
	(copy-tree (dl-old-value dl-inference))
	(dl-new-value dl-inference)
	(if (dl-winner dl-inference)
	    (push (dl-item dl-inference)
		  (dl-new-value dl-inference))
	    (delete (dl-item dl-inference)
		    (dl-new-value dl-inference))))
  (let ((r (find-restart 'resignal dl-inference)))
    (when r (invoke-restart r dl-inference))))

(defun handle-new-value (dl-inference)
  (unless (slot-boundp dl-inference 'new-value)
    (return-from handle-new-value))
  (let ((r (find-restart 'write-new-value dl-inference)))
    (when r
      (invoke-restart r (dl-new-value dl-inference)))))

The handlers can be used individually with

(defparameter *c* (make-condition 'dl-inference))
(handle-new-value *c*)

for example.

Further plans

I have several examples coming along soon which shockingly completely explain what deep learning, including what proprietary large language models actually do is, which their vendors present as not knowing.

This slightly revised presentation of the algorithm before is much improved from the last one, for example the handler-bind is correctly outside of the inference function, not inside it, which was inaccessible and confusing, and not really run time handled!

This symbolic FFNN inferencing implementation will be the posterchild of distribution via my Leonardo System via itch.io as well, which is an important upcoming crystallization of my lisp writing.