Dyna – Logic Programming for Machine Learning

Summary

About the Dyna Programming Language Dyna is a programming language designed by and for machine learning researchers. Dyna builds on the paradigm of logic programming languages such as Datalog and Prolog. However, Dyna goes much further in that it allows for flexible execution orders and for rules in the program to be "weighted". This means that we can efficiently express complicated programs in a few lines of code without having to worry about how the program will execute and only focus on what we want computed. For example, we can write our matrix multiplication in a single line of code, as well as the fibonacci sequence, CKY parsing, and even an "infinite" neural network in a few lines of code as shown below. % Dyna rule for multiplying the matrix `a` and `b` together c(I,K) += a(I,J) * b(J,K). % The fibonacci sequence fib(N) := fib(N-1) + fib(N-2). % recursive step fib(0) := 0. % override recursive step for 0 and 1 fib(1) := 1. % CKY Parsing phrase(X,I,K) max= phrase(Y,I,J) * phrase(Z,J,K) * rule(X,Y,Z). % binary CKY rule phrase(X,I,K) max= phrase(Y,I,K) * rule(X,Y). % uniary CKY rule phrase(X,I,I+1) max= rule(X, word(I)). % terminal rule % A Neural Network defined in terms of edges sigma(X) = 1/(1+exp(-X)). % define sigmoid function out(J) = sigma(in(J)). % apply sigmoid function in(J) += out(I) * edge(I,J). % vector-matrix product loss += (out(J) - target(J))**2. % L2 loss edge(input(X,Y),hidden(X+DX,Y+DY)) = weight(DX,DY). % define edges in terms of weight matrix weight(DX,DY) := random(-1,1) for DX:-4..4, DY:-4..4. % random initialization for matrix % To see more example Dyna programs, please checkout our research papers. History of the Dyna Project The Dyna project was initially started in 2004 as an umbrella project to make a programming language for Machine Learning (ML) researchers. Most algorithms that ML researchers implement are expressible in a few lines of math. In the process of researching new algorithms, researchers often have to iterate many time...