Commit 94993a97 authored by Radford Neal's avatar Radford Neal

change syntax from "with_gradient" to "with gradient", etc.

parent a6879fa0
......@@ -447,7 +447,7 @@ lookup} for further details.
@findex SET_GRADINDEX
Bits 8-15 in a node in a pairlist of gradient values holds the index
of the variable whose gradient is recorded, within the list in the relevant
@code{with_gradient} or @code{track_gradient} construct, starting with 1.
@code{with gradient} or @code{track gradient} construct, starting with 1.
@findex MISSING
@findex SET_MISSING
......@@ -591,7 +591,7 @@ with tag the symbol and CAR the bound value. Also contains
the logical OR of the @code{symbits} fields of all symbols that
have ever been stored in the environment, allowing quick determination
that some symbols cannot be in the environment. Finally, if the
environment is for the body of a @code{with_gradient} or @code{track_gradient}
environment is for the body of a @code{with gradient} or @code{track gradient}
construct, it contains a vector list of symbols for the variables
whose gradient will be tracked.
......@@ -604,7 +604,7 @@ set to @code{R_NilValue}.
Pointers to the CAR, CDR (usually a @code{LISTSXP} or @code{R_NilValue}) and
TAG (usually a @code{SYMSXP} or @code{R_NilValue}). For LISTSXP nodes used
to record gradient values, the TAG is the environment for the relevant
@code{with_gradient} or @code{trace_gradient} construct.
@code{with gradient} or @code{trace gradient} construct.
@item LANGSXP
A special type of @code{LISTSXP} used for function calls. (The CAR
......
......@@ -137,9 +137,9 @@ extern0 SEXP R_DownSymbol; /* "down" */
extern0 SEXP R_IfSymbol; /* "if" */
extern0 SEXP R_NextSymbol; /* "next" */
extern0 SEXP R_BreakSymbol; /* "break" */
extern0 SEXP R_WithGradientSymbol; /* "with_gradient" */
extern0 SEXP R_TrackGradientSymbol; /* "track_gradient" */
extern0 SEXP R_ComputeGradientSymbol; /* "compute_gradient" */
extern0 SEXP R_WithGradientSymbol; /* "with gradient" */
extern0 SEXP R_TrackGradientSymbol; /* "track gradient" */
extern0 SEXP R_ComputeGradientSymbol; /* "compute gradient" */
extern0 SEXP R_AsSymbol; /* "as" */
extern0 SEXP R_ColonSymbol; /* ":" */
extern0 SEXP R_DoubleColonSymbol; /* "::" */
......
......@@ -5,9 +5,9 @@
\name{Gradient}
\alias{Gradient}
\alias{with_gradient}
\alias{track_gradient}
\alias{compute_gradient}
\alias{with gradient}
\alias{track gradient}
\alias{compute gradient}
\alias{gradient_of}
\alias{no_gradient}
\title{Gradient Computation}
......@@ -25,18 +25,18 @@
}
\usage{
with_gradient (var=v.expr) expr
with_gradient (var) expr
with_gradient (var1=v1.expr,var2=v2.expr) expr
with_gradient (var1,var2) expr
with gradient (var=v.expr) expr
with gradient (var) expr
with gradient (var1=v1.expr,var2=v2.expr) expr
with gradient (var1,var2) expr
track_gradient (var=v.expr) expr
track_gradient (var) expr
track_gradient (var1=v1.expr,var2=v2.expr) expr
track_gradient (var1,var2) expr
track gradient (var=v.expr) expr
track gradient (var) expr
track gradient (var1=v1.expr,var2=v2.expr) expr
track gradient (var1,var2) expr
compute_gradient (var1=v1.expr, var2=v2.expr) expr as (g1.expr, g2.expr)
compute_gradient (var1, var2) expr as (g1.expr, g2.expr)
compute gradient (var1=v1.expr, var2=v2.expr) expr as (g1.expr, g2.expr)
compute gradient (var1, var2) expr as (g1.expr, g2.expr)
gradient_of (e)
no_gradient (e)
......@@ -56,28 +56,24 @@ no_gradient (e)
}
\details{
The \code{with_gradient}, \code{track_gradient}, and \code{compute_gradient}
keywords used in these constructs are reserved words; \code{as} is not
reserved.
The \code{with_gradient} construct evaluates \code{expr} in a new
The \code{with gradient} construct evaluates \code{expr} in a new
environment (with the current environment as its parent) containing
the variable \code{var} (or variables \code{var1}, \code{var2}, etc),
with initial value \code{v.expr} (or initial values \code{v1.expr},
\code{v2.expr}, etc.). The value of \code{expr} is returned as that
of \code{with_gradient}, with a \code{"gradient"} attribute attached
of \code{with gradient}, with a \code{"gradient"} attribute attached
containing the derivative of the value of \code{expr} with respect to
\code{var}, or the derivatives with respect to \code{var1},
\code{var2}, etc.. (Any existing \code{"gradient"} attribute is
discarded.)
During the evaluation of \code{expr} within \code{with_gradient},
During the evaluation of \code{expr} within \code{with gradient},
assignments to local variables (in the new environment) will record
the gradient of the assigned value with respect to \code{var} (or
\code{var1}, \code{var2}, etc.), and with respect to the variables
listed in any enclosing \code{with_gradient} or \code{track_gradient}
listed in any enclosing \code{with gradient} or \code{track gradient}
constructs, so that if the value of the variable is used to compute
the final value for \code{with_gradient}, its gradient can be
the final value for \code{with gradient}, its gradient can be
computed. Note, however, that the gradients of attribute values are
not recorded, nor are gradients recorded when non-local assignments
are made with \code{<<-}.
......@@ -88,30 +84,30 @@ for S3 methods, but not S4 methods.
Within \code{expr}, the gradient of an expression, \code{e}, with
respect the variable or variables of the enclosing
\code{with_gradient} construct can be found with
\code{with gradient} construct can be found with
\code{gradient_of(e)}. Tracking of gradients can be explicitly
suppressed (to save time) with \code{no_gradient(e)}. Gradient
tracking is automatically suppressed when evaluating expressions that
are used as \code{if} or \code{while} conditions, as indexes, or as
expressions iterated over with \code{for}.
The \code{track_gradient} construct is like \code{with_gradient},
The \code{track gradient} construct is like \code{with gradient},
except that the gradient is not attached to the final result. It is
therefore useful only in combination with calls of \code{gradient_of},
or if it is inside another \code{track_gradient} or
\code{with_gradient} construct. When these constructs are nested,
or if it is inside another \code{track gradient} or
\code{with gradient} construct. When these constructs are nested,
derivatives with respect to an inner variable may determine
derivatives with respect to an outer variable, by application of the
chain rule.
In forms of \code{with_gradient} and \code{track_gradient} in which no
In forms of \code{with gradient} and \code{track gradient} in which no
expression follows a variable, the expression is assumed to be the
same as the variable (evaluated in the current (not the new)
environment).
User-defined functions can specify the gradient using the
\code{compute_gradient} construct. The \code{expr} within
\code{compute_gradient} is evaluated in a new environment in which one
\code{compute gradient} construct. The \code{expr} within
\code{compute gradient} is evaluated in a new environment in which one
or more variables have been defined (two in the above templates). The
initial values of these variables are as specified, defaulting to the
variable's value evaluated in the current environment. Gradients with
......@@ -120,7 +116,7 @@ instead specified by the expressions after \code{as}. The chain rule
is used to translate these gradients to gradients with respect to
variables used to compute \code{v1.expr}, \code{v2.expr}, etc.
If computation of a gradient has not been requested, \code{compute_gradient}
If computation of a gradient has not been requested, \code{compute gradient}
will evaluate only the value, skipping evaluation of the expressions after
\code{as}. Computation of gradients for built-in functions is also
skipped when it is known that the gradient will not be needed.
......@@ -165,28 +161,28 @@ rweibull
\value{
The gradient returned by \code{gradient_of} or attached as a \code{"gradient"}
attribute by \code{with_gradient} will be a scalar real if the gradient
is with respect to only one variable. When the \code{with_gradient} or
\code{track_gradient} construct has more than one variable, the gradient
attribute by \code{with gradient} will be a scalar real if the gradient
is with respect to only one variable. When the \code{with gradient} or
\code{track gradient} construct has more than one variable, the gradient
will be a list of scalar real values, with names corresponding to the
variables.
}
\examples{
a <- with_gradient (x=3) sin(x)
a <- with gradient (x=3) sin(x)
attr(a,"gradient") # should be cos(3)
a <- with_gradient (x=3,y=2) sin(x+2*y)
a <- with gradient (x=3,y=2) sin(x+2*y)
attr(a,"gradient")$x # should be cos(7)
attr(a,"gradient")$y # should be 2*cos(7)
x <- 3
a <- with_gradient (x) { r <- sin(x); r^2 }
a <- with gradient (x) { r <- sin(x); r^2 }
attr(a,"gradient") # should be 2*sin(3)*cos(3)
sqr <- function (y) y^2 # gradients can be tracked through sqr
x <- 3
a <- with_gradient (x) { r <- sin(x); sqr(r) }
a <- with gradient (x) { r <- sin(x); sqr(r) }
attr(a,"gradient") # should be 2*sin(3)*cos(3)
funny <- function (x,y) { # has a discontinuity
......@@ -197,23 +193,23 @@ funny <- function (x,y) { # has a discontinuity
cos(x+y)
}
track_gradient (a = 3) {
track gradient (a = 3) {
print (gradient_of(funny(a,a))) # prints 2*cos(3+3)
print (gradient_of(funny(a,8*a))) # prints -9*sin(3+24)
}
sigmoid <- function (x)
compute_gradient (x) { v <- 1 / (1+exp(-x)); v }
compute gradient (x) { v <- 1 / (1+exp(-x)); v }
as (v * (1-v))
sigmoid(1) # no gradient computed, only value
with_gradient (x=1) sigmoid(x) # both value and gradient computed
with gradient (x=1) sigmoid(x) # both value and gradient computed
track_gradient (x=1) # should compute the same gradient
track gradient (x=1) # should compute the same gradient
gradient_of (1/(1+exp(-x))) # as above, but perhaps more slowly
# (though maybe not since x is scalar)
set.seed(123); with_gradient (r=5) rexp(1,r)
set.seed(123); with gradient (r=5) rexp(1,r)
set.seed(123); v1<-rexp(1,4.999)
set.seed(123); v2<-rexp(1,5.001)
......
......@@ -540,11 +540,11 @@ attribute_hidden FUNTAB R_FunTab_gradient[] =
{
/* printname c-entry offset eval arity pp-kind precedence rightassoc */
{"with_gradient", do_gradient, 0, 1200, -1, {PP_FUNCALL, PREC_FN, 0}},
{"with gradient", do_gradient, 0, 1200, -1, {PP_FUNCALL, PREC_FN, 0}},
{"track_gradient", do_gradient, 1, 1200, -1, {PP_FUNCALL, PREC_FN, 0}},
{"track gradient", do_gradient, 1, 1200, -1, {PP_FUNCALL, PREC_FN, 0}},
{"compute_gradient", do_compute_grad, 0, 1200, -1, {PP_FUNCALL, PREC_FN, 0}},
{"compute gradient", do_compute_grad, 0, 1200, -1, {PP_FUNCALL, PREC_FN, 0}},
{"gradient_of", do_gradient_of, 0, 1200, 1, {PP_FUNCALL, PREC_FN, 0}},
......
......@@ -414,9 +414,9 @@ static void SymbolShortcuts(void)
R_IfSymbol = install("if");
R_NextSymbol = install("next");
R_BreakSymbol = install("break");
R_WithGradientSymbol = install("with_gradient");
R_TrackGradientSymbol = install("track_gradient");
R_ComputeGradientSymbol = install("compute_gradient");
R_WithGradientSymbol = install("with gradient");
R_TrackGradientSymbol = install("track gradient");
R_ComputeGradientSymbol = install("compute gradient");
R_AsSymbol = install("as");
R_ColonSymbol = install(":");
R_DoubleColonSymbol = install("::");
......@@ -538,7 +538,7 @@ static char *Spec_name[] = {
"$", "[", "[[", "@",
"$<-", "[<-", "[[<-", "@<-",
".C", ".Fortran", ".Call", ".External", ".Internal",
"with_gradient", "track_gradient", "compute_gradient",
"with gradient", "track gradient", "compute gradient",
0
};
......
......@@ -319,7 +319,8 @@ enum token_type {
};
/* Names for tokens with codes >= 256. These must correspond in order
with the codes for token types above. They are used for error messages. */
with the codes for token types above. They are used for error messages,
and text for "with gradient", etc. in parse data. */
static const char *const token_name[] = {
"input", "end of input", "input", "string constant","numeric constant",
......@@ -329,8 +330,8 @@ static const char *const token_name[] = {
"'>'", "'>='", "'<'", "'<='", "'=='",
"'!='", "'&'", "'|'", "'&&'", "'||'",
"'::'", "':::'", "'**'", "SPECIAL", "':='",
"'..'", "'!!'", "'with_gradient'", "'track_gradient'",
"'compute_gradient'"
"'..'", "'!!'", "'with gradient'", "'track gradient'",
"'compute gradient'"
};
#define NUM_TRANSLATED 7 /* Number above (at front) that are translated */
......@@ -575,12 +576,20 @@ static void set_parent_in_rec (SEXP rec, SEXP parent_rec)
used to put some higher-level syntactic context into the token
itself. */
static void set_token_in_rec (SEXP rec, char *token)
static void set_token_in_rec (SEXP rec, const char *token)
{
if (rec != R_NilValue)
SET_VECTOR_ELT (rec, PDATA_REC_TOKEN, mkChar(token));
}
/* Change the text string in a record. Needed for "with gradient", etc. */
static void set_text_in_rec (SEXP rec, const char *text)
{
if (rec != R_NilValue)
SET_VECTOR_ELT (rec, PDATA_REC_TEXT, mkChar(text));
}
/* Set comments outside any expression (and not previously handled) to
have MINUS the id of the current top-level expression as their parent. */
......@@ -1525,14 +1534,89 @@ static SEXP parse_expr (int prec, int flags, int *paren)
res = PROTECT_N (LANG2 (op, res));
}
/* Symbols, string constants, and namespace references built from
one or the other or both of these. */
/* Symbols, string constants, namespace references built from
one or the other or both of these, and gradient constructs. */
else if (NEXT_TOKEN == SYMBOL || NEXT_TOKEN == STR_CONST) {
SEXP op, sym;
res = TOKEN_VALUE();
get_next_token(1);
if (!NL_END && (NEXT_TOKEN == NS_GET || NEXT_TOKEN == NS_GET_INT)) {
get_next_token(1); /* won't create parse data record for 'gradient' */
/* Gradient constructs. */
if (!NL_END && NEXT_TOKEN == SYMBOL && TYPEOF(res) == SYMSXP
&& ps->next_token_val == R_GradientSymbol) {
if (strcmp(CHAR(PRINTNAME(res)),"with") == 0
|| strcmp(CHAR(PRINTNAME(res)),"track") == 0
|| strcmp(CHAR(PRINTNAME(res)),"compute") == 0) {
char opname[20];
strcpy(opname,CHAR(PRINTNAME(res)));
strcat(opname," gradient");
SEXP op = install(opname);
enum token_type tk =
strcmp(CHAR(PRINTNAME(res)),"with") == 0 ? WITHGRAD :
strcmp(CHAR(PRINTNAME(res)),"track") == 0 ? TRACKGRAD :
COMPUTEGRAD;
set_token_in_rec (prev_token_rec(1), pdata_token_name[tk-256]);
set_text_in_rec (prev_token_rec(1), token_name[tk-256]);
SEXP var, val, body, grad, last;
get_next_token(0);
PROTECT_N (res = last = LCONS (op, R_NilValue));
EXPECT('(');
for (;;) {
if (NEXT_TOKEN != SYMBOL)
PARSE_UNEXPECTED();
set_token_in_rec (prev_token_rec(1), "SYMBOL_FORMALS");
var = val = TOKEN_VALUE();
get_next_token(0);
if (NEXT_TOKEN == EQ_ASSIGN) {
set_token_in_rec (prev_token_rec(1), "EQ_FORMALS");
get_next_token(0);
PARSE_SUB(val = parse_expr
(EQASSIGN_PREC, subflags, NULL));
}
SETCDR (last, cons_with_tag (val, R_NilValue, var));
last = CDR(last);
if (NEXT_TOKEN != ',') break;
get_next_token(0);
}
EXPECT(')');
PARSE_SUB(body = parse_expr (0, flags, NULL));
SETCDR (last, CONS (body, R_NilValue));
if (strcmp(opname,"compute gradient")==0) {
last = CDR(last);
if (NEXT_TOKEN != SYMBOL
|| ps->next_token_val != R_AsSymbol)
PARSE_UNEXPECTED();
get_next_token(0);
EXPECT('(');
for (;;) {
PARSE_SUB(grad = parse_expr
(EQASSIGN_PREC, subflags, NULL));
SETCDR (last, CONS (grad, R_NilValue));
last = CDR(last);
if (NEXT_TOKEN != ',') break;
get_next_token(0);
}
EXPECT(')');
}
}
}
/* Namespace references. */
else if (!NL_END && (NEXT_TOKEN==NS_GET || NEXT_TOKEN==NS_GET_INT)) {
op = TOKEN_VALUE();
set_token_in_rec (prev_token_rec(2), "SYMBOL_PACKAGE");
get_next_token(0);
......@@ -1766,59 +1850,6 @@ static SEXP parse_expr (int prec, int flags, int *paren)
get_next_token(0);
}
/* Gradient constructs. */
else if (NEXT_TOKEN == WITHGRAD || NEXT_TOKEN == TRACKGRAD
|| NEXT_TOKEN == COMPUTEGRAD) {
int compute_gradient = ps->next_token == COMPUTEGRAD;
SEXP op, var, val, body, grad, last;
op = TOKEN_VALUE();
get_next_token(0);
PROTECT_N (res = last = LCONS (op, R_NilValue));
EXPECT('(');
for (;;) {
if (NEXT_TOKEN != SYMBOL)
PARSE_UNEXPECTED();
set_token_in_rec (prev_token_rec(1), "SYMBOL_FORMALS");
var = val = TOKEN_VALUE();
get_next_token(0);
if (NEXT_TOKEN == EQ_ASSIGN) {
set_token_in_rec (prev_token_rec(1), "EQ_FORMALS");
get_next_token(0);
PARSE_SUB(val = parse_expr (EQASSIGN_PREC, subflags, NULL));
}
SETCDR (last, cons_with_tag (val, R_NilValue, var));
last = CDR(last);
if (NEXT_TOKEN != ',') break;
get_next_token(0);
}
EXPECT(')');
PARSE_SUB(body = parse_expr (0, flags, NULL));
SETCDR (last, CONS (body, R_NilValue));
if (compute_gradient) {
last = CDR(last);
if (NEXT_TOKEN != SYMBOL || ps->next_token_val != R_AsSymbol)
PARSE_UNEXPECTED();
get_next_token(0);
EXPECT('(');
for (;;) {
PARSE_SUB(grad = parse_expr (EQASSIGN_PREC, subflags, NULL));
SETCDR (last, CONS (grad, R_NilValue));
last = CDR(last);
if (NEXT_TOKEN != ',') break;
get_next_token(0);
}
EXPECT(')');
}
}
else
PARSE_UNEXPECTED();
......@@ -2943,9 +2974,6 @@ static struct { char *name; int token; } keywords[] = {
{ "else", ELSE },
{ "next", NEXT },
{ "break", BREAK },
{ "with_gradient", WITHGRAD },
{ "track_gradient", TRACKGRAD },
{ "compute_gradient", COMPUTEGRAD },
{ "..", DOTDOT }, /* delete if don't want .. to be reserved */
{ 0, 0 }
};
......@@ -3023,15 +3051,6 @@ static int KeywordLookup(const char *s)
ps->next_token_val = R_BreakSymbol;
break;
break;
case WITHGRAD:
ps->next_token_val = R_WithGradientSymbol;
break;
case TRACKGRAD:
ps->next_token_val = R_TrackGradientSymbol;
break;
case COMPUTEGRAD:
ps->next_token_val = R_ComputeGradientSymbol;
break;
case IN:
case ELSE:
ps->next_token_val = R_NilValue;
......@@ -3624,8 +3643,10 @@ static int token (int c, int no_sym_un)
If no_sym_un is non-zero, symbols and unary operators are not
expected, which therefore allows the .. and !! operators to be
recognized. (But no error is signaled here if a symbol or unary
operator is seen anyway.)
recognized. No error is signaled here if a symbol or unary
operator is seen anyway. But if a second symbol is "gradient",
no parse data record is created for it - instead a previous "with",
"track", or "compute" record is modified appropriately.
Returns 0 if end of file was immediately encountered, with no
whitespace before, and 1 if not (even when END_OF_INPUT is the
......@@ -3666,11 +3687,13 @@ static int get_next_token(int no_sym_un)
ps->token_loc.last_byte = ps->sr->xxbyteno;
ps->token_loc.last_parsed = ps->sr->xxparseno;
if (ps->next_token != END_OF_INPUT && ps->next_token != '\n') {
if (ps->next_token != END_OF_INPUT && ps->next_token != '\n'
&& (!no_sym_un || ps->next_token_val != R_GradientSymbol)) {
SEXP rec;
char t[4] = { '\'', (char) ps->next_token, '\'', 0 };
rec = start_parseData_record (&ps->token_loc, ps->next_token < 256 ? t
: pdata_token_name[ps->next_token-256],
rec = start_parseData_record (&ps->token_loc,
ps->next_token < 256 ? t
: pdata_token_name[ps->next_token-256],
yytext, TRUE);
end_parseData_record (rec, &ps->token_loc);
}
......
This diff is collapsed.
This diff is collapsed.
......@@ -59,7 +59,7 @@ lang_elements <-
'@', '@<-', '@internal<-',
'[', '[<-', '[[', '[[<-', 'break', 'for', 'function', 'if', 'next',
'repeat', 'return', 'while', '{', '||', '~',
'with_gradient', 'track_gradient', 'compute_gradient')
'with gradient', 'track gradient', 'compute gradient')
known <- c(ls(.GenericArgsEnv, all.names=TRUE),
ls(.ArgsEnv, all.names=TRUE),
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment