Mike Gerwitz

Activist for User Freedom

diff options
authorMike Gerwitz <gerwitzm@lovullo.com>2016-11-15 12:35:54 -0500
committerMike Gerwitz <gerwitzm@lovullo.com>2016-11-17 23:55:23 -0500
commit76f41e6250eda5869333eab86454e655462c45c1 (patch)
parent94b423db168e8e4df75cdf383d0e51374e7bd2ad (diff)
Symbol format and type documentation
It's important that others understand the system before I keep adding to the mystery. This is at least a good start. It also obviates certain awkward design issues that have evolved over time and need addressing. Note that it also mentions that `keep' is marked for removal---this is the situation that prompted this documentation; changes will be made to work toward its removal to improve the terrible linker performance when given many thousands of symbols defined in over 500 separate packages. * src/symtable.xsl: Add menu for symbols. * src/symtable/symbols.xsl: Added.
2 files changed, 264 insertions, 0 deletions
diff --git a/src/symtable.xsl b/src/symtable.xsl
index 93a9923..1179c4a 100644
--- a/src/symtable.xsl
+++ b/src/symtable.xsl
@@ -98,4 +98,16 @@
+ @menu
+ * Symbol Format:: Anatomy of a symbol table entry
+ * Symbol Types:: Symbols describing various objects
+ @end menu
+ @lowersections
+ @include ../src/symtable/symbols.texi
+ @raisesections
diff --git a/src/symtable/symbols.xsl b/src/symtable/symbols.xsl
new file mode 100644
index 0000000..21bf54b
--- /dev/null
+++ b/src/symtable/symbols.xsl
@@ -0,0 +1,252 @@
+<?xml version="1.0"?>
+ Semantic analysis for symbol generation
+ Copyright (C) 2016 LoVullo Associates, Inc.
+ This file is part of TAME.
+ TAME is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ General Public License for more details.
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see
+ <http://www.gnu.org/licenses/>.
+ Permission is granted to copy, distribute and/or modify this document
+ under the terms of the GNU Free Documentation License, Version 1.3 or
+ any later version published by the Free Software Foundation; with no
+ Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
+ A copy of the license is included in the section entitled ``GNU Free
+ Documentation License''.
+<stylesheet version="2.0"
+ xmlns="http://www.w3.org/1999/XSL/Transform"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema"
+ xmlns:lv="http://www.lovullo.com/rater"
+ xmlns:symtable="http://www.lovullo.com/tame/symtable"
+ xmlns:preproc="http://www.lovullo.com/rater/preproc">
+<import href="symbols.xsl.apply" />
+ @macro sym{}
+ @math{\\sigma}
+ @end macro
+ @macro obj{}
+ @math{\\theta}
+ @end macro
+ @node Symbol Format
+ @section Symbol Format
+ A @dfn{symbol} is nothing more than an abstract representation of an
+ object,
+ represented by a @code{preproc:sym} node in the symbol table
+ (@pxref{Symbol Table}),
+ For some symbol@tie{}@sym{} describing some object@tie{}@obj{}, the
+ following attributes are recognized:
+ @table @code
+ @item name
+ Unique identifier of@tie{}@sym{}. @emph{Required.}
+ @item type
+ Type of object described by@tie{}@sym{}.
+ Multiple symbols of different types may share the same@tie{}@obj{}.
+ @emph{Required.}
+ @item desc
+ Human-readable description of@tie{}@obj{}. @emph{Required.}
+ @item dim
+ Dimensions of@tie{}@obj{} as an integer.
+ Standard dimensions are scalar@tie{}(0),
+ vector@tie{}(1),
+ and matrix@tie{}(2).
+ @emph{Required}
+ @item dtype
+ Reference to some symbol describing the datatype of@tie{}@sym{},
+ if applicable.
+ @item tex
+ TeX code used to render@tie{}@sym{} in a math block.
+ @item parent
+ Reference to a symbol from which@tie{}@sym{} is derived.
+ For example, a @code{cgen} symbol would have the associated
+ @code{class} as its parent.
+ @item extern
+ Whether@tie{}@sym{} is unresolved in the given context.
+ @item missing
+ If extern, a human-readable message describing@tie{}@obj{}.
+ This is useful to display to the user on error when@tie{}@sym{}
+ is the result of generated code
+ (e.g. @ref{Macro Expansion,,template expansion}).
+ @item allow-circular
+ Permit@tie{}@sym{} to be part of a cycle (@pxref{Dependency Graph}).
+ This is desirable for recursive functions, but should otherwise be
+ avoided like the plague.
+ @item keep
+ Always link@tie{}@sym{},
+ even if disjoint from the program's dependency graph.
+ @emph{This is marked for removal.}@footnote{
+ ``Keeps'' cause various tricky and inelegant situations,
+ cause unnecessary coupling of unrelated concerns,
+ and create a performance nightmare for the linker.
+ They were created as a kluge during a near-rewrite of @tame{}
+ (which introduced the symbol table) to get around the fact that
+ certain systems weren't in place to pull in symbols as
+ dependencies for certain operations;
+ it wasn't intended to stick.
+ They will be removed entirely and will not be replaced with
+ anything@mdash{
+ }if a symbol is to be linked, it must be part of the
+ dependency graph.}
+ @end table
+ An attribute not marked as required may be omitted.
+ If an implementation wishes to attach additional metadata to a
+ symbol,
+ it @emph{must} use a different namespace.
+ @node Symbol Types
+ @section Symbol Types
+ The @dfn{symbol type} describes the context in which a symbol may be
+ used,
+ and the attributes it supports.
+ Generally, both the compiler and linker must be aware of a given
+ symbol type:
+ the compiler uses the symbol type to determine the type of code
+ to generate,
+ and the linker uses that information to determine what section
+ the object code should be linked to.
+ There are exceptions,
+ noted in their individual sections.
+ @tame{} aims to reduce the number of symbol types in favor of a
+ generalized system with few primitives@mdash{
+ }before defining a new symbol type,
+ consider whether the existing can be reused in a novel way.
+ Certain symbols will be consolidated in the future,
+ time permitting.@footnote{
+ @code{class} can be a special case of @code{rate}.
+ @code{gen} can acquire the multidimensional capabilities of
+ @code{cgen} and the latter can be removed.
+ @code{param} and @code{lparam} can be merged if the former
+ is defined by a wide "local" scope.
+ @code{const} is a special case of @code{rate} or @code{cgen}.
+ @code{func} can be eliminated if all current @code{rate} are
+ considered to be immediate function applications;
+ this would also allow for mocking inputs for testing and
+ debugging.
+ If it's not yet clear, @tame{} started as a very rigid,
+ narrowly-focused system and began generalizing over time.}
+ @table @code
+ @item rate
+ Calculation.@footnote{
+ The term @dfn{rate} (a verb) is a consequence of @tame{}'s origin
+ as an insurance rating system.
+ This symbol type will be renamed to @code{calc} eventually.}
+ Calculations yield a single scalar value.
+ If a child @code{cgen} exists,
+ the yield of the calculation is equivalent to the sum of all its
+ elements.
+ @item gen
+ Generator.@footnote{
+ The name is regrettable@mdash{
+ }it originated from the concept of a generator function,
+ but never turned out that way.
+ @code{gen} is actually a list comprehension,
+ and will likely be renamed in the future to reflect that.}
+ Generators produce a vector element for each iteration of some
+ calculation.
+ Generators always have a parent @code{rate} symbol.
+ @item class
+ Classification.
+ Classifications are universal or existential quantifiers most often
+ used as predicates.
+ @item cgen
+ Classification generator.@footnote{
+ See footnote for @code{gen}.}
+ Analogous to @code{gen},
+ produces a boolean element for each classification result.
+ Unlike @code{gen}, the dimension of a given @code{cgen} is the
+ largest of all of its predicates.
+ @item param
+ Global parameter.
+ All TAME programs can be viewed as a function operating on a set of
+ global inputs.
+ Their domains are defined by their datatype,
+ represented by the symbol attribute @code{dtype}
+ (@pxref{Symbol Types}).
+ @item lparam
+ Local parameter.
+ In contrast to @code{param}, local parameters are restricted to a
+ defined scope (e.g. function parameters; let expressions).
+ This symbol is not used by the linker.
+ @item const
+ Global constant.
+ Constant values (of any dimension).
+ These values may be inlined at the compiler's discretion.
+ @item tpl
+ Template definition.
+ Templates define expandable text in the form of macros
+ (@pxref{Macro Expansion}).
+ This symbol is not used by the linker.
+ @item type
+ Datatype.
+ A type describes the domain of a symbol.
+ Types do not include dimension information@mdash{
+ }such is determined by the @code{dim} symbol attribute
+ (@pxref{Symbol Format}).
+ @item func
+ Function.
+ @tame{} supports functions as a means of calculation reuse and
+ abstraction.
+ Unlike calculations,
+ functions can recurse (see @code{allow-circular} in
+ @ref{Symbol Format}).
+ Functions are not first-class@mdash{
+ }@tame{} is not a functional language.
+ @item meta
+ Arbitrary metadata about the program.
+ These metadata are key-value and are intended to be compiled
+ statically into the program for static reference without having to
+ actually invoke the program.
+ @end table