big update to documentation

doc/texinfo/sake-doc.texi

@@ -1324,11 +1324,155 @@ The Sakefile in the example above will yield a visualization like this:
 
 @c -----------------------
 @section Using Patterns
-hi
+Sake may have a strong preference for elegance and simplicity, but that
+doesn't mean Sake is lacking advanced features. One such feature is
+'formula patterns'.
+
+Very often in project building, the same procedure has to be performed
+on many similar files. Formula patterns allow you to condense all of the
+targets down to one that gets automatically autoexpanded. If you find
+yourself with targets that have identical formulas except for a file
+that changes from target to target, this might be a perfect application
+for formula patterns.
+
+Concretely, it is a common C programming/compilation idiom to compile all
+C files in a directory into object files (which then get linked together).
+If there are four such .c files (file1.c, file2.c file3.c, and file4.c)
+in a directory that all need to be object-compiled, you certainly
+@emph{can} create four different targets to perform this, but you can
+also leverage formula patterns here.
+
+If the original Sakefile snippet looked like this:
+@example
+compile C file1:
+    help: compiles C file 1
+    dependencies:
+        - "file1.c"
+    formula: >
+        gcc -c -o file1.o file1.c
+    output:
+        - file1.o
+
+compile C file2:
+    help: compiles C file 2
+    ....
+@end example
+
+the new target to replace the four targets above will look like:
+@example
+compile %cfiles:
+    help: compiles file %cfiles into object file
+    dependencies:
+        - "%cfiles.c"
+    formula: >
+        gcc -c -o %cfiles.o %cfiles.c
+    output:
+        - %cfiles
+@end example
+
+This may appear as but one target in the Sakefile, but as far as Sake is
+concerned, these are four different targets (or how ever many files the
+pattern matches--it was just four in our example).
+
+To prove it, @code{sake help} will yield:
+@example
+"compile file1":
+  - compiles file file1 into object file
+
+"compile file2":
+  - compiles file file2 into object file
+...
+@end example
+
+Additionally, since these are separate targets, @code{sake visual} will
+render them separately, and parallel sake will run them in parallel, if
+applicable.
+
+Note that anywhere in the target the string "@code{%cfiles}" is used, the
+substitution is of the base name of the file, and not the file extentions.
+
+For another example, say you have to convert all png images in a directory
+to jpeg files; you have a program called "picconvert" that performes the
+conversion. The Sakefile snippet will look a little like this:
+@example
+convert %picfile:
+    dependencies:
+        - %picfile.png
+    formula: >
+        picconvert --from png --to jpg %picfile.png %picfile.jpg
+    output:
+        - %picfile.jpg
+@end example
+
+When using patterns in your Sakefiles, there are four things to keep in mind
+@enumerate
+@item
+The pattern must appear in the dependency field.
+@item
+A target that has a pattern in the dependency must have the pattern in the
+target name. Because Sake treats every file match as a new target, and
+different targets cannot share a name, the pattern must be in the target name
+to get outexpanded and keep the target names different and specific.
+@item
+A target using a pattern must use the pattern in its formula. 
+@item
+A target using a pattern must have an output, and must use the pattern
+in the output
+@end enumerate
+Don't be worried about this requirements; It is hard to think of a
+legitimate usage of formula patterns that will violate any of the rules above.
 @c -----------------------
 
+@c -----------------------
 @section Using Includes
-hi
+Sake's second advanced feature is called "includes".
+
+Includes are a lot like storing macros in a different file and then
+importing those macro definitions into your Sakefile.
+
+The primary use case for this feature would be to use the same Sakefile
+across different architectures/platforms but be able to substitute
+values for platform dependent variables.
+
+For example, let's say a C project decides to support the two most
+popular open source C compilers, clang and gcc. It would be silly to write
+two different Sakefiles--one that uses gcc for compilation, and one that uses
+clang for compilation--and switch which Sakefile you use dependending on
+the machine being used.
+
+Instead, a program can be run to interrogate a system, find the name of the
+extant/preferred C compiler, and drop that into a yaml file as a Sake
+compatible macro. That very macro can then be included in the Sakefile.
+
+Concretely, let's say a configure script is run and it determines that gcc is
+the preferred C compiler. It can then create a file called @code{config.yaml}
+with the following content:
+@example
+#! C_Compiler=gcc
+@end example
+
+In the Sakefile, we can include a line like this:
+@example
+#< config.yaml
+@end example
+This will make the @code{C_Compiler} macro available for use in the Sakefile.
+(Note that "@code{config.yaml}" could have been named anything.)
+
+If you include a non-existent config file, @code{sake} will throw a fatal
+error. There may be a use case, however, where you would like to include a
+macro definition file, but don't want to strictly @emph{require} it. In this
+case, you can label the include file @code{optional}, as in this line:
+@example
+#< filetoinclude.yaml optional
+@end example
+
+It is also possible to raise a warning when an optional include file is
+missing; the user will see this warning everytime @code{sake} is run until
+the include file becomes existent. This can be done thusly:
+@example
+#< filetoinclude.yaml or warning: include file missing
+@end example
+Everything after the "@code{or}" will be considered the warning message.
 @c -----------------------
 
 @c --------------------------------------------------------------------

functests/test3/Sakefile.yaml

@@ -4,7 +4,7 @@
 ##
 
 #< config.yaml
-###< optional.yaml or this was optional anyways
+######< optional.yaml or this was optional anyways
 #< optional.yaml optional
 
 #! CFLAGS=-w -O2 -I./include

sakelib/acts.py

@@ -169,7 +169,7 @@ def expand_macros(raw_text, macros={}):
     includes = {}
     result = []
     pattern = re.compile("#!\s*(\w+)\s*=\s*(.+$)", re.UNICODE)
-    ipattern = re.compile("#<\s*(\S+)\s*(optional|or\s+(.+))?$")
+    ipattern = re.compile("#<\s*(\S+)\s*(optional|or\s+(.+))?$", re.UNICODE)
     for line in raw_text.split("\n"):
         line = string.Template(line).safe_substitute(macros)
         # note that the line is appended to result before it is checked for macros