CHANGELOG

2018-03-16  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-beta-4:

	FIXED: Some commands entered at the CLI did not respect some of the
	command line options, e.g. "test" did not adopt the "--libpath"
	setting.

	FIXED: Some loader/compiler/parser messages, which are called
	"violations", did not convert properly to JSON in librosie for return to
	the calling program.

2018-03-15  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-beta-3:

	ENHANCED: The sample docker files in extra/docker now includes python 3
	and the official arch distro.

	ENHANCED: There should be NO compile warnings from gcc as long as it is
	gcc version 7 or higher.  Older versions will warn about the fallthrough
	attribute, because they don't understand it.

	FIXED: In rosie.py, a compiled pattern is now a proper python object,
	and it contains an engine reference so that the engine cannot be GC'd
	before its compiled patterns.  (This does not happen in Python 2.7, but
	it does in Python 3.6.)

	FIXED: Rosie is once again easy to install on OS X!  See
	https://github.com/jamiejennings/homebrew-rosie

2018-03-09  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-beta-2:

	CHANGED: The (undocumented) output encoder "none" has been removed.

	NEW: An output encoder called "bool" was added.  It is not useful from
	the command line, but when using librosie's match() function, it will
	return a code 0/1 indicating a match or no match.

	NEW: Python example program extra/examples/generic_sloc.py.

	ENHANCED: When the user specifies an output encoder name that is not
	supported, a proper error is returned.  Previously, the error that was
	returned could be confused with a bug in librosie.

	ENHANCED: `make install` now installs the rosie man page as well.

	ENHANCED: Makefile was tweaked so that it works with the new brew
	(http://brew.sh) formula for install Rosie on OS X.

	FIXED: a bug in pattern net.fqdn_practical that caused it to include a
	leading space in the capture.

2018-02-25  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-beta-1

	ENHANCED: The python module, rosie.py, now works with Python 2.7 and
	Python 3.6.

	CHANGED: The python module, rosie.py, now requires python 'bytes'
	arguments, not python strings.  And it returns 'bytes' values as
	well. Probably the rosie.py API should be enhanced to accept string
	arguments and convert them to bytes, assuming they are UTF-8 encoded.

	CHANGED: The all.things pattern has been modified such that text like
	"C1" will be recognized as an identifier instead of a hex number.

	CHANGED: The num.any pattern now looks for how the number ends, so that
	the first 3 characters of "1.2.3.4" will not match as a float "1.2".
	Note that num.float (and num.decimal) will match that, but num.any will
	not.

	CHANGED: In package net, introduced fqdn_practical to be an alternative
	to fqdn_strict. (And, net.any looks for fqdn_practical now.)  The
	difference is that fqdn_strict will match "1.2" as a partially qualified
	domain name, but fqdn_practical will not.

	CHANGED: The sample go client cannot be used with go1.9.4 due to a bug
	in that go release.  The src/librosie/go/setup.sh script now checks for
	that go version and produces an error.  We recommend go1.10.

	CHANGED: syntax of grammar statements, which now require an 'in'
	clause.  This was done to accommodate future enhancements to the RPL
	language.  The RPL version level is now 1.2.

	NEW: ~/.rosierc is processed if found, unless the --norcfile option is
	given.  And the --rcfile option can be used to load a different
	initialization file.

	NEW: --colors <str> option to define which colors to use when printing
	colorized output.

	ENHANCED: Previously, the REPL did not interpolate strings entered as
	sample data.  Now, escape sequences like \n, \xAA, and the Unicode
	escape sequences can be used in sample data.

	ENHANCED: The "standard prelude" can be imported like any other
	package.

	CHANGED: The config() API now returns an array of JSON objects, where
	each object is a table of configuration settings.  The first is for the
	rosie installation, the second for the engine specifically, and the last,
	if present, is a table of output encoder parameters.

2018-02-03  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-10

	NEW: output encoder 'jsonpp' (json pretty printing)

	NEW: additional sample docker file 'ubuntu-go', which installs golang,
	and then builds and tests the go client of librosie.

	UPDATED: Most of the docs are now up to date.  Please open an issue if
	you find an error.

	FIXED: a bug in error printing at the command line.

2018-01-20  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-9

	NEW: Unicode patterns in rpl/Unicode for each Unicode script, block,
	general category, word|line|sentence|grapheme break, numeric type, and
	(binary) property.
	  * Limitation: Unicode aliases are not yet supported, so you cannot
	    write, e.g. "Category.Uppercase_Letter".  You have to use the
	    official short name, "Category.Lu".
	  * Limitation: The RPL compiler is slow, particularly on long
	    (automatically generated) patterns like the ones in
	    rpl/Unicode/Category.rpl.  Some planned refactoring will result in
	    all the Unicode patterns being pre-compiled, meaning that they can
	    be loaded in binary form from disk when needed.
	  * Future: A user should be able to download any Unicode Character
	    Database from http://unicode.org and run a rosie utility to generate
	    patterns.  Today, we support only Unicode 10.0.0.

	NEW: A working Go client of librosie, in src/librosie/go/rosie/rosie.go

	BREAKING CHANGE: CLI option "--version" to a command "version".

	BREAKING CHANGE: The librosie functions now return a NULL pointer for
	"messages" when there are none, instead of a string representation of an
	empty JSON object.

	BREAKING CHANGE: The librosie APIs for alloc_limit and libpath changed.

	BREAKING CHANGE (in theory): Rosie is now completely independent of the
	operating system's locale system.  If you ever wrote patterns using,
	e.g. [:alpha:] and counted on the meaning to change when you changed
	your OS locale setting, then please OPEN AN ISSUE.  Rosie can certainly
	support this capability (in a portable way), but doing so is not a known
	requirement at this time.

	CHANGED: CLI now prints error messages to stderr, not stdout.

	CHANGED: Packages "list"ed now show the file from which they were loaded.

	CHANGED: The output of the test command now includes the *full* filename.

	CHANGED: Sample docker files now force re-build when the branch has
	changed.  This is a hack, and it relies on (1) the extra/docker/run
	script, and (2) access by that script to the .git directory in the rosie
	build directory.

	CHANGED: The Lua repl, which is used for white-box testing and
	debugging, is now optional instead of always present.  Compile with
	"make LUADEBUG=1" to get the Lua repl, and to use it, invoke rosie with
	"rosie -D".

	CHANGED: The symbols in librosie.[so|dylib] are now hidden, except of
	course for the functions rosie_* that the library exports.

	REMOVED: "make installtest" (until we decide on a good way to implement
	tests of the system installation of rosie).

	CHANGED: the rosie executable into a single statically linked binary,
	instead of a script.

	CHANGED: Now, librosie.a|so|dylib will look for the rosie files
	(lib/*.luac, rpl/*) in one fixed location in the file system, and that
	location is compiled in.  Any location could be specified, but the
	Makefile knows how to generate two versions:
	  * local, which looks for rosie in the root of the build directory
	  * system, which looks for rosie in DESTDIR (defaults to /usr/local)
	This affects anything built with librosie, which now includes the rosie
	CLI.

	CHANGED: librosie is itself now statically linked.  It no longer loads
	any dynamic libraries.  As a result, there are no longer any so|dylib
	files in the lib directory, and the only libraries that are installed
	via "make install" are librosie.a and librosie.so|dylib.

	FIXED: a subtle bug which revealed that, on some linux variants, dlopen
	caches (at least the) dli.fname value.  Discovered this after learning
	that basename/dirname may change their (char *) argument.  This seems
	like a security hole, and we should dig into it.  Fortunately, it no
	longer affects Rosie, since we no longer utilize dli.fname.

	NEW: rpl/builtin/prelude.rpl and the rpl/builtin dir.

	CHANGED: Removed support for 5,6-byte UTF-8 since those are deprecated.

	CHANGED: Removed "halt" (which was undocumented) from standard prelude.

	CHANGED: The multi-threaded test program now writes output files to /tmp.

	FIXED: Tracing now works on built-in definitions.

	FIXED: When the CLI auto-import fails, it now does so silently, because
	the user may have used other means to ensure that packages are loaded,
	such as an import statement on the command line.

2018-01-05  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-8

	NEW: The "fancy" new character set support is ready for users to try
	out.  Expressions are now allowed within square brackets.  The new
	operator & is useful here, since it provides short notation for set
	intersection.  Examples:
	[[A-Z] & [A-F]] is equivalent to [A-F];
	[[ \t] comment] is equivalent to {[ \t] / comment};
	[comment] is a syntax error; it is interpreted as a character list, and
	it has a duplicate character ("m");
	[cmt] is a character list matching "c", "m", and "t";
	[[] cmt] is equivalent to pattern cmt;
	[[] p1 p2] is equivalent to {p1 / p2};
	[] matches nothing (it is a disjunction of zero characters);
	[^] matches everything (it is the complement of []);

	NEW: A couple of scripts that count source lines of code were added in
	the extra/examples directory.

	NEW: Three kinds of new escape sequences are supported.  They are valid
	in literal strings (in double quotes) and in character lists/ranges (in
	square brackets).  They are:
	\xHH  hex escape; HH are hex digits; range \x00-\xFF
	\uHHHH  unicode escape; 4 hex digits; range \u0000-\uFFFF
	\UHHHHHHHH  long unicode escape; 8 hex digits; range \U00000000-\U0010FFFF
	Note that the hex digits may be upper or lower case.

	NEW: Every part of Rosie should now be Unicode-aware, as long as the
	encoding of the input data is UTF-8.

	CHANGED: Unfortunately, this is a change that can BREAK EXISTING CODE.
	Character lists like [abc] and ranges like [a-z] now enforce these two
	rules:
	(1) To include these characters, they MUST be escaped: [ ] ^ -
	For example, [+\-] matches + and -, and [^\^] matches anything but ^.
	(2) In a range, the ends of the range MUST be in order.  So [a-z] is
	legal, but [z-a] is not.

	FIXED: The "make install" command now copies liblua.5.3.so/dylib into
	the destination directory, alongside librosie.so/dylib.

	FIXED: Bug in the repl command ".load" introduced in the previous
	release.

	FIXED: The dot "." is intended to match any unicode character, or,
	failing that, a single non-character byte.  It did not match a
	non-character byte until this fix.

	FIXED: Default compiler is cc (clang) on OS X.  Use
	"make CC=gcc" to force gcc.

2017-12-22  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-7

	CHANGED: The rosie configuration returned by rosie_config() in librosie
	now has a more simple structure.  It is a list of entries, each of which
	contains a name, description, value, and possibly other attributes.

	CHANGED: The sample Go client remains unfinished.  The part that is
	implemented only works because the main goroutine is locked to the OS
	thread.  Without this, the tiny stacks allocated by Go cause problems
	for goroutines that use librosie.

	CHANGED: librosie now provides rosie_loadfile.

	FIXED: REPL engine now gets the libpath set on the command line (if any).

	FIXED: Violation messages returned by librosie now have internal data
	stripped away, leaving only text that is useful to a librosie client.

	CHANGED: librosie now requires pthreads to compile.

	CHANGED: Now building librosie.dylib on OS X because .dylib and .so
	files are treated differently by Python's cffi package (grrrr).

	NEW: There are now 3 sample clients for librosie that are written in C:
	one statically linked, one dynamically linked, and one statically linked
	and multi-threaded.  See src/librosie/C/Makefile ("test" target) for how
	to invoke each one.

	INFO: Some future changes may be coming to how librosie is built.  Today
	it links with lua.5.3.so in the same directory, which can complicate
	'make install'.

	CHANGED: In librosie, the signature of rosie_import changed to provide
	an additional "output arg" that contains the name of the package
	actually imported.

	NEW: The customized version of lpeg that rosie uses has been modified to
	never read past the end of the input string.  It would peek beyond the
	end by one char, which caused no harm when the input was a string
	produced by lua_tolstring, which guarantees a null terminator.  In
	Rosie, the input can be a "pointer and length" struct, where it is
	incorrect to reference the char at the address pointer+length.

	NEW: librosie is now thread-safe.  Example program in
	src/librosie/C/mt.c.  The required stack size for a thread running a
	rosie matching engine is currently 1MB.  Some implementation changes
	could reduce this, possibly down to 512KB.  If you would like to see
	this, please open an issue on Github.

	FIXED: Updated docker files in extra/docker to reflect best practices
	and to run all librosie client tests.

2017-11-25  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-6

	FIXED: Unreported bug where the "rosie list" command did not show the
	correct colors in the table of patterns.

	CHANGED: The names of the platform-specific MAC patterns in net.rpl.
	THIS COULD BREAK EXISTING RPL CODE, which is why it's happening now, in
	an alpha release.

	CHANGED: With implementation of Issue #68, the "text" output encoder was
	renamed "matches" (to be analogous to "subs"), and the "nocolor" option
	was removed because it's now redundant.

	NEW: Makefile now builds librosie.a in addition to librosie.so.

	NEW: Issue #68 implemented.  Color output now prints the entire input
	line, with match segments in color.  When no color is defined for a
	match, the default is a bold version of the default shell color.

	NEW: Started working on a Go client for librosie, but it's not done.


2017-11-14  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-5

	MERGED PR #67 (from IBMer vmorris) containing a fix to a link in the
	docs.

	FIXED: Issue #69 (bug in how find/findall handle tokenized sequences)

	MISC:
	Output encoder "subs" now omits items that have no subs
	CLI failed to print compile errors occasionally

	NEW: librosie client programs now in their own directories within src/librosie
	NEW: 'make test' now takes optional CLIENTS arg, e.g. CLIENTS="c python"
	NEW: Enhancements (incl 'loadfile') to rosie.py
	NEW: Two sample C clients written (work in progress), one for static
	linking with librosie.o, and one for dynamic linking with librosie.so


2017-10-26  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-4

	FIXED: Prior change log entry (in this file) was labeled alpha-2 when it
	should have been alpha-3.

	FIXED: Issue #65 where RPL files containing carriage returns (DOS-style
	line endings) were rejected.

	ENHANCED: A maximum of 99 syntax errors are reported when loading RPL
	code.  Otherwise, it can take a long time for all the error reports to
	be generated when loading a random (non-RPL) file.

	FIXED: Added necessary reporting of an error when RPL code tries to bind
	an imported identifier (e.g. 'word.any' as opposed to 'any').

	NEW: "Dark launch" of enhanced character set expressions, in which
	identifiers and other RPL expressions can appear.  E.g.
	'[ "Hi" [:alpha:] num.int ]' will match the two-character string "Hi", a
	single alpha character, or an integer (from the num package).
	Restriction: there must be at least one bracket subexpression, which in
	the example is '[:alpha:]'.  This feature should be considered
	EXPERIMENTAL.

	NEW: Sample docker files.  (We use these for testing, and thought we
	would share them.)


2017-10-23  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-3

	FIXED: Bug triggered by multiple import statements in a single rpl file,
	where the error message did not print and some imports did not load.

	ENHANCED: To accomodate patterns that contain many thousands of
	alternatives, the maximum number of captures handled by the lpeg vm has
	been increased to 1 million.  It can go higher, but should it?

	NEW: "Dark launch" of a new operator called 'and', bound to the
	ampersand, e.g.  A & B & C === >A >B C which means "the input matches A
	and B and C, and the capture will be done using pattern C".

2017-10-22  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-2

	NEW: Python module (librosie.so, rosie.py)

	NEW: Tests of some basic macros (halt, message, find, findall, and a
	shallow version of the case-insensitive macro called ci

	Note: The halt pattern is implemented, but the abend status of an
	attempted match is not yet available to the user.  I.e. the halt pattern
	is not very useful yet.

	Note: Memory management in librosie has been carefully designed,
	e.g. buffers are reused whenever possible.  BUT testing with a tool like
	valgrind has not been done yet.

	Modified: Backtracking limit for a single pattern is now 1000 (was 400)

	CHANGED: the find macro now looks for a cooked exp and adds boundary
	patterns automatically

	FIXED: bug in REPL (when entering rpl language declaration, e.g. 'rpl 1.0')

	FIXED: issue #63 (typo in README)

2017-09-20  Jamie Jennings  <jjennings@us.ibm.com>

	* 1.0.0-alpha-1

	First release under semantic versioning.