improve paragraph breaks

[mengwong]

[mengwong]

We discovered that a lot of the spurious paragraph breaks come from crossing footnote boundaries.

We discovered that the footnotes aren't actually footnotes, even in the original Word doc. They're just manually generated superscripts with manually font-size-10 at the bottom of each page; presumably someone there doesn't know how to use actual footnote functionality in Word, and has been editing files by hand.

We deleted all the font-size-10 using find-and-replace in Word.

Proposed algorithm for deleting spurious line breaks again: run diff; for each diff chunk, construct list of words; beore and after each line break, are the words the same? if so, delete the line break.

[mengwong]

LOL, without improved breaks, we see that diff loses the plot and gets offset by one, like missing a button when buttoning up your shirt

aebf735