2019.07.a

Merge branch 'master' into master-legacy

2019.07

Merge branch 'master' into master-legacy

2019.05

Merge branch 'master' into master-legacy

Truncation

[2.2.0] - 2019-05-09

Added

• Truncate URLs:
  • show only domains plus counts to anonymous users
  • show full data to logged-in users
  • allow anonymous users to request full access for limited personal use
• Memory profiling (superadmins only)
• Database indexes to speed up some slow queries
• Allow for elasticsearch index names to be customized via ENV
• Show 'submitted to Lumen' date in search results
• Make risk triggers work
• Add turnout back
• Some dependency updates

Changed

• Asset-pipeline-related gems no longer loaded in prod to save on memory
• Prevent deep pagination on search results, since Elasticsearch doesn't support it
• Much lower throttling limits in rack-attack, coupled with mechanism for whitelisting IPs
• Increased cache time-to-live

Hotfix caching

Overly aggressive caching in the notice advanced search functions causes garbage data to show up. Let's not.

Hotfix ReindexRun

apply_metadata needs to be public, per errors in logs.

2.1.6

2.1.6 - 2018-12-20

Added

• "Submitter" field on notice submissions
• VCR dependency for SearchesModels spec
• Placeholder notice for use in Google Canadian law notice responses
• Auto-redaction of work descriptions for newly added works
• Rake task for redaction of descriptions of existing works (this will be applied slowly over time to redact our existing 203M works)
• Custom message for hidden notices

Changed/Fixed

• Bug preventing notices submitted via OldChill from being imported into Lumen
• Excessive instantiations on SearchesModels#search leading to slow notice searches and topic/notice display pages; the API for #search has been changed to allow its consumers to reduce their data demands
• Excessive db queries on NoticesController#create
• Excessive db queries onRailsAdmin#MainController#edit (resulting in minor changes to admin interface)
• More extensive caching of expensive fragments
• Improved code style
• Dependencies with security vulnerabilities have been upgraded
• Elasticsearch configuration has been streamlined
• Most intermittent test failures have been eliminated
• Bug whereby supporting documents submitted via the notice submission webform were miscategorized as original
• Bug whereby entity types were not being set properly via the notice submission webform
• Bug whereby admins could not delete notices

Removed

• Links to related FAQs and blog entries on notice pages
• DMCA counternotice creator

A 2.1.6 prerelease also included a feature to truncate URLs for unauthorized users, but this has been removed pending further testing.

Hotfix cron job take 2

Direct remaining stderr to log file in hopes that cron will stop emailing admins

Hotfix cron job, submitter field

2.1.5.1 - 2018-09-12

Added

• Submitter fields on government request notice creation web form

Changed/Fixed

• Cache clearing cron job (no longer tries to delete nonempty directories)

2.1.5

2.1.5 - 2018-09-05

Added

• Rake task for indexing added/changed notices after a given date
• Redirect for the frequently accessed /dmca/counter512.pdf to a URL which actually exists
• Rubocop and coverage checking in build process
• PR and issue templates

Changed/Fixed

• Log formats (logs now include timestamps; API keys are logged)
• Opted in to config.active_record.raise_in_transactional_callbacks warnings (the prior behavior of setting these to false was deprecated)
• Updated caching strategy to avoid occasional race condition wherein cache keys existed at time of check but cache contents had been deleted by time of use
• Updated http to https links where possible
• Improved handling of forged POSTs to notices (thereby removing an API call error message)
• Fixed error from trying to iterate over empty tweets
• Handled Sass deprecation warnings that had been clogging the logs
• Fixed bug whereby users with a connected entity could not submit web forms despite auth token
• Minor wording updates
• Added timeout to Elasticsearch client
• Removed excess whitespace from notice URL list (which added up to surprisingly much content for notices with many URLs)
• Updated proxy caching strategy
• Updated references to "chillingeffects" to "lumen" where possible
• Switched to postgres UPSERT for infringing URLs in order to speed up notice creation when there are very large numbers of associated URLs
• Switched to .env rather than bin/init-env.sh for environment variable handling
• Documented the release process
• Improved test isolation, so that the test suite is dramatically more likely to pass
• Linted for consistency with current ruby style guidelines
• Suppressed spurious to_ary warning

Removed

• No longer displaying received date on notices when it is the same as the sent date
• Removed public access to the notice submission form