Elasticsearch migration / List of missing features

[fxprunayre]

Following #2830 some GN3 features are not yet available. This issue is a list of what is not implemented in GeoNetwork 4.0.0

Features removed

• Admin / Batch process (not editor > Batch editing) / admin.console#/tools/batch are not available and most of the cases can be handled in batch editing tool Admin / Batch editing / Removed unused code. #6400 (Feature removed - Done in 4.2.1)
• CSW / GetDomain / Not implemented: Use search API aggregations

GeoNetwork 3.x features not implemented in version 4.x

API

• Permalink to search page with URL parameters for 3.x will probably not work with 4.x. Check the changes on parameters names and facets and the corresponding permalinks. eg. https://localhost:8080/geonetwork/srv/eng/catalog.search#/search?any=idp_*&facet.q=status%2Fnotobsolete&resultType=details&sortBy=relevance

• ❌ Search RDF (replaced with use CSW with DCAT output format)
• ❌ Search Z39.50 (no known sponsor for this functionality)
• ❌ Search SRU (no known sponsor for this functionality)
• ❌ Search OpenSearch)
• Search Elasticsearch API
• Search CSW 2.0.2 are for now available
• Search / OAI-PMH Migrate OAIPMH server to Elasticsearch / Spring MVC #7583
• ❌ Search / q service is replaced by Elasticsearch endpoint (use elastic search directly)
• API / RSS (can be replaced by https://github.com/geonetwork/geonetwork-microservices/tree/main/modules/services/ogc-api-records)
• API / INSPIRE OpenSearchDescription atom service & InspireAtomUtil API / INSPIRE Atom / Migration to Elasticsearch and Spring #6395 (Added in 4.2.1)
• API / Directory / Collect entries from records see DirectoryUtils.search /registries/actions/entries/collect (Added in 4.0.7 Subtemplate / Improve ISO19115-3 support. #6292)
• API / ReportUploads (this may of been migrated in a later release)
• ❌ XLink / Remove directory entry used in other record (SYSTEM_XLINK_ALLOW_REFERENCED_DELETION) ie. searcherForReferencingMetadata (no known sponsor is interested in this activity)
• ❌ Lucene / LuceneSearcher.getMetadataFromIndexById (Lucene is no longer available, migrated to Elastisearch api)

UI

• Search / Advanced form (some of the filters have been replaced by facet filter)

• Search / Facet as tab (only facet on the left side panel and home page can be fully customized) (Added in 4.2.0 Collection, data products, intermediate metadata ... titellus/core-geonetwork#81)

• Subtemplate search filter (Added in 4.0.3) Directory improvements #5401
• Multilingual subtemplate support

Index

• Search / Multilingual support (Added in 4.2.0)
  • Index / Store all translations (Added for codelist and thesaurus in 4.0.2)
  • Index / Define analyzer depending on languages Indexing / Multilingual support / Add per field Analyzer #5911 (added in 4.0.6)
  • Search / Configuration of search fields to use depending on UI language (added in 4.0.6)
  • Facet / Choose field depending on language Multilingual support #5193 (Added in 4.0.2)
  • Results / Display information depending on UI language (Improved in 4.0.2)
• Indexing / Spatial / Search on bounding polygons (currently only the geometry is indexed but not used in search)
• Multilingual indexing of contacts and links (Search / Multilingual support for contact, links and overview description (indexing and aggregations) #6588)

Admin

• Information about the index - not needed, can be replaced by Kibana admin?
• Schematron rules (See SchematronCriteriaGroupServiceIntegrationTest)
• Search / Statistics and search stat dashboards (not really working in 3.x)
• Reports (Elasticsearch migration / Restore admin reports. #5683 in 4.0.5)

Security

• CAS (Tested in 4.0.2) Security / CAS / Update config to Spring Security version used in 4.x #5277
• ECAS not tested

Harvester

• CSW / Check on resource identifier (not only check record based on its UUID)
• GeoNetwork 4 can not be harvested using GeoNetwork protocol by an older version. It has to use CSW instead. In the future, OGC API records may be the best option. GeoNetwork 4.x harvester #7579

[etorresmoya]

Francois good day, i need some help, i am new in Geonetwork and i want to use facet using a local thesaurus, but i dont find information, sorry i am not developer only spatial user, but i have loking for information, and it is very difficult to find a detail guide . I am very interested in the new facet funcionalities from geonetwork 4.

I hope some help.

Best regards.

Enrique T

[fxprunayre]

i want to use facet using a local thesaurus,

You've examples in https://github.com/geonetwork/core-geonetwork/blob/4.0.x/web-ui/src/main/resources/catalog/js/CatController.js#L294-L324 and you can customize this in admin > settings UI.

If you have difficulties finding the name of the field to use, you can use Kibana to explore the index content or check one document using http://localhost:9200/gn-records/_doc/7c7923b1-c387-49ac-b6c7-391ca187b7fa

[etorresmoya]

Francois good day, I am trying to follow this guide "https://geonetwork-opensource.org/manuals/trunk/eng/users/customizing-application/configuring-faceted-search.html" and also your indications, but for my it is very confuse, sorry (i am not developer), is there an specific guide for this topic?, like step by step?

In many documents talk about search field, for example: https://geonetwork-opensource.org/manuals/trunk/en/customizing-application/configuring-search-fields.html, but the guide indicates some files, but there is nothing (see image).

I hope some help.

Best regards.

Enrique T

[fxprunayre]

Documentation for version 4 is https://geonetwork-opensource.org/manuals/4.0.x/en/customizing-application/configuring-faceted-search.html.

In your case, to add a facet on your thesaurus,

• check the content of your document in the index http://localhost:9200/gn-records/_doc/7c7923b1-c387-49ac-b6c7-391ca187b7fa eg. thesaurus_geonetworkthesaurusexternalthemegemet
• go admin > settings UI > search > facet configuration
• add a facet on your thesaurus

'inspireThemeUri': {
  'terms': {
    'field': 'thesaurus_geonetworkthesaurusexternalthemegemet'
  }
}

[etorresmoya]

Francois good day, I followed your recomendatios about Documentation for version 4 is https://geonetwork-opensource.org/manuals/4.0.x/en/customizing-application/configuring-faceted-search.html., and the results are good.
I want to change the label name in the facets, but if I change that the system doesn't count well the metadates so How can I change those label and that the metadata counter work well?

Best regards, Enrique T

[fxprunayre]

How can I change those label and that the metadata counter work well?

The idea in the long run is to use the translation API to store translations in the database #4823. You can use the API for that http://localhost:8080/geonetwork/doc/api. The other option is to add them directly in https://github.com/geonetwork/core-geonetwork/blob/4.0.x/web-ui/src/main/resources/catalog/locales/en-v4.json

[etorresmoya]

Francois good day, thank a lot, I added the labels in the https://github.com/geonetwork/core-geonetwork/blob/4.0.x/web-ui/src/main/resources/catalog/locales/en-v4.json, and finally everything was successful. Here the results.

[ianwallen]

@fxprunayre, have any of these missing features been fixed in 4.0.3 or 4.0.4? I don't see any comments indicated any new fixes in the last 2 releases.

We are mostly interested in the - "Search / Multilingual support" feature as our metadata is multilingual.

[fxprunayre]

We are mostly interested in the - "Search / Multilingual support" feature as our metadata is multilingual.

Consider using 4.0.2+ for multilingual support. The work done so far was mainly funded by Ifremer https://sextant.ifremer.fr/ also having a mix of french or english only and french / english. Version 4 is in testing at Ifremer currently and we are quite happy with the current search results.

indicates that some of GeoNetwork 3 features are not yet supported but maybe are not really needed. At least, it requires testing search live on multilingual catalog and define strategy(ies) for better indexing/search using Elasticsearch capabilities.

In http://metawal.wallonie.be/ we have been defining custom analyzer for French https://github.com/geonetwork/core-geonetwork/blob/4.0.x/web/src/main/webResources/WEB-INF/data/config/index/records_french.json which provides better results on some particular cases.

We have to check how to combine multiple analyzer, also having a configuration to search only in a specific language or all may be useful on the long run but so far, it is not high priority here.

So 4.0.2 adds indexing and facet support with multilingual capabilities with
#5193

[Nicolas-Contois]

Hi,

We made the same things like @etorresmoya and it's work, thank you @fxprunayre
But we want to add icons like Inspire topics, how we can do this ?

Nicolas C

[fxprunayre]

But we want to add icons like Inspire topics, how we can do this ?

There is no easy config way for this. You've to update the HTML template https://github.com/geonetwork/core-geonetwork/blob/main/web-ui/src/main/resources/catalog/views/default/templates/home.html#L98.

[jodygarnett]

Added to docs as part of #7644

[frittentheke]

May I ask if the list of open tasks is still current after the release 4.4.3?