Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Add support for elasticsearch and openSearch - docs and namespaces #254

Merged
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ci/files/config/services_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ services:
class: Pimcore\Tests\Support\Helper\DataType\Calculator

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Search\LocateInTreeServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\LocateInTreeService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\LocateInTreeService
public: true

Pimcore\Bundle\GenericDataIndexBundle\Service\Search\SearchService\SearchResultIdListServiceInterface:
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ title: Generic Data Index

# Pimcore Generic Data Index

The Pimcore Generic Data Index Bundle provides a centralized way to index and search assets and data objects in Pimcore via OpenSearch indices.
It is shipped with the OpenSearch client and provides a central configuration for it in order to be used in other bundles.
The Pimcore Generic Data Index Bundle provides a centralized way to index and search elements (assets, data objects and documents) in Pimcore via indices (e.g OpenSearch, Elasticsearch).
It is shipped with the OpenSearch and Elasticsearch clients and provides a central configuration for them in order to be used in other bundles.
This bundle can be extended and customized to fit your specific needs, for example if you would like to extend the search indices with custom attributes.

## Features in a Nutshell
- Based on OpenSearch
- Based on OpenSearch/Elasticsearch
- Centralized data index for multiple bundles (Portal Engine, Studio API/UI, etc.)
- Indexing of all documents, assets and data objects
- Provides search services and models to search, filter and aggregate the data saved in the OpenSearch indices
- Provides search services and models to search, filter and aggregate the data saved in the search indices

## Documentation Overview
- [Installation](./doc/01_Installation/README.md)
Expand Down
5 changes: 4 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,10 @@
"autoload": {
"psr-4": {
"Pimcore\\Bundle\\GenericDataIndexBundle\\": "src/"
}
},
"files": [
"src/autoload.php"
]
},
"autoload-dev": {
"psr-4": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
imports:
- { resource: 'services/search-index-adapter/open-search/modifiers/*' }
- { resource: 'services/search-index-adapter/open-search/pql-field-name-transformers.yaml' }
- { resource: 'services/search-index-adapter/default-search/modifiers/*' }
- { resource: 'services/search-index-adapter/default-search/pql-field-name-transformers.yaml' }

services:
_defaults:
Expand All @@ -9,71 +9,71 @@ services:
public: false

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\SearchIndexServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\OpenSearchService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\DefaultSearchService
arguments:
$client: '@generic-data-index.search-client'

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\IndexAliasServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\IndexAliasService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\IndexAliasService
arguments:
$client: '@generic-data-index.search-client'

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\SearchExecutionServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\SearchExecutionService
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\SearchExecutionServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\SearchExecutionService
arguments:
$client: '@generic-data-index.search-client'

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\BulkOperationServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\BulkOperationService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\BulkOperationService
arguments:
$client: '@generic-data-index.search-client'

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\PathServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\PathService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\PathService
arguments:
$client: '@generic-data-index.search-client'

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Search\Pagination\PaginationInfoServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\Pagination\PaginationInfoService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Pagination\PaginationInfoService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Search\LocateInTreeServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\LocateInTreeService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\LocateInTreeService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Search\FetchIdsServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\FetchIdsService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\FetchIdsService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\FetchIdsBySearchServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\FetchIdsBySearchService
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\FetchIdsBySearchServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\FetchIdsBySearchService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\MappingAnalyzerServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\MappingAnalyzerService
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\MappingAnalyzerServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\MappingAnalyzerService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\IndexStatsServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\IndexStatsService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\IndexStatsService
arguments:
$client: '@generic-data-index.search-client'

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\IndexMappingServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\IndexMappingService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\IndexMappingService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Workspace\QueryServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Workspace\QueryService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Workspace\QueryService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Workspace\ElementWorkspacesQueryServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Workspace\ElementWorkspacesQueryService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Workspace\ElementWorkspacesQueryService

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\QueryLanguage\PqlAdapterInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\PqlAdapter
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\PqlAdapter

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\SubQueriesProcessorInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\SubQueriesProcessor
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\SubQueriesProcessorInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\SubQueriesProcessor


Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameValidator\:
resource: '../../../src/SearchIndexAdapter/OpenSearch/QueryLanguage/FieldNameValidator'
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameValidator\:
resource: '../../../src/SearchIndexAdapter/DefaultSearch/QueryLanguage/FieldNameValidator'
tags: ['pimcore.generic_data_index.pql_field_name_validator']

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DataObject\IndexIconUpdateServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\DataObject\IndexIconUpdateService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\DataObject\IndexIconUpdateService
arguments:
$client: '@generic-data-index.search-client'
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ services:
public: false

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\Search\Modifier\SearchModifierServiceInterface:
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Search\Modifier\SearchModifierService
class: Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\SearchModifierService
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
services:
_defaults:
autowire: true
autoconfigure: true
public: false


Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Aggregation\AssetAggregations: ~
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Aggregation\TreeAggregations: ~

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Filter\BasicFilters: ~
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Filter\AssetFilters: ~
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Filter\TreeFilters: ~
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Filter\DependencyFilters: ~
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Filter\FieldTypeFilters: ~

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\FullTextSearch\FullTextSearchHandlers: ~

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Sort\TreeSortHandlers: ~
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Sort\OrderByFieldsHandlers: ~

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\QueryLanguage\QueryLanguageHandlers: ~

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Search\Modifier\Filter\Workspace\WorkspaceQueryHandler: ~
Original file line number Diff line number Diff line change
Expand Up @@ -4,38 +4,38 @@ services:
autoconfigure: true
public: false

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\AssetMetadataDefaultLanguageTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\AssetMetadataDefaultLanguageTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 9}
- {name: pimcore.generic_data_index.pql_field_name_transformer_sort, priority: 9}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\FieldCategoryTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\FieldCategoryTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 10}
- {name: pimcore.generic_data_index.pql_field_name_transformer_sort, priority: 10}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\IdTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\IdTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 5}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\ImageGalleryTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\ImageGalleryTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 6}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\KeywordTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\KeywordTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 0}
- {name: pimcore.generic_data_index.pql_field_name_transformer_sort, priority: 0}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\RelationsTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\RelationsTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 0}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\SortTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\SortTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer_sort, priority: 1}

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\QueryLanguage\FieldNameTransformer\SynonymTransformer:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\QueryLanguage\FieldNameTransformer\SynonymTransformer:
tags:
- {name: pimcore.generic_data_index.pql_field_name_transformer, priority: 11}
- {name: pimcore.generic_data_index.pql_field_name_transformer_sort, priority: 11}

This file was deleted.

10 changes: 5 additions & 5 deletions config/services/search/asset/field-definition-adapters.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,31 @@ services:
autoconfigure: true
public: false

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Asset\FieldDefinitionAdapter\TextKeywordAdapter:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Asset\FieldDefinitionAdapter\TextKeywordAdapter:
shared: false
tags:
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "input" }
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "textarea" }


Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Asset\FieldDefinitionAdapter\KeywordAdapter:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Asset\FieldDefinitionAdapter\KeywordAdapter:
shared: false
tags:
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "select" }

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Asset\FieldDefinitionAdapter\RelationAdapter:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Asset\FieldDefinitionAdapter\RelationAdapter:
shared: false
tags:
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "asset" }
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "document" }
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "object" }

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Asset\FieldDefinitionAdapter\DateAdapter:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Asset\FieldDefinitionAdapter\DateAdapter:
shared: false
tags:
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "date" }

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Asset\FieldDefinitionAdapter\BooleanAdapter:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Asset\FieldDefinitionAdapter\BooleanAdapter:
shared: false
tags:
- { name: "pimcore.generic_data_index.asset.search_index_field_definition", type: "checkbox" }
2 changes: 1 addition & 1 deletion config/services/search/asset/mapping-provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ services:
autoconfigure: true
public: false

Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\OpenSearch\Asset\PredefinedAssetMetadataProvider:
Pimcore\Bundle\GenericDataIndexBundle\SearchIndexAdapter\DefaultSearch\Asset\PredefinedAssetMetadataProvider:
tags:
- { name: "pimcore.generic_data_index.asset.mapping_provider", type: "predefined" }

Expand Down
Loading
Loading