Migrate to lutaml-model

[andrew2net]

No description provided.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[ronaldtse]

@andrew2net is this in progress?

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[andrew2net]

@ronaldtse yes, this is in progress. I can finish XML serialization in a week. But there is the blocking issue metanorma/metanorma-standoc#952

[ronaldtse]

That is a blocking issue for Metanorma but not Relaton. Please finish this and merge it.

[andrew2net]

That is a blocking issue for Metanorma but not Relaton. Please finish this and merge it.

@ronaldtse lutaml-model doesn't allow any additional siblings element with map_all content https://github.com/lutaml/lutaml-model#mapping-all-content-xml-only
So it does block from parsing eref element. Here is description relaton/relaton-models#56 (comment)
But I can finish without supporting eref.

[ronaldtse]

lutaml-model doesn't allow any additional siblings element with map_all content

Can you please file an issue and link to it? However, can’t you model down a level so that map_all occurs on the “root” and then the “eref” is handled by a separate model?

[andrew2net]

Can you please file an issue and link to it?

@ronaldtse do you mean to file a lutaml-model issue? I can, but I think it's quite tricky to select all nodes (including text nodes) what aren't any of other defined siblings and are placed exact position. Anyway I will file an issue. Let's see what Hassan will say.

However, can’t you model down a level so that map_all occurs on the “root” and then the “eref” is handled by a separate model?

That's I want to do, but it needs to change the grammar.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[andrew2net]

@ronaldtse we have the note option for the to_xml method https://github.com/relaton/relaton-bib#adding-notes
I have no idea how to implement the equivalent with lutaml-model. Do we still need the option?

[ronaldtse]

@ronaldtse we have the note option for the to_xml method https://github.com/relaton/relaton-bib#adding-notes I have no idea how to implement the equivalent with lutaml-model. Do we still need the option?

The proper way is to move this note from to_xml into an attribute. Then you can set it before calling any serialization.

[ronaldtse]

FYI @opoudjis if Metanorma uses this.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[ronaldtse]

Lutaml::Model supports equivalence checks of attributes

• https://github.com/lutaml/lutaml-model#comparison

[ronaldtse]

As above.

[ronaldtse]

@andrew2net for the migrated models, why not convert all the model elements like docidentifier to lutaml-model directly instead of using the model ... keyword to maintain separate serialization and data models? They can be defined in the same model unless there is something very special that is done...

[andrew2net]

@ronaldtse there are some reasons to have separate models

• RelatonXML has 2 variants, bibitem and bidata. With data separated from mappings we can use Bibitem.to_xml(item) or Bibdata.to_xml(item) we can have output what we need.
• Except RelatonXML we require rendering BibXML. So we can use different renderers for same data BibXML.to_xml(item) or Bibitem.to_xml(item)
• We also have asciibib and bibtex formats that lutaml-model doesn't support. I'm certain we shouldn't have all that in one class.
• Many elements have extra functionality. For example, Docidentifier has remove_part, remove_date, all_parts. I think it's better to separate Relaton data model and renderers.
• In some cases, Relaton uses collections with extra functionalities instead of Arrays. For example, TitleCollection and RelationCollection.

[ronaldtse]

Thank you @andrew2net for the insights. These are all valid concerns and use cases that lutaml-model does not handle well currently.

For items 1 and 2, they are about "alternative serializer definitions". I'm hoping to address it by an extension of this task, which is about a proposed Transform class:

• Support model-to-model (lutaml-model to lutaml-model) transformation lutaml/lutaml-model#165

For item 3, it is about additional serialization formats that lutaml-model does not support (asciibib, bibtex) and likely will not support. We need to find a way to allow users to define their own serializer formats in lutaml-model.

For item 4, it is about methods that are only about the "data model" (attribute and type definitions), and are irrelevant to "serializers" (the xml do... etc blocks). Right now, the arrangement of lutaml-model mixing "data model" (attributes) and "serializer" (xml do..) is not ideal. I still think that item 4 can be implemented in lutaml-model models today.

For item 5, it is about handling collections that are more complex than arrays, e.g. they need additional methods and have serialization formats. We have this task:

• Support collections using custom classes lutaml/lutaml-model#215

I will file the other issues in lutaml-model as enhancements. Thank you for the details.