Meta Burritos

[jag3773]

We need them! More description to come...

[jag3773]

One use case here is to represent our conceptual Book Packages which consist of 5 distinct resources. We want to distribute the resources as standalone resources but we also want to distribute them as a set. For example, we want to distribute a "Jude Book Package" that includes two translations and 3 related translation helps resources (TA, TN, TW).

Here is a potential example using Jude:

{
  "meta": {
    "variant": "source",
    "version": "0.2.0-beta",
    "dateCreated": "2022-09-07T12:00:00.3+05:00",
    "generator": {
      "softwareName": "ByHand",
      "softwareVersion": "0.0.0"
    },
    "comments": [
      "Version 0"
    ],
    "defaultLanguage": "en"
  },
  "idServers": {
    "idServerKey": {
      "id": "https://git.door43.org"
    }
  },
  "identification": {
    "idServer": "dcs",
    "name": {
      "en": "unfoldingWord® Book Package for Jude"
    },
    "abbreviation": {
      "en": "en_bp_JUD"
    },
    "description": {
      "en": "All resources for Jude"
    },
    "systemId": {
      "dcs": {
        "id": "dcs::unfoldingword:en_bp-jud",
        "revision": "v1"
      }
    }
  },
  "confidentiality": {
    "metadata": "unrestricted",
    "ingredients": "unrestricted"
  },
  "type": {
    "flavorType": {
      "name": "peripheral",
      "flavor": {
        "name": "bookPackage"
      },
      "canonType": [
        "ot",
        "nt"
      ],
      "canonSpec": {
        "ot": {
          "name": "western"
        },
        "nt": {
          "name": "western"
        }
      },
      "currentScope": {
        "JUD": [
          "1"
        ]
      }
    }
  },
  "languages": [
    {
      "tag": "en",
      "name": {
        "en": "English"
      }
    }
  ],
  "agencies": [
    {
      "id": "dcs::unfoldingword",
      "url": "https://www.unfoldingword.org",
      "name": {
        "en": "unfoldingWord"
      },
      "roles": [
        "rightsHolder",
        "content",
        "publication",
        "management",
        "finance",
        "qa"
      ],
      "abbr": {
        "en": "uW"
      }
    }
  ],
  "copyright": {
    "licenses": [
      {
        "url": "https://creativecommons.org/licenses/by-sa/2.0/"
      }
    ]
  },
  "ingredients": {
    "LICENSE.md": {
      "mimeType": "text/markdown",
      "checksum": {
        "md5": "ed036c7c3429d7d3254fed7ee06458aa"
      },
      "size": 1523
    },
    "README.md": {
      "mimeType": "text/markdown",
      "checksum": {
        "md5": "4a1a02ab7a76e250027bd055792df303"
      },
      "size": 1677
    },
    "ingredients/LT/66-JUD.usfm": {
      "mimeType": "text/usfm",
      "checksum": {
        "md5": "21ea282e41f02cd5f65353a4358bb439"
      },
      "size": 101008,
      "scope": {
        "JUD": [
          "1"
        ]
      },
      "role": "JUD"
    },
    "ingredients/ST/66-JUD.usfm": {
      "mimeType": "text/usfm",
      "checksum": {
        "md5": "332c51444ccd27afec1ffa265472689b"
      },
      "size": 127193,
      "scope": {
        "JUD": [
          "1"
        ]
      },
      "role": "JUD"
    },
    "ingredients/TWL/twl_JUD.tsv": {
      "mimeType": "text/tsv",
      "checksum": {
        "md5": "a7d82750728db30fdfdeca53301a2e15"
      },
      "size": 9369,
      "scope": {
        "JUD": [
          "1"
        ]
      },
      "role": "lexical"
    },
    "ingredients/TQ/tq_JUD.tsv": {
      "mimeType": "text/tsv",
      "checksum": {
        "md5": "8b80630f8bfe39c0ea4cb41aafc01a60"
      },
      "size": 3492,
      "scope": {
        "JUD": [
          "1"
        ]
      },
      "role": "lexical"
    },
    "ingredients/TN/en_tn_66-JUD.tsv": {
      "mimeType": "text/tsv",
      "checksum": {
        "md5": "99d92ce988695a762753057fc994f0c2"
      },
      "size": 63945,
      "scope": {
        "JUD": [
          "1"
        ]
      },
      "role": "exegetical"
    },
    "ingredients/TA/figs-123person/01.md": {
      "mimeType": "text/markdown",
      "checksum": {
        "md5": "96c2cd54ea1f68b6e67e185fa5f85352"
      },
      "size": 3856,
      "role": "didactic"
    },
    "ingredients/TA/figs-123person/sub-title.md": {
      "mimeType": "text/markdown",
      "checksum": {
        "md5": "6be6bbb510ec11a2fea37638680f3065"
      },
      "size": 125,
      "role": "didactic"
    },
    "ingredients/TA/figs-123person/title.md": {
      "mimeType": "text/markdown",
      "checksum": {
        "md5": "eb4f4006f81c96816efb61d791c3c01b"
      },
      "size": 29,
      "role": "didactic"
    },
    "ingredients/TW/kt/amen.md": {
      "mimeType": "text/markdown",
      "checksum": {
        "md5": "9f1ccea0eaad7ca10ac9abcb683a0608"
      },
      "size": 1882,
      "role": "lexical"
    }
  }
}

[jonathanrobie]

One use case here is to represent our conceptual Book Packages which consist of 5 distinct resources. We want to distribute the resources as standalone resources but we also want to distribute them as a set. For example, we want to distribute a "Jude Book Package" that includes two translations and 3 related translation helps resources (TA, TN, TW).

I prefer a design that would act like a package manager, identifying the burritos that must be present, with no changes to the individual burritos. We could even call it a Burrito Package ... or a Box of Burritos. This is useful, say, for audio+text. This can be distributed in a variety of ways. A zip file could contain other zip files. A directory could contain subdirectories, in a file system or on GitHub. This could be in some database system.

[jonathanrobie]

Minutes from today's discussion:

The SB Working Group wonders if this is in scope, or if this should be a separate standard. We also would like to know who the stakeholders are who intend to exchange this data - we should be working with a stakeholder group, as we do for other flavors. It might also make sense to wait for implementation experience before standardizing.

We're not saying no, but we have a lot of questions.

[jag3773]

@mandolyte do you have any thoughts on this or a different perspective than what I represented above?

[mandolyte]

@jag3773 It seems like a burrito of burritos has the wrong level of granularity, at least for unfoldingWord. Since we manage things by organization, language, and resource type, a burrito that represents that level of granularity makes sense. But a book package is a bit orthogonal to how we manage resources. A book package cuts across and selects a strict subset of each resource type.

As in the example above, it is a cross-cutting set of files in each resource that are needed to complete the book of Jude. So whereas a Translation Academy resource (i.e., the "TA" burrito) might contain 200 documents, the subset needed for Jude might be 20 of them.

A package manager type of solution would require a burrito per document. This would be trivial if we managed individual documents using a traditional document database, such as MongoDB or similar databases. But as mentioned above, we manage at the resource level using a Git-repo-based approach which doesn't inherently lend itself to managing the lifecyle of individual files.

[jonathanrobie]

@mandolyte As you point out, this isn't really a collection of burritos at all, then. You want a burrito flavor that lets you mix whatever resources you want for a given book, regardless of type. I think you would use a custom burrito flavor (x- flavor) for that and document your format. You don't need us to standardize a new flavor to do that.

If we find that different agencies are regularly using this x- flavor to exchange data, then it is a candidate for standardization in a future version of SB.

[jonathanrobie]

Even if this isn't @mandolyte's use case, we do see a need for a way to declare a set of burritos that must be present, similar to a package manager in JavaScript, where a single burrito is analogous to a JavaScript library and the package manager uses a manifest to ensure that they are all present. For instance, in NPM, this is package.json.

Note that this may include multiple burritos of the same flavor, with different metadata, and there may be identical file names and paths in these burritos.