Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP Model Caching #2096

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

WIP Model Caching #2096

wants to merge 9 commits into from

Conversation

christopherswenson
Copy link
Contributor

@christopherswenson christopherswenson commented Jan 15, 2025

Changes:

  • Changes constructor of Runtime (and related subclasses) to take an object with named things, rather than an arbitrary list of things: new Runtime({ urlReader, connections })
  • Changes Malloy.compile to allow you to pass in a url: URL or source: string directly, rather than having to first call Malloy.parse. Therefore, you only need to use Malloy.parse if you only want to parse. If you always want to compile, you may as well use Malloy.compile.
  • Adds a new thing you can give to a runtime, CacheManager, which maintains the information needed to interact with a ModelCache which is an async map from URL to {modelDef: ModelDef; invalidationKeys: {[url: string]: InvalidationKey}}, where invalidationKeys should be the set of invalidation keys for all the files (including the root file itself) that the cached model depends on.
export interface ModelCache {
  getModel(
    url: URL
  ): Promise<
    | {modelDef: ModelDef; invalidationKeys: {[url: string]: InvalidationKey}}
    | undefined
  >;
  setModel(
    url: URL,
    modelDef: ModelDef,
    invalidationKeys: {[url: string]: InvalidationKey}
  ): Promise<boolean>;
}
  • Modifies URLReader's interface to add knowledge of invalidation keys. It has been modified in such a way that if you don't change your url readers, they'll still work. If you aren't passing invalidation keys, we'll just hash the file contents for you.
export interface URLReader {
  readURL: (
    url: URL
  ) => Promise<string | {contents: string; invalidationKey?: InvalidationKey}>;
  getInvalidationKey?: (url: URL) => Promise<InvalidationKey>;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant