This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This chapter sets out definitions and key concepts that are used throughout the rest of the book.
-Analysis is the collection, manipulation and interpretation of information and data for use in decision making. Analysis can vary widely between situations and many different types of analysis may be used to form the evidence base that supports the decision-making process.
-Examples of types of analysis that are frequently encountered in government are:
-Analytical assurance is the process and set of practices to ensure that the analysis is fit for purpose.
-Assurance activities are anything that is undertaken to validate and verify analysis
-For example:
-Business critical analysis is defined as analysis which plays such a role in decision making that it influences significant financial and funding decisions, is necessary to the achievement of a Departmental business plan, or an error could have a significant reputational, economic or legal impact for the public sector.
-Version 1 of the AQuA book described business critical models. This has been generalised to business critical analysis, as it is possible for analysis to be business critical without including a model. Some departments may continue to use the term business critical models (BCM).
-Change control is the set of processes followed when changes are made to a piece of analysis. For example, version numbering, documentation, assurance of changes.
-This documentation captures the initial engagement with the commissioner and identifies the question, the context, and any boundaries of the analysis. It provides a definition of the scope and a mechanism for agreeing the project constraints, e.g., deadlines, available resources, and capturing what level of assurance is required by the commissioner.
-This document outlines the design of the analysis, including conceptual models to illustrate the analytical problem, and forms an important tool for ensuring that the analytical assurer has the confidence that the analyst can deliver quality analysis.
-A register of assumptions whether provided by the commissioner or derived by the analysis that have been risk assessed and signed off by an appropriate governance group or stakeholder. Assumption logs should describe each assumption, quantify its impact and reliability and set out when it was made, why it was made, who made it and who signed it off.
-A register of decisions whether provided by the commissioner or derived by the analysis that have been risk assessed and signed off by an appropriate governance group or stakeholder. Decisions logs should describe each decision and set out when it was made, why it was made, who made it and who signed it off.
-A register of data provided by the commissioner or derived by the analysis that have been risked assessed and signed-off by an appropriate governance group or stakeholder.
-A detailed plan of what verification and validation activities should be produced identifying the key risk areas within the analysis. This should be thought about throughout the analytical lifecycle, from initial scoping, designing the analysis, to delivering the analysis. The areas identified in the plan can also form the basis of a log for those analysts conducting the verification and validation checks. Any additional verification and validation checks that have been performed should be recorded on the quality assurance plan.
-All analysis shall have user-documentation, even if the user is only the analyst leading the analysis. This is to ensure that they have captured sufficient material to assist them if the analysis is revisited in due course. For analysis that is likely to be revisited or updated in the future, documentation should be provided to assist a future analyst and should be more comprehensive. This documentation should include a summary of the analysis including the context to the question being asked, what analytical methods were considered, what analysis was planned and why, what challenges were encountered and how they were overcome and what verification and validation steps were performed. In addition, guidance on what should be considered if the analysis is to be revisited or updated is beneficial.
-It is important to ensure that changes that have been made to analysis can be easily seen and quality assured by the analytical assurer, and the latest version of the analysis is being used. Tools and templates can be used to support with evidencing updates and the checks completed throughout a project providing a log of changes that have occurred, why, when, and by whom.
-A brief description of the analytical assurance that have been performed to assure the analysis. The statement should refer to known limitations and conditions associated with the analysis.
-Some models, often complex and large, are used by more than one user or group of users for related but differing purposes, these are known as multi-use models.
-Often, a Steering Group is created to oversee the analysis. This Steering Group would be chaired by the senior officer in charge of the area that maintains the model, and contain senior, ideally empowered, representatives of each major user area.
-No single piece of guidance provides a definitive assessment of whether a piece of analysis is of sufficient quality for an intended purpose. However, the following principles support commissioning and production of fit-for-purpose analysis:
-Proportionate: Quality assurance effort should be appropriate to the risk associated with the intended use of the analysis and the complexity of the analytical approach. These risks include financial, legal, operational and reputational impacts. More details can be found in chapter [3]
-Assurance throughout development: Quality assurance should be considered throughout the life cycle of the analysis and not just at the end. Effective communication is crucial when understanding the problem, designing the analytical approach, conducting the analysis and relaying the outputs. More details on the analysis life cycle can be seen in chapter [5].
-Verification and validation: Analytical quality assurance is more than checking that the analysis is error-free and satisfies its specification (verification). It should also include checks that the analysis is appropriate, i.e. fit for the purpose for which it is being used (validation). Validation and verification are covered in more depth in chapters [5-9].
-Accept that uncertainty is inherent in the inputs and outputs of any piece of analysis. Chapter [8] covers assurance of the analytical phase of the project, including the treatment of uncertainty . Further support can be found in the Uncertainty Toolkit for Analysts in Government (analystsuncertaintytoolkit.github.io)
-Analysis with RIGOUR: One acronym some users find helpful to consider when completing analysis is RIGOUR. This is described in the box below.
-Throughout all the stages of an analytical project, the analyst should ask questions of their own analysis. The helpful mnemonic “RIGOUR” may assist:
-Repeatable: For an analytical process to be considered valid we might reasonably expect that the analysis produces the same outputs for the same inputs and constraints. Different analysts might approach the analytical problem in different ways, while methods might include randomised processes. In such cases, exact matches are not guaranteed or expected. Taking this into account, repeatability means that if an approach is repeated the results should be as expected.
-Independent: Analysis should be free of prejudice or bias. Care should be taken to balance views appropriately across all stakeholders and experts.
-Grounded in reality: Quality analysis takes the Commissioner and Analyst on a journey as views and perceptions are challenged and connections are made between the analysis and its real consequences. Connecting with reality like this guards against failing to properly grasp the context of the problem that is being analysed.
-Objective: Effective engagement and suitable challenge reduce the risk of bias and enables the Commissioner and the Analyst to be clear about the interpretation of results.
-Uncertainty-managed: Uncertainty is identified, managed and communicated throughout the analytical process.
-Robust: Analytical results are error free in the context of residual uncertainty and accepted limitations that make sure the analysis is used appropriately.
-Quality analysis is analysis which is fit for the purpose(s) it was commissioned to meet. It should be accurate, have undergone appropriate assurance, be evidenced, proportionate to its impact, adequately communicated, documented and accepted by its commissioners.
-Reproducible Analytical Pipelines (RAPs) are automated analytical processes. They incorporate elements of software engineering best practice to ensure that the pipelines are reproducible, auditable, efficient, and high quality.
-Organisations may have their own titles for the main functional roles involved in analysis that are* set out here.
-Each role may be fulfilled by a team or committee of people. However, a single individual will have overall accountability (such as the chair of a committee) for each role.
-The AQuA book defines the roles as:
-The outcome of a decision is never known perfectly in advance. For each option within analysis, a range of real outcomes is possible: the outcome is uncertain.
-Wikipedia defines uncertainty as referring to epistemic situations involving imperfect or unknown information. It applies to predictions of future events, to physical measurements that are already made, or to the unknown.
-There are different types of uncertainty. A common classification divides uncertainty into known knowns, known unknowns, and unknown unknowns. The type of uncertainty will impact the analytical approach and assurance activities required.
-The Uncertainty Toolkit for Analysts in Government is a tool produced by a cross government group to help assessing and communicating uncertainty.
- - -This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-This guidance uses the following terms to indicate whether recommendations are mandatory or advisory.
-The terms are:
-These are the same terms as those in the UK Government functional standards.
- - - -This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-The last version of the Analytical Quality Assurance (AQuA) Handbook was published in 2015, following Sir Nicholas Macpherson’s Review of quality assurance of government models. Since then, assurance has become part of the fabric of good practice for developing evidence to support policy development, delivery and operational excellence. The AQuA Book has made a significant contribution to the cultural change in assurance practices in government.
-The AQuA Book is about the process for assuring analytical evidence in all forms. It sets out the core framework for assuring all forms of analytical evidence. It describes what you need to do but not how to do it, aqlthough it does contain many worked examples. Large organisations will have their own processes and practices covering “the how”. For those of you who do not work in places with bespoke guidance you will find a collection of helpful resources in chapter 10.
-The AQuA Book is a key supporting guide for the Analysis Function Standard. It is referred to by the Green Book, Magenta Book and Finance Function Standards.
-The AQuA Book is the work of many authors from across the Government Analysis Function. The original version of the book was compiled by the Quality Assurance Working Group set up after Sir Nicholas Macpherson’s review of modelling in government. This revised edition of the book was produced by a task and finish group drawn from across the Analysis Function community. We would like to thank everybody who has given of their time and expertise to produce this revised edition.
- - -This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-The world of analysis has developed since we published the first edition of the AQuA Book. Increasingly in our data driven world, insights from analysis underpin almost all policies and support operational excellence. This underlines the continuing importance of assuring our evidence. In parallel our working practices have developed. The dominant analytical tools when we wrote the last edition were spreadsheets and proprietary software. Since then we have broadened the range of methods to include open-source software, machine learning and artificial intelligence.
-Users of the AQuA book have pointed out some things we did not cover in the first edition and areas where guidance was unclear or insufficient. In this edition we have added guidance on:
-We provide improved guidance on what a proportionate approach to assurance means (in chapter X) and have made the whole guide applicable to all types of analysis.
-In this edition we have tried to make our guidance relevant to anyone who commissions, uses, undertakes or assures analysis. It is about the whole process of producing analysis that is fit for purpose and not just about the checks after the analysis has been completed. We would like to see analysts from all backgrounds using this book. Our intended audience includes;
-Here are a few reasons why.
-See Knuth (1984) for additional discussion of literate programming.
- - - -Black box: system which can be viewed in terms of its inputs and outputs (or transfer characteristics), without any knowledge of its internal workings.↩︎
Managing Public Money, Annex 4.2 Use of models↩︎
This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-All analysis shall be assured, and the assurance should be proportionate to the potential impact, size and complexity of the analysis. The level of assurance shall be guided by structured assessment of the business risks.
-There shall always be an element of independence between the assurer and the analyst. The degree of separation depends on many factors including the importance of the output, and the size and complexity of the analysis. This does not mean that the analyst should not undertake assurance. It means that there should be some formal independent assurance.
-Think about and deliver appropriate (proportionate) levels of assurance for your analysis. There is a need to be confident in analysis delivered, but there is no point spending months assuring simple analysis that will inform a decision that will make minimal impact.
-Table 1 provides a list of key factors that should be considered when determining what level of assurance is appropriate.
-Further detail and considerations may be found on the Analysis Function’s Quality Questions and Red Flags page.
-| Factor | -Comments | -
|---|---|
| Business criticality | -Different issues will vary their financial, legal, operational, political and reputational impacts. | -
| Relevance of the analysis to the decision making process | -When analysis forms only one component of a broad evidence base, less assurance is required for that specific analysis than if the decision is heavily dependent on the analysis alone. Significant assurance is still likely to be required for the evidence base. | -
| Type and complexity of analysis | -Highly complex analysis requires more effort to assure. The nature of that analysis may also require the engagement of appropriate subject matter experts. | -
| Novelty of approach | -A previously untried method requires more assurance. Confidence will grow as the technique is repeatedly tested. | -
| Precision of the analysis outputs | -Imprecise analysis can require different assurance than precise analysis, e.g. because of inherent limitations of the analytical technique, or lack of data on assumptions. | -
| Amount of resource available for the analysis and assurance | -The value for money of any additional assurance must be balanced alongside the benefits and risk appetite that exists. Approaches that underpin a lot of different things may require greater levels of QA than might be suggested by any individual decision they support. | -
| Longevity of the analysis | -Change control. | -
| Repeat runs for the same analysis | -Control and assurance of data and parameters for each run | -
Figure 1 shows some assurance techniques that might be considered for different levels of analysis complexity and business risk. The key message is the need for more assurance interventions increases with the complexity of, and the business risk associated with analysis.
-
The interventions in Figure 1 must not be viewed in isolation. Some complex and risky analysis that would benefit from an external review will still require the interventions closer to the axes, for example version control and analyst led testing.
-One way to view assurance interventions would be to consider individual interventions in a layer. An individual intervention will reduce risks in a particular area, but will leave many other risks unmitigated. Adding more interventions (layers) will start to increase coverage and reduce overall risk.
-The total elimination of risk will never be achievable, and the balance needs to be found that reduces the overall business risk to an acceptable level. The diagram indicates a few practical assurance techniques. In practice there are many different techniques that need to be considered and implemented as appropriate. Refer to the table in chapter 10.
-Many of these interventions are mentioned elsewhere in the AQUA Book, and are not repeated here.
-To guide what assurance is needed it is necessary to take a structured approach when reviewing business risks. Business risk should be viewed as the combination of the potential impact of analysis errors, and the likelihood of errors occurring. In situations where the potential business impact is high, it is more important that the likelihood of errors is reduced.
-This can be visualised by considering the situation as a risk matrix, illustrated in Figure 2. The impact of the analysis will usually be beyond the control of the analyst to change, so there will be few options to move an assessment down the table. However, there will usually be treatments (or mitigations), involving additional assurance measures, that will allow the assessed business risk to move to the left.
- - -This version of the AQuA book is a preliminary ALPHA draft. It is still in development, and we are still working to ensure that it meets user needs.
-The draft currently has no official status. It is a work in progress and is subject to further revision and reconfiguration (possibly substantial change) before it is finalised.
-${missingFields[0]} field.`,
- message: `The items being returned for this search do not include all the required fields. Please ensure that your index items include the ${missingFields[0]} field or use index-fields in your _quarto.yml file to specify the field names.`,
- };
- } else if (missingFields.length > 1) {
- const missingFieldList = missingFields
- .map((field) => {
- return `${field}`;
- })
- .join(", ");
-
- throw {
- name: `Error: Search index is missing the following fields: ${missingFieldList}.`,
- message: `The items being returned for this search do not include all the required fields. Please ensure that your index items includes the following fields: ${missingFieldList}, or use index-fields in your _quarto.yml file to specify the field names.`,
- };
- }
- }
-}
-
-let lastQuery = null;
-function showCopyLink(query, options) {
- const language = options.language;
- lastQuery = query;
- // Insert share icon
- const inputSuffixEl = window.document.body.querySelector(
- ".aa-Form .aa-InputWrapperSuffix"
- );
-
- if (inputSuffixEl) {
- let copyButtonEl = window.document.body.querySelector(
- ".aa-Form .aa-InputWrapperSuffix .aa-CopyButton"
- );
-
- if (copyButtonEl === null) {
- copyButtonEl = window.document.createElement("button");
- copyButtonEl.setAttribute("class", "aa-CopyButton");
- copyButtonEl.setAttribute("type", "button");
- copyButtonEl.setAttribute("title", language["search-copy-link-title"]);
- copyButtonEl.onmousedown = (e) => {
- e.preventDefault();
- e.stopPropagation();
- };
-
- const linkIcon = "bi-clipboard";
- const checkIcon = "bi-check2";
-
- const shareIconEl = window.document.createElement("i");
- shareIconEl.setAttribute("class", `bi ${linkIcon}`);
- copyButtonEl.appendChild(shareIconEl);
- inputSuffixEl.prepend(copyButtonEl);
-
- const clipboard = new window.ClipboardJS(".aa-CopyButton", {
- text: function (_trigger) {
- const copyUrl = new URL(window.location);
- copyUrl.searchParams.set(kQueryArg, lastQuery);
- copyUrl.searchParams.set(kResultsArg, "1");
- return copyUrl.toString();
- },
- });
- clipboard.on("success", function (e) {
- // Focus the input
-
- // button target
- const button = e.trigger;
- const icon = button.querySelector("i.bi");
-
- // flash "checked"
- icon.classList.add(checkIcon);
- icon.classList.remove(linkIcon);
- setTimeout(function () {
- icon.classList.remove(checkIcon);
- icon.classList.add(linkIcon);
- }, 1000);
- });
- }
-
- // If there is a query, show the link icon
- if (copyButtonEl) {
- if (lastQuery && options["copy-button"]) {
- copyButtonEl.style.display = "flex";
- } else {
- copyButtonEl.style.display = "none";
- }
- }
- }
-}
-
-/* Search Index Handling */
-// create the index
-var fuseIndex = undefined;
-var shownWarning = false;
-
-// fuse index options
-const kFuseIndexOptions = {
- keys: [
- { name: "title", weight: 20 },
- { name: "section", weight: 20 },
- { name: "text", weight: 10 },
- ],
- ignoreLocation: true,
- threshold: 0.1,
-};
-
-async function readSearchData() {
- // Initialize the search index on demand
- if (fuseIndex === undefined) {
- if (window.location.protocol === "file:" && !shownWarning) {
- window.alert(
- "Search requires JavaScript features disabled when running in file://... URLs. In order to use search, please run this document in a web server."
- );
- shownWarning = true;
- return;
- }
- const fuse = new window.Fuse([], kFuseIndexOptions);
-
- // fetch the main search.json
- const response = await fetch(offsetURL("search.json"));
- if (response.status == 200) {
- return response.json().then(function (searchDocs) {
- searchDocs.forEach(function (searchDoc) {
- fuse.add(searchDoc);
- });
- fuseIndex = fuse;
- return fuseIndex;
- });
- } else {
- return Promise.reject(
- new Error(
- "Unexpected status from search index request: " + response.status
- )
- );
- }
- }
-
- return fuseIndex;
-}
-
-function inputElement() {
- return window.document.body.querySelector(".aa-Form .aa-Input");
-}
-
-function focusSearchInput() {
- setTimeout(() => {
- const inputEl = inputElement();
- if (inputEl) {
- inputEl.focus();
- }
- }, 50);
-}
-
-/* Panels */
-const kItemTypeDoc = "document";
-const kItemTypeMore = "document-more";
-const kItemTypeItem = "document-item";
-const kItemTypeError = "error";
-
-function renderItem(
- item,
- createElement,
- state,
- setActiveItemId,
- setContext,
- refresh,
- quartoSearchOptions
-) {
- switch (item.type) {
- case kItemTypeDoc:
- return createDocumentCard(
- createElement,
- "file-richtext",
- item.title,
- item.section,
- item.text,
- item.href,
- item.crumbs,
- quartoSearchOptions
- );
- case kItemTypeMore:
- return createMoreCard(
- createElement,
- item,
- state,
- setActiveItemId,
- setContext,
- refresh
- );
- case kItemTypeItem:
- return createSectionCard(
- createElement,
- item.section,
- item.text,
- item.href
- );
- case kItemTypeError:
- return createErrorCard(createElement, item.title, item.text);
- default:
- return undefined;
- }
-}
-
-function createDocumentCard(
- createElement,
- icon,
- title,
- section,
- text,
- href,
- crumbs,
- quartoSearchOptions
-) {
- const iconEl = createElement("i", {
- class: `bi bi-${icon} search-result-icon`,
- });
- const titleEl = createElement("p", { class: "search-result-title" }, title);
- const titleContents = [iconEl, titleEl];
- const showParent = quartoSearchOptions["show-item-context"];
- if (crumbs && showParent) {
- let crumbsOut = undefined;
- const crumbClz = ["search-result-crumbs"];
- if (showParent === "root") {
- crumbsOut = crumbs.length > 1 ? crumbs[0] : undefined;
- } else if (showParent === "parent") {
- crumbsOut = crumbs.length > 1 ? crumbs[crumbs.length - 2] : undefined;
- } else {
- crumbsOut = crumbs.length > 1 ? crumbs.join(" > ") : undefined;
- crumbClz.push("search-result-crumbs-wrap");
- }
-
- const crumbEl = createElement(
- "p",
- { class: crumbClz.join(" ") },
- crumbsOut
- );
- titleContents.push(crumbEl);
- }
-
- const titleContainerEl = createElement(
- "div",
- { class: "search-result-title-container" },
- titleContents
- );
-
- const textEls = [];
- if (section) {
- const sectionEl = createElement(
- "p",
- { class: "search-result-section" },
- section
- );
- textEls.push(sectionEl);
- }
- const descEl = createElement("p", {
- class: "search-result-text",
- dangerouslySetInnerHTML: {
- __html: text,
- },
- });
- textEls.push(descEl);
-
- const textContainerEl = createElement(
- "div",
- { class: "search-result-text-container" },
- textEls
- );
-
- const containerEl = createElement(
- "div",
- {
- class: "search-result-container",
- },
- [titleContainerEl, textContainerEl]
- );
-
- const linkEl = createElement(
- "a",
- {
- href: offsetURL(href),
- class: "search-result-link",
- },
- containerEl
- );
-
- const classes = ["search-result-doc", "search-item"];
- if (!section) {
- classes.push("document-selectable");
- }
-
- return createElement(
- "div",
- {
- class: classes.join(" "),
- },
- linkEl
- );
-}
-
-function createMoreCard(
- createElement,
- item,
- state,
- setActiveItemId,
- setContext,
- refresh
-) {
- const moreCardEl = createElement(
- "div",
- {
- class: "search-result-more search-item",
- onClick: (e) => {
- // Handle expanding the sections by adding the expanded
- // section to the list of expanded sections
- toggleExpanded(item, state, setContext, setActiveItemId, refresh);
- e.stopPropagation();
- },
- },
- item.title
- );
-
- return moreCardEl;
-}
-
-function toggleExpanded(item, state, setContext, setActiveItemId, refresh) {
- const expanded = state.context.expanded || [];
- if (expanded.includes(item.target)) {
- setContext({
- expanded: expanded.filter((target) => target !== item.target),
- });
- } else {
- setContext({ expanded: [...expanded, item.target] });
- }
-
- refresh();
- setActiveItemId(item.__autocomplete_id);
-}
-
-function createSectionCard(createElement, section, text, href) {
- const sectionEl = createSection(createElement, section, text, href);
- return createElement(
- "div",
- {
- class: "search-result-doc-section search-item",
- },
- sectionEl
- );
-}
-
-function createSection(createElement, title, text, href) {
- const descEl = createElement("p", {
- class: "search-result-text",
- dangerouslySetInnerHTML: {
- __html: text,
- },
- });
-
- const titleEl = createElement("p", { class: "search-result-section" }, title);
- const linkEl = createElement(
- "a",
- {
- href: offsetURL(href),
- class: "search-result-link",
- },
- [titleEl, descEl]
- );
- return linkEl;
-}
-
-function createErrorCard(createElement, title, text) {
- const descEl = createElement("p", {
- class: "search-error-text",
- dangerouslySetInnerHTML: {
- __html: text,
- },
- });
-
- const titleEl = createElement("p", {
- class: "search-error-title",
- dangerouslySetInnerHTML: {
- __html: ` ${title}`,
- },
- });
- const errorEl = createElement("div", { class: "search-error" }, [
- titleEl,
- descEl,
- ]);
- return errorEl;
-}
-
-function positionPanel(pos) {
- const panelEl = window.document.querySelector(
- "#quarto-search-results .aa-Panel"
- );
- const inputEl = window.document.querySelector(
- "#quarto-search .aa-Autocomplete"
- );
-
- if (panelEl && inputEl) {
- panelEl.style.top = `${Math.round(panelEl.offsetTop)}px`;
- if (pos === "start") {
- panelEl.style.left = `${Math.round(inputEl.left)}px`;
- } else {
- panelEl.style.right = `${Math.round(inputEl.offsetRight)}px`;
- }
- }
-}
-
-/* Highlighting */
-// highlighting functions
-function highlightMatch(query, text) {
- if (text) {
- const start = text.toLowerCase().indexOf(query.toLowerCase());
- if (start !== -1) {
- const startMark = "";
- const endMark = "";
-
- const end = start + query.length;
- text =
- text.slice(0, start) +
- startMark +
- text.slice(start, end) +
- endMark +
- text.slice(end);
- const startInfo = clipStart(text, start);
- const endInfo = clipEnd(
- text,
- startInfo.position + startMark.length + endMark.length
- );
- text =
- startInfo.prefix +
- text.slice(startInfo.position, endInfo.position) +
- endInfo.suffix;
-
- return text;
- } else {
- return text;
- }
- } else {
- return text;
- }
-}
-
-function clipStart(text, pos) {
- const clipStart = pos - 50;
- if (clipStart < 0) {
- // This will just return the start of the string
- return {
- position: 0,
- prefix: "",
- };
- } else {
- // We're clipping before the start of the string, walk backwards to the first space.
- const spacePos = findSpace(text, pos, -1);
- return {
- position: spacePos.position,
- prefix: "",
- };
- }
-}
-
-function clipEnd(text, pos) {
- const clipEnd = pos + 200;
- if (clipEnd > text.length) {
- return {
- position: text.length,
- suffix: "",
- };
- } else {
- const spacePos = findSpace(text, clipEnd, 1);
- return {
- position: spacePos.position,
- suffix: spacePos.clipped ? "…" : "",
- };
- }
-}
-
-function findSpace(text, start, step) {
- let stepPos = start;
- while (stepPos > -1 && stepPos < text.length) {
- const char = text[stepPos];
- if (char === " " || char === "," || char === ":") {
- return {
- position: step === 1 ? stepPos : stepPos - step,
- clipped: stepPos > 1 && stepPos < text.length,
- };
- }
- stepPos = stepPos + step;
- }
-
- return {
- position: stepPos - step,
- clipped: false,
- };
-}
-
-// removes highlighting as implemented by the mark tag
-function clearHighlight(searchterm, el) {
- const childNodes = el.childNodes;
- for (let i = childNodes.length - 1; i >= 0; i--) {
- const node = childNodes[i];
- if (node.nodeType === Node.ELEMENT_NODE) {
- if (
- node.tagName === "MARK" &&
- node.innerText.toLowerCase() === searchterm.toLowerCase()
- ) {
- el.replaceChild(document.createTextNode(node.innerText), node);
- } else {
- clearHighlight(searchterm, node);
- }
- }
- }
-}
-
-function escapeRegExp(string) {
- return string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); // $& means the whole matched string
-}
-
-// highlight matches
-function highlight(term, el) {
- const termRegex = new RegExp(term, "ig");
- const childNodes = el.childNodes;
-
- // walk back to front avoid mutating elements in front of us
- for (let i = childNodes.length - 1; i >= 0; i--) {
- const node = childNodes[i];
-
- if (node.nodeType === Node.TEXT_NODE) {
- // Search text nodes for text to highlight
- const text = node.nodeValue;
-
- let startIndex = 0;
- let matchIndex = text.search(termRegex);
- if (matchIndex > -1) {
- const markFragment = document.createDocumentFragment();
- while (matchIndex > -1) {
- const prefix = text.slice(startIndex, matchIndex);
- markFragment.appendChild(document.createTextNode(prefix));
-
- const mark = document.createElement("mark");
- mark.appendChild(
- document.createTextNode(
- text.slice(matchIndex, matchIndex + term.length)
- )
- );
- markFragment.appendChild(mark);
-
- startIndex = matchIndex + term.length;
- matchIndex = text.slice(startIndex).search(new RegExp(term, "ig"));
- if (matchIndex > -1) {
- matchIndex = startIndex + matchIndex;
- }
- }
- if (startIndex < text.length) {
- markFragment.appendChild(
- document.createTextNode(text.slice(startIndex, text.length))
- );
- }
-
- el.replaceChild(markFragment, node);
- }
- } else if (node.nodeType === Node.ELEMENT_NODE) {
- // recurse through elements
- highlight(term, node);
- }
- }
-}
-
-/* Link Handling */
-// get the offset from this page for a given site root relative url
-function offsetURL(url) {
- var offset = getMeta("quarto:offset");
- return offset ? offset + url : url;
-}
-
-// read a meta tag value
-function getMeta(metaName) {
- var metas = window.document.getElementsByTagName("meta");
- for (let i = 0; i < metas.length; i++) {
- if (metas[i].getAttribute("name") === metaName) {
- return metas[i].getAttribute("content");
- }
- }
- return "";
-}
-
-function algoliaSearch(query, limit, algoliaOptions) {
- const { getAlgoliaResults } = window["@algolia/autocomplete-preset-algolia"];
-
- const applicationId = algoliaOptions["application-id"];
- const searchOnlyApiKey = algoliaOptions["search-only-api-key"];
- const indexName = algoliaOptions["index-name"];
- const indexFields = algoliaOptions["index-fields"];
- const searchClient = window.algoliasearch(applicationId, searchOnlyApiKey);
- const searchParams = algoliaOptions["params"];
- const searchAnalytics = !!algoliaOptions["analytics-events"];
-
- return getAlgoliaResults({
- searchClient,
- queries: [
- {
- indexName: indexName,
- query,
- params: {
- hitsPerPage: limit,
- clickAnalytics: searchAnalytics,
- ...searchParams,
- },
- },
- ],
- transformResponse: (response) => {
- if (!indexFields) {
- return response.hits.map((hit) => {
- return hit.map((item) => {
- return {
- ...item,
- text: highlightMatch(query, item.text),
- };
- });
- });
- } else {
- const remappedHits = response.hits.map((hit) => {
- return hit.map((item) => {
- const newItem = { ...item };
- ["href", "section", "title", "text", "crumbs"].forEach(
- (keyName) => {
- const mappedName = indexFields[keyName];
- if (
- mappedName &&
- item[mappedName] !== undefined &&
- mappedName !== keyName
- ) {
- newItem[keyName] = item[mappedName];
- delete newItem[mappedName];
- }
- }
- );
- newItem.text = highlightMatch(query, newItem.text);
- return newItem;
- });
- });
- return remappedHits;
- }
- },
- });
-}
-
-let subSearchTerm = undefined;
-let subSearchFuse = undefined;
-const kFuseMaxWait = 125;
-
-async function fuseSearch(query, fuse, fuseOptions) {
- let index = fuse;
- // Fuse.js using the Bitap algorithm for text matching which runs in
- // O(nm) time (no matter the structure of the text). In our case this
- // means that long search terms mixed with large index gets very slow
- //
- // This injects a subIndex that will be used once the terms get long enough
- // Usually making this subindex is cheap since there will typically be
- // a subset of results matching the existing query
- if (subSearchFuse !== undefined && query.startsWith(subSearchTerm)) {
- // Use the existing subSearchFuse
- index = subSearchFuse;
- } else if (subSearchFuse !== undefined) {
- // The term changed, discard the existing fuse
- subSearchFuse = undefined;
- subSearchTerm = undefined;
- }
-
- // Search using the active fuse
- const then = performance.now();
- const resultsRaw = await index.search(query, fuseOptions);
- const now = performance.now();
-
- const results = resultsRaw.map((result) => {
- const addParam = (url, name, value) => {
- const anchorParts = url.split("#");
- const baseUrl = anchorParts[0];
- const sep = baseUrl.search("\\?") > 0 ? "&" : "?";
- anchorParts[0] = baseUrl + sep + name + "=" + value;
- return anchorParts.join("#");
- };
-
- return {
- title: result.item.title,
- section: result.item.section,
- href: addParam(result.item.href, kQueryArg, query),
- text: highlightMatch(query, result.item.text),
- crumbs: result.item.crumbs,
- };
- });
-
- // If we don't have a subfuse and the query is long enough, go ahead
- // and create a subfuse to use for subsequent queries
- if (now - then > kFuseMaxWait && subSearchFuse === undefined) {
- subSearchTerm = query;
- subSearchFuse = new window.Fuse([], kFuseIndexOptions);
- resultsRaw.forEach((rr) => {
- subSearchFuse.add(rr.item);
- });
- }
- return results;
-}
diff --git a/_book/summary.html b/_book/summary.html
deleted file mode 100644
index 80d149b..0000000
--- a/_book/summary.html
+++ /dev/null
@@ -1,594 +0,0 @@
-
-
-
-
-
-
-
-
-
-In summary, this book has no content whatsoever.
- - - -