From b6cbbd8653b108e437d4d0310cc437a29068a242 Mon Sep 17 00:00:00 2001 From: Zachary Sutherby Date: Mon, 21 Jun 2021 13:01:01 -0700 Subject: [PATCH 1/3] Updates to clone & update_attachment_keywords notebooks Updates to code of the clone workflow which downloads the cloned survey form item using the correct name and file extension. Then updates the form item with the downloaded zip to ensure online resources exist. Narrative updates to the update_attachment_keywords notebook. --- ...eys_from_one_organization_to_another.ipynb | 274 +++++++++--------- ...chment_keywords_in_arcgis_enterprise.ipynb | 242 ++++++++-------- 2 files changed, 263 insertions(+), 253 deletions(-) diff --git a/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb b/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb index 4ae68c0..3ae1e05 100644 --- a/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb +++ b/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb @@ -2,15 +2,14 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, "source": [ "# Clone surveys from one organization to another\n", "\n" - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "### Introduction\n", "\n", @@ -33,55 +32,63 @@ "Please note that cloning relies on the sharing model to determine the items a user can clone. If a user can access an item, that user can clone it. However, a user can't create any items in the target organization if they don't have the appropriate privileges to create content. For more information, see User types, roles, and privileges for ArcGIS Online or User types, roles, and privileges for ArcGIS Enterprise.\n", "\n", "For more information on the `clone_items` method, see the ArcGIS API for Python Cloning content guide and API reference. " - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "### Prepare to clone\n", "\n", "To start, we are going to need two GIS connections: one to our \"source\" organization, which is the organization in which the survey and content currently resides that we would like to clone; and another to a \"target\" organization, which is the organization that we would like to clone the survey and content to." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 1, - "metadata": {}, + "source": [ + "import arcgis\r\n", + "from arcgis.gis import GIS\r\n", + "import os\r\n", + "\r\n", + "# Connect to source and target organizations\r\n", + "source = GIS(profile=\"sourceOrg\")\r\n", + "target = GIS(profile=\"TargetOrg\")\r\n", + "print(\"Source GIS:\", source,\"\\nTarget GIS:\", target)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Source GIS: GIS @ https://Survey123ninja.maps.arcgis.com version:8.4 \n", "Target GIS: GIS @ https://Survey123.maps.arcgis.com version:8.4\n" ] } ], - "source": [ - "import arcgis\n", - "from arcgis.gis import GIS\n", - "import os\n", - "\n", - "# Connect to source and target organizations\n", - "source = GIS(profile=\"sourceOrg\")\n", - "target = GIS(profile=\"TargetOrg\")\n", - "print(\"Source GIS:\", source,\"\\nTarget GIS:\", target)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "The first example highlights a workflow where you have a few surveys shared to a group that you would like to clone to a different organization. In order to work with your surveys a Survey Manager is defined. A survey in the Survey Manager is a single instance of a survey project that contains the item information and properties and provides access to the underlying survey dataset. In this example, four surveys are shared to a group. Using the group ID, a connection is made to the group and a list is created containing all form items in the group." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "source": [ + "# Get surveys by group ID and then download each format supported\r\n", + "survey_manager = arcgis.apps.survey123.SurveyManager(source)\r\n", + "group = source.groups.get('0dd43178863545fbb5526278e1bdfc4c')\r\n", + "sourceForms = group.search('type:form')['results']\r\n", + "sourceForms" + ], "outputs": [ { + "output_type": "execute_result", "data": { "text/plain": [ "[,\n", @@ -90,29 +97,21 @@ " ]" ] }, - "execution_count": 2, "metadata": {}, - "output_type": "execute_result" + "execution_count": 2 } ], - "source": [ - "# Get surveys by group ID and then download each format supported\n", - "survey_manager = arcgis.apps.survey123.SurveyManager(source)\n", - "group = source.groups.get('0dd43178863545fbb5526278e1bdfc4c')\n", - "sourceForms = group.search('type:form')['results']\n", - "sourceForms" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Now that we have your forms as a list, you are ready to clone the content from the source organization to the target organization. As previously noted, a use case for using the `clone_items()` method is to clone surveys between development, staging, and production organizations. This first example clones the surveys from an existing group (as defined above) located in the source organization to the target organization and shares the cloned surveys to a group with the same name in the target organization." - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Clone related items\n", @@ -124,16 +123,57 @@ "Next, a new folder in the target organization is created based on the survey name, and the items in the list are cloned. Since the `copy_data` parameter is set to \"False\" in the `clone_items()` method, the resulting services created in the target organization will not contain any data. This is ideal when cloning from a development environment to a staging or production environment, as you might not wish to retain any test data.\n", "\n", "Finally, the form items are shared to the group in the target organization that was defined previously, including the feature service, and tags are added to each item." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "source": [ + "# Create new group in the target environment\r\n", + "shared_group = target.groups.create(title=group.title, tags=group.tags)\r\n", + "\r\n", + "# Clone source items to the target environment\r\n", + "for form_item in sourceForms:\r\n", + "\r\n", + " # Obtain the feature service associated with the survey\r\n", + " related_items = form_item.related_items('Survey2Service','forward')\r\n", + " # Obtain the additional items that have a relationship with the survey\r\n", + " additional_items = form_item.related_items('Survey2Data','forward')\r\n", + " all_items = [form_item] + related_items + additional_items\r\n", + " print(\"Source items: \", *all_items, sep=\"\\n\")\r\n", + " \r\n", + " # Create new folder according to survey name \r\n", + " title = form_item.title.replace(\" \", \"_\")\r\n", + " folderName = \"Survey-\" + title\r\n", + " target.content.create_folder(folderName)\r\n", + " \r\n", + " # Clone all items to the new folder in target environment\r\n", + " cloned_items = target.content.clone_items(items=all_items, folder=folderName, copy_data=False)\r\n", + " \r\n", + " # Check the feature count in cloned feature layer. Feature count should 0 because existing data is not copied\r\n", + " print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\r\n", + " \r\n", + " for item in cloned_items:\r\n", + " if item.type == 'Form':\r\n", + " # Update the form item to ensure all resources are rebuilt\r\n", + " downloaded_item = item.download(file_name=item.id+'.zip')\r\n", + " item.update({}, downloaded_item)\r\n", + " os.remove(downloaded_item)\r\n", + " # Share the form item to the group\r\n", + " item.update(item_properties={'tags':'PrdEnv, PythonAPI'})\r\n", + " item.share(groups=shared_group.id)\r\n", + " # Share source feature service to group\r\n", + " source_fs = item.related_items('Survey2Service','forward')[0]\r\n", + " source_fs.update(item_properties={'tags':'PrdEnv, PythonAPI'})\r\n", + " source_fs.share(groups=shared_group.id)\r\n", + " else:\r\n", + " pass" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Source items: \n", "\n", @@ -156,61 +196,29 @@ ] } ], - "source": [ - "# Create new group in the target environment\n", - "shared_group = target.groups.create(title=group.title, tags=group.tags)\n", - "\n", - "# Clone source items to the target environment\n", - "for form_item in sourceForms:\n", - "\n", - " # Obtain the feature service associated with the survey\n", - " related_items = form_item.related_items('Survey2Service','forward')\n", - " # Obtain the additional items that have a relationship with the survey\n", - " additional_items = form_item.related_items('Survey2Data','forward')\n", - " all_items = [form_item] + related_items + additional_items\n", - " print(\"Source items: \", *all_items, sep=\"\\n\")\n", - " \n", - " # Create new folder according to survey name \n", - " title = form_item.title.replace(\" \", \"_\")\n", - " folderName = \"Survey-\" + title\n", - " target.content.create_folder(folderName)\n", - " \n", - " # Clone all items to the new folder in target environment\n", - " cloned_items = target.content.clone_items(items=all_items, folder=folderName, copy_data=False)\n", - " \n", - " # Check the feature count in cloned feature layer. Feature count should 0 because existing data is not copied\n", - " print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\n", - " \n", - " for item in cloned_items:\n", - " if item.type == 'Form':\n", - " # Share the form item to the group\n", - " item.update(item_properties={'tags':'PrdEnv, PythonAPI'})\n", - " item.share(groups=shared_group.id)\n", - " # Share source feature service to group\n", - " source_fs = item.related_items('Survey2Service','forward')[0]\n", - " source_fs.update(item_properties={'tags':'PrdEnv, PythonAPI'})\n", - " source_fs.share(groups=shared_group.id)\n", - " else:\n", - " pass" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "As a confidence check, let's query the newly created group and confirm our four surveys have been shared. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 4, - "metadata": { - "scrolled": true - }, + "source": [ + "# Check items in the new group in target environment.\r\n", + "import time\r\n", + "time.sleep(30)\r\n", + "targetForms = target.content.search('type:form group:'+shared_group.id)\r\n", + "print(\"Cloned surveys: \", *targetForms, sep=\"\\n\")" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Cloned surveys: \n", "\n", @@ -220,26 +228,21 @@ ] } ], - "source": [ - "# Check items in the new group in target environment.\n", - "import time\n", - "time.sleep(30)\n", - "targetForms = target.content.search('type:form group:'+shared_group.id)\n", - "print(\"Cloned surveys: \", *targetForms, sep=\"\\n\")" - ] + "metadata": { + "scrolled": true + } }, { "cell_type": "markdown", - "metadata": {}, "source": [ "The example above is a useful workflow for cloning surveys between development, staging, and production environments and will only clone items that are linked to the form item, including: report templates, linked web maps, CSVs, and map packages. But what if you would like to clone not only the survey and its related items, but also the survey data that's already been collected? Additionally, what if you have web maps, web apps, or dashboards that use the survey data but aren't linked to the survey? \n", "\n", "If those items are stored in the survey's folder, a slightly different method can be used to clone all the content in this folder." - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Clone survey folder\n", @@ -247,16 +250,31 @@ "The code below demonstrates connecting to one specific survey in your organization. Using the properties of the survey, the folder ID where the survey resides is assigned to a variable. Next, all the folders in the source organization for the source username are listed. Using list comprehension, the folder in the `full_folder` variable is matched with the folder ID obtained from the survey properties.\n", "\n", "Once the correct folder has been identified the contents of the folder are listed." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 5, - "metadata": {}, + "source": [ + "source_item_with_data = survey_manager.get(\"65b0ce4cfa2145eb8ce90122e54029e6\")\r\n", + "survey_folder = source_item_with_data.properties['ownerFolder']\r\n", + "\r\n", + "usr = arcgis.gis.User(source, source.users.me.username)\r\n", + "\r\n", + "full_folder = usr.folders\r\n", + "\r\n", + "# Identify the folder associated with the survey\r\n", + "fldr = next((f for f in full_folder if f['id'] == survey_folder), 0)\r\n", + "\r\n", + "#List all the items within that folder to be cloned later on\r\n", + "fldr_items = usr.items(folder=fldr)\r\n", + "print(\"Folder items:\", *fldr_items, sep=\"\\n\")" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Folder items:\n", "\n", @@ -271,37 +289,38 @@ ] } ], - "source": [ - "source_item_with_data = survey_manager.get(\"65b0ce4cfa2145eb8ce90122e54029e6\")\n", - "survey_folder = source_item_with_data.properties['ownerFolder']\n", - "\n", - "usr = arcgis.gis.User(source,'NinjaGreen')\n", - "\n", - "full_folder = usr.folders\n", - "\n", - "# Identify the folder associated with the survey\n", - "fldr = next((f for f in full_folder if f['id'] == survey_folder), 0)\n", - "\n", - "#List all the items within that folder to be cloned later on\n", - "fldr_items = usr.items(folder=fldr)\n", - "print(\"Folder items:\", *fldr_items, sep=\"\\n\")" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Now that all items to be cloned are in a list, a new folder is created in the target organization to store the content. After the folder is created the content is cloned to the target environment. Since the `copy_data` parameter is not defined the default value for the parameter is \"True\", meaning all the underlying data will also be cloned. This means the resulting content in the target organization will be an identical clone of the original data. If you do not wish to retain the source data, setting the `copy_data` parameter to \"False\" will only clone the data schema and architecture to the target organization. The survey, web maps, web apps, dashboards, and other items will be configured as per their original items; the only difference is the feature layer will be empty." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 6, - "metadata": {}, + "source": [ + "# Create a new folder with the same name as the source folder to clone the contents to\r\n", + "target.content.create_folder(folder=fldr['title']+\"_Python\")\r\n", + "\r\n", + "# Clone items to the new folder\r\n", + "cloned_items = target.content.clone_items(items=fldr_items, folder=fldr['title']+\"_Python\")\r\n", + "print(*cloned_items, sep=\"\\n\")\r\n", + "print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\r\n", + "\r\n", + "# Search for the cloned survey and update the form item to ensure all resources are rebuilt\r\n", + "search_clone_survey = target.content.search('title: '+source_item_with_data.properties['title']+' and owner: '+target.users.me.username,'Form')\r\n", + "cloned_survey = search_clone_survey[0]\r\n", + "download_survey = cloned_survey.download(file_name=cloned_survey.id+'.zip')\r\n", + "cloned_survey.update({},download_survey)\r\n", + "os.remove(download_survey)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "\n", "\n", @@ -316,30 +335,21 @@ ] } ], - "source": [ - "# Create a new folder with the same name as the source folder to clone the contents to\n", - "target.content.create_folder(folder=fldr['title']+\"_Python\")\n", - "\n", - "# Clone items to the new folder\n", - "cloned_items = target.content.clone_items(items=fldr_items, folder=fldr['title']+\"_Python\")\n", - "print(*cloned_items, sep=\"\\n\")\n", - "print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ - "This notebook covered two use cases for the `clone_items()` method and is intended to be used as a guide; you can take what's here and incorporate it into your own workflows.\n", - "\n", - "What workflows or use cases do you have that we missed? Please let us know your use cases and workflows and we can work on incorporating them into the notebook.\n", - "\n", - "Notes on limitations:\n", - "- Clone fails with non-ASCII characters in service name.\n", - "- Cloning is limited to 1000 records.\n", - "- BUG-000136846 - The clone_items() method fails when attempting to clone a public hosted feature layer view hosted by another organization with the error message, \"User does not have permissions to access this service.\"\n", - "- Spaces in survey names can cause issues when cloning between organizations." - ] + "This notebook covered two use cases for the `clone_items()` method and is intended to be used as a guide; you can take what's here and incorporate it into your own workflows.\r\n", + "\r\n", + "What workflows or use cases do you have that we missed? Please let us know your use cases and workflows and we can work on incorporating them into the notebook.\r\n", + "\r\n", + "Notes on limitations:\r\n", + "- Clone fails with non-ASCII characters in service name.\r\n", + "- Cloning is limited to 1000 records.\r\n", + "- BUG-000136846 - The clone_items() method fails when attempting to clone a public hosted feature layer view hosted by another organization with the error message, \"User does not have permissions to access this service.\"" + ], + "metadata": {} } ], "metadata": { @@ -363,4 +373,4 @@ }, "nbformat": 4, "nbformat_minor": 4 -} +} \ No newline at end of file diff --git a/Update Attachment Keywords/update_attachment_keywords_in_arcgis_enterprise.ipynb b/Update Attachment Keywords/update_attachment_keywords_in_arcgis_enterprise.ipynb index 522906b..708c38b 100644 --- a/Update Attachment Keywords/update_attachment_keywords_in_arcgis_enterprise.ipynb +++ b/Update Attachment Keywords/update_attachment_keywords_in_arcgis_enterprise.ipynb @@ -2,14 +2,13 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, "source": [ "# Update attachment keywords in ArcGIS Enterprise" - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\n", "\n", @@ -33,28 +32,28 @@ "1. Overwrite the hosted feature service in your ArcGIS Enterprise portal with the updated file geodatabase.\n", "\n", "If you are using a feature service that points to an enterprise geodatabase, run the Upgrade Attachments tool on the attachment table directly." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 1, - "metadata": {}, - "outputs": [], "source": [ - "import logging\n", - "import os\n", - "import re\n", - "import datetime\n", - "import requests\n", - "import arcpy\n", - "import tempfile\n", - "from IPython.display import display\n", + "import logging\r\n", + "import os\r\n", + "import re\r\n", + "import datetime\r\n", + "import requests\r\n", + "import arcpy\r\n", + "import tempfile\r\n", + "from IPython.display import display\r\n", "from arcgis.gis import GIS" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Start by defining your variables. The variables are as follows: \n", "\n", @@ -64,77 +63,76 @@ "* **portal_url** - The URL for your organization (e.g. www.arcgis.com for ArcGIS Online).\n", "* **multiple_image_questions** - Does your survey have multiple image questions? Accepts `yes` or `no`. `yes` means that your survey contains multiple image questions; `no` means that your survey contains only one image question (this question can use use multiline appearance).\n", "* **attachment_keyword** - Only relevant if there is one image question in your survey. The script will prompt for the keyword, otherwise the script will extract the attachment keyword from the photo name if there are multiple image questions. For more information on how to identify the correct attachment keyword(s) for your survey, see the How To: Update the attachment keywords for existing ArcGIS Survey123 data Knowledge Base article." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 2, - "metadata": {}, - "outputs": [], "source": [ - "# What is the ID of the hosted feature layer associated with your survey?\n", - "feature_layer_id = ''\n", - "portal_username = ''\n", - "portal_password = ''\n", - "portal_url = 'https:///'\n", - "\n", - "# Do you have multiple image questions in your survey?\n", - "# 'yes' means you do have multiple image questions in your survey\n", - "# 'no' means you only have one image question (can use multiline appearance)\n", - "multiple_image_questions = 'yes'\n", - "\n", - "# If one image question, obtain attachment keyword from user else grab it from the attachment name later on\n", - "if multiple_image_questions == \"no\":\n", - " attachment_keyword = str(input(\"Please enter the attachment keyword to use: \"))\n", - "else:\n", + "# What is the ID of the hosted feature layer associated with your survey?\r\n", + "feature_layer_id = ''\r\n", + "portal_username = ''\r\n", + "portal_password = ''\r\n", + "portal_url = 'https:///'\r\n", + "\r\n", + "# Do you have multiple image questions in your survey?\r\n", + "# 'yes' means you do have multiple image questions in your survey\r\n", + "# 'no' means you only have one image question (can use multiline appearance)\r\n", + "multiple_image_questions = 'yes'\r\n", + "\r\n", + "# If one image question, obtain attachment keyword from user else grab it from the attachment name later on\r\n", + "if multiple_image_questions == \"no\":\r\n", + " attachment_keyword = str(input(\"Please enter the attachment keyword to use: \"))\r\n", + "else:\r\n", " attachment_keyword= ''" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "The next steps are to connect to your GIS, obtain the token for your session, and make a connection to the feature layer." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 3, - "metadata": {}, - "outputs": [], "source": [ - "# Connect to GIS and get feature layer information\n", - "if portal_username == '' and portal_password == '':\n", - " gis = GIS(profile='Survey123_prof')\n", - "else:\n", - " gis = GIS(portal_url, portal_username, portal_password)\n", - "\n", - "token = gis._con.token\n", + "# Connect to GIS and get feature layer information\r\n", + "if portal_username == '' and portal_password == '':\r\n", + " gis = GIS(profile='Survey123_prof')\r\n", + "else:\r\n", + " gis = GIS(portal_url, portal_username, portal_password)\r\n", + "\r\n", + "token = gis._con.token\r\n", "item_object = gis.content.get(feature_layer_id)" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ - "The `update_attachment()` function below is used to update the attachment keyword. The inputs of the function are as follows:\n", - "- The layers REST URL\n", - "- Token generated previously\n", - "- The current object ID used to construct the URL\n", - "- Attachment file path\n", - "- Attachment ID\n", - "- The keyword to use.\n", - "\n", - "Once all the parameters are obtained the function constructs the URL using the REST endpoint and current object ID. It then opens the attachment to be used as a file in the POST request. Next, it defines the parameters with the remaining input parameters and sends the POST request. \n", - "\n", - "The response JSON is returned indicating if the request was successful or not." - ] + "The `update_attachment()` function is used to update the keyword for each attachment. The arguments for the function are as follows:\r\n", + "- REST URL of the feature layer\r\n", + "- Token generated earlier\r\n", + "- Current object ID\r\n", + "- File path to local copy of the attachment\r\n", + "- Attachment ID\r\n", + "- Keyword to apply to each attachment\r\n", + "\r\n", + "Once all arguments are obtained the function constructs the URL using the REST endpoint and current object ID. It then opens the attachment to be used as a file in the POST request. Next, it defines the request parameters with the remaining input arguments and sends the POST request. \r\n", + "\r\n", + "The JSON response indicates if the request was successful or not." + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 4, - "metadata": {}, - "outputs": [], "source": [ "def update_attachment(url, token, oid, attachment, attachID, keyword):\r\n", " att_url = '{}/{}/updateAttachment'.format(url, oid)\r\n", @@ -152,42 +150,43 @@ " params = {'token': token,'f': 'json', 'attachmentId': attachID, 'keywords': keyword}\r\n", " r = requests.post(att_url, params, files=files)\r\n", " return r.json()" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "If there are multiple image questions in the survey, the `findattachmentkeyword()` function below is used to obtain the keyword from the name of each attachment. The function takes the attachment name as the input and extracts the text before the hyphen ('-') or underscore ('\\_').\n", "\n", "By default, attachments captured or uploaded in the Survey123 field app are named using the format `-`, and attachments captured in the Survey123 web app are named using the format `_`. The function only accepts attachment names in these two formats. All other names are ignored. There are two cases where the attachment name might not be in an acceptable format:\n", "- The name was modified from the default by the user.\n", "- The attachment was uploaded in the web app (as opposed to it being captured in the form by camera, signature, and so on). When an attachment is uploaded in the web app it retains the name of the source file." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 5, - "metadata": {}, - "outputs": [], "source": [ - "def findattachmentkeyword(attach_name):\n", - " kw = ''\n", - " # For attachments submitted in the field app\n", - " if any(\"-\" in s for s in attach_name):\n", - " part = attach_name.partition(\"-\")\n", - " kw = part[0]\n", - " # For attachments submitted in the web app\n", - " elif any(\"_\" in s for s in attach_name):\n", - " part = attach_name.partition(\"_\")\n", - " kw = part[0]\n", - "\n", + "def findattachmentkeyword(attach_name):\r\n", + " kw = ''\r\n", + " # For attachments submitted in the field app\r\n", + " if any(\"-\" in s for s in attach_name):\r\n", + " part = attach_name.partition(\"-\")\r\n", + " kw = part[0]\r\n", + " # For attachments submitted in the web app\r\n", + " elif any(\"_\" in s for s in attach_name):\r\n", + " part = attach_name.partition(\"_\")\r\n", + " kw = part[0]\r\n", + "\r\n", " return kw" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "The function below runs the entire workflow. \n", "\n", @@ -195,50 +194,12 @@ "- It then proceeds with attachments for all layers and tables in the feature service that have attachments enabled.\n", "- For each layer, the function queries the features in the layer and obtains a list of attachments for each object ID. \n", "- Each attachment is then downloaded to the temporary directory and is then updated using the `update_attachment()` function, including the attachment keyword either entered above or obtained from the `findattachmentkeyword()` function." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 1 on ObjectID 1 \n", - " With the response of {'updateAttachmentResult': {'objectId': 1, 'globalId': '{1804AA9F-4270-4575-A006-2A3B3CA2A370}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 2 on ObjectID 2 \n", - " With the response of {'updateAttachmentResult': {'objectId': 2, 'globalId': '{331182FF-25AC-4B90-9998-632F2DB1EAB8}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 3 on ObjectID 3 \n", - " With the response of {'updateAttachmentResult': {'objectId': 3, 'globalId': '{84C43A0A-F2B8-4AE5-8A07-1A439DEA6DDF}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 1 on ObjectID 1 \n", - " With the response of {'updateAttachmentResult': {'objectId': 1, 'globalId': '{D60DAFD3-B01C-4065-9397-55EBB1137FF9}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 2 on ObjectID 1 \n", - " With the response of {'updateAttachmentResult': {'objectId': 2, 'globalId': '{3C94303D-E26F-40BB-A58C-B5A60E460735}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 3 on ObjectID 1 \n", - " With the response of {'updateAttachmentResult': {'objectId': 3, 'globalId': '{55FC99FB-86A0-460D-9EA7-904DF1F8CD29}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 4 on ObjectID 1 \n", - " With the response of {'updateAttachmentResult': {'objectId': 4, 'globalId': '{53B288BA-78A0-4785-8666-CA7AA9E9BE2C}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 5 on ObjectID 2 \n", - " With the response of {'updateAttachmentResult': {'objectId': 5, 'globalId': '{1E2901D2-8F29-4B33-9664-0791CBE584CC}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 6 on ObjectID 2 \n", - " With the response of {'updateAttachmentResult': {'objectId': 6, 'globalId': '{3DF38311-0C43-465F-925F-B694A4A0DC4A}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 7 on ObjectID 2 \n", - " With the response of {'updateAttachmentResult': {'objectId': 7, 'globalId': '{27944AE4-2FEC-4B96-A2E9-87171A4F06B3}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 8 on ObjectID 2 \n", - " With the response of {'updateAttachmentResult': {'objectId': 8, 'globalId': '{4B004255-7CEF-45CD-918F-0AC50B77446A}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 9 on ObjectID 3 \n", - " With the response of {'updateAttachmentResult': {'objectId': 9, 'globalId': '{D451D3C9-03A4-4238-8A73-E93197C40A84}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 10 on ObjectID 3 \n", - " With the response of {'updateAttachmentResult': {'objectId': 10, 'globalId': '{312D5E72-6E3D-47CD-93C9-43C2E05E0328}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 11 on ObjectID 3 \n", - " With the response of {'updateAttachmentResult': {'objectId': 11, 'globalId': '{BC7CA91F-EB10-4A5E-9D99-7AAC77223FF5}', 'success': True}}\n", - "Completed updating attachment on feature layer 0-defects with ID 12 on ObjectID 3 \n", - " With the response of {'updateAttachmentResult': {'objectId': 12, 'globalId': '{43DD83EE-C399-4A83-9F1A-29606F73655A}', 'success': True}}\n" - ] - } - ], "source": [ "def update_attachments():\r\n", " with tempfile.TemporaryDirectory() as tmp:\r\n", @@ -290,7 +251,46 @@ " \r\n", "\r\n", "update = update_attachments()" - ] + ], + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 1 on ObjectID 1 \n", + " With the response of {'updateAttachmentResult': {'objectId': 1, 'globalId': '{1804AA9F-4270-4575-A006-2A3B3CA2A370}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 2 on ObjectID 2 \n", + " With the response of {'updateAttachmentResult': {'objectId': 2, 'globalId': '{331182FF-25AC-4B90-9998-632F2DB1EAB8}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 3 on ObjectID 3 \n", + " With the response of {'updateAttachmentResult': {'objectId': 3, 'globalId': '{84C43A0A-F2B8-4AE5-8A07-1A439DEA6DDF}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 1 on ObjectID 1 \n", + " With the response of {'updateAttachmentResult': {'objectId': 1, 'globalId': '{D60DAFD3-B01C-4065-9397-55EBB1137FF9}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 2 on ObjectID 1 \n", + " With the response of {'updateAttachmentResult': {'objectId': 2, 'globalId': '{3C94303D-E26F-40BB-A58C-B5A60E460735}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 3 on ObjectID 1 \n", + " With the response of {'updateAttachmentResult': {'objectId': 3, 'globalId': '{55FC99FB-86A0-460D-9EA7-904DF1F8CD29}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 4 on ObjectID 1 \n", + " With the response of {'updateAttachmentResult': {'objectId': 4, 'globalId': '{53B288BA-78A0-4785-8666-CA7AA9E9BE2C}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 5 on ObjectID 2 \n", + " With the response of {'updateAttachmentResult': {'objectId': 5, 'globalId': '{1E2901D2-8F29-4B33-9664-0791CBE584CC}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 6 on ObjectID 2 \n", + " With the response of {'updateAttachmentResult': {'objectId': 6, 'globalId': '{3DF38311-0C43-465F-925F-B694A4A0DC4A}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 7 on ObjectID 2 \n", + " With the response of {'updateAttachmentResult': {'objectId': 7, 'globalId': '{27944AE4-2FEC-4B96-A2E9-87171A4F06B3}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 8 on ObjectID 2 \n", + " With the response of {'updateAttachmentResult': {'objectId': 8, 'globalId': '{4B004255-7CEF-45CD-918F-0AC50B77446A}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 9 on ObjectID 3 \n", + " With the response of {'updateAttachmentResult': {'objectId': 9, 'globalId': '{D451D3C9-03A4-4238-8A73-E93197C40A84}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 10 on ObjectID 3 \n", + " With the response of {'updateAttachmentResult': {'objectId': 10, 'globalId': '{312D5E72-6E3D-47CD-93C9-43C2E05E0328}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 11 on ObjectID 3 \n", + " With the response of {'updateAttachmentResult': {'objectId': 11, 'globalId': '{BC7CA91F-EB10-4A5E-9D99-7AAC77223FF5}', 'success': True}}\n", + "Completed updating attachment on feature layer 0-defects with ID 12 on ObjectID 3 \n", + " With the response of {'updateAttachmentResult': {'objectId': 12, 'globalId': '{43DD83EE-C399-4A83-9F1A-29606F73655A}', 'success': True}}\n" + ] + } + ], + "metadata": {} } ], "metadata": { From 5399fcce5f3b84f5480f3d02b6429e87eaa6258e Mon Sep 17 00:00:00 2001 From: Zachary Sutherby Date: Fri, 23 Jul 2021 10:27:44 -0700 Subject: [PATCH 2/3] updates to clone surveys notebook --- ...eys_from_one_organization_to_another.ipynb | 40 ++-- ...er_for_an_arcgis_survey123_form_item.ipynb | 200 +++++++++--------- ...orts_using_the_arcgis_api_for_python.ipynb | 172 +++++++-------- 3 files changed, 206 insertions(+), 206 deletions(-) diff --git a/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb b/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb index 3ae1e05..c903f53 100644 --- a/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb +++ b/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb @@ -11,26 +11,26 @@ { "cell_type": "markdown", "source": [ - "### Introduction\n", - "\n", - "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\n", - "\n", - "A common question the Survey123 team has received from organization administrators is, \"What's the best way to clone my surveys from one organization to another?\"\n", - "\n", - "There are two common use cases for cloning surveys:\n", - "1. Create a copy of a survey in another ArcGIS organization. For example, a city's transportation and water departments have different ArcGIS Online organizations and the water department would benefit from having a copy of one of the transportation department's surveys as well as its associated web map and dashboard.\n", - "2. Clone a survey from a development organization in ArcGIS Enterprise to staging and production organizations.\n", - "\n", - "This sample Python notebook demonstrates how to clone surveys and associated content from one organization to another. This workflow can be used to clone surveys from ArcGIS Online to ArcGIS Online, ArcGIS Online to ArcGIS Enterprise, or ArcGIS Enterprise to ArcGIS Enterprise. The direction of cloning does not matter.\n", - "\n", - "This notebook demonstrates two cloning methods:\n", - "* Clone related items\n", - "* Clone survey folder\n", - "\n", - "The foundation of the workflow is the `clone_items()` method in the ArcGIS API for Python. This is the infrastructure that allows us to clone surveys from a source organization to a target organization. Given the different content and item types, possible ArcGIS Enterprise and ArcGIS Online configurations, security considerations, and item dependencies, the `clone_items()` method aims to produce an exact duplicate of an item that retains all of its functionality.\n", - "\n", - "Please note that cloning relies on the sharing model to determine the items a user can clone. If a user can access an item, that user can clone it. However, a user can't create any items in the target organization if they don't have the appropriate privileges to create content. For more information, see User types, roles, and privileges for ArcGIS Online or User types, roles, and privileges for ArcGIS Enterprise.\n", - "\n", + "### Introduction\r\n", + "\r\n", + "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\r\n", + "\r\n", + "A common question the Survey123 team has received from organization administrators is, \"What's the best way to clone my surveys from one organization to another?\"\r\n", + "\r\n", + "There are two common use cases for cloning surveys:\r\n", + "1. Create a copy of a survey in another ArcGIS organization. For example, a city's transportation and water departments have different ArcGIS Online organizations and the water department would benefit from having a copy of one of the transportation department's surveys as well as its associated web map and dashboard.\r\n", + "2. Clone a survey from a development organization in ArcGIS Enterprise to staging and production organizations.\r\n", + "\r\n", + "This sample Python notebook demonstrates how to clone surveys and associated content from one organization to another. This workflow can be used to clone surveys from ArcGIS Online to ArcGIS Online, ArcGIS Online to ArcGIS Enterprise, or ArcGIS Enterprise to ArcGIS Enterprise. The direction of cloning does not matter.\r\n", + "\r\n", + "This notebook demonstrates two cloning methods:\r\n", + "* Clone related items\r\n", + "* Clone survey folder\r\n", + "\r\n", + "The foundation of the workflow is the `clone_items()` method in the ArcGIS API for Python. This is the infrastructure that allows us to clone surveys from a source organization to a target organization. Given the different content and item types, possible ArcGIS Enterprise and ArcGIS Online configurations, security considerations, and item dependencies, the `clone_items()` method aims to produce an exact duplicate of an item that retains all of its functionality.\r\n", + "\r\n", + "Please note that cloning relies on the sharing model to determine the items a user can clone. The user specified in the source organization will need admin access to the content that will be cloned, and the user specified in the target organization will need the ability to create content in that organization.\r\n", + "\r\n", "For more information on the `clone_items` method, see the ArcGIS API for Python Cloning content guide and API reference. " ], "metadata": {} diff --git a/Update Media Folder/update_contents_of_the_media_folder_for_an_arcgis_survey123_form_item.ipynb b/Update Media Folder/update_contents_of_the_media_folder_for_an_arcgis_survey123_form_item.ipynb index 1803cf1..8c939eb 100644 --- a/Update Media Folder/update_contents_of_the_media_folder_for_an_arcgis_survey123_form_item.ipynb +++ b/Update Media Folder/update_contents_of_the_media_folder_for_an_arcgis_survey123_form_item.ipynb @@ -2,14 +2,13 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, "source": [ "# Update contents of the media folder for an ArcGIS Survey123 form item" - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\n", "\n", @@ -32,261 +31,262 @@ "* **itemID** - The item ID for the ArcGIS Survey123 form item in your ArcGIS organization (e.g. 89bc8c7844e548e09baa3aad4695e78b)\n", "* **updated_files** - The updated file name containing the extension (e.g. myphoto.png)\n", "* **source_loc** - Folder directory where the updated file is located (e.g. C:/Users/username/ArcGIS/My Survey Designs/...)" - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 1, - "metadata": {}, - "outputs": [], "source": [ - "import arcgis\n", - "from arcgis.gis import GIS\n", - "import tempfile\n", - "import zipfile\n", - "import shutil\n", - "import os\n", - "\n", - "portalURL = r'https://www.arcgis.com'\n", - "username = ''\n", - "password = ''\n", - "itemID = 'c3318288b6254e1ebcd1f1dba70dc3e0'\n", - "# Add one or more file names to the list.\n", - "updated_files = ['employees.csv', 'locations.csv']\n", + "import arcgis\r\n", + "from arcgis.gis import GIS\r\n", + "import tempfile\r\n", + "import zipfile\r\n", + "import shutil\r\n", + "import os\r\n", + "\r\n", + "portalURL = r'https://www.arcgis.com'\r\n", + "username = ''\r\n", + "password = ''\r\n", + "itemID = 'c3318288b6254e1ebcd1f1dba70dc3e0'\r\n", + "# Add one or more file names to the list.\r\n", + "updated_files = ['employees.csv', 'locations.csv']\r\n", "source_loc = r'/arcgis/home/'" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Uncomment the first GIS connection if using the `portalURL`, `username`, and `password` variables from above. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 2, - "metadata": {}, - "outputs": [], "source": [ - "# Connect to GIS\n", - "# gis = GIS(portalURL, username, password)\n", + "# Connect to GIS\r\n", + "# gis = GIS(portalURL, username, password)\r\n", "gis = GIS(\"home\")" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "## Download survey\n", "\n", "Start by obtaining the properties of the Survey123 form item. These properties are used to reconstruct the ZIP file name that is uploaded. The first line in the code block below defines a Survey Manager. A survey in the Survey Manager is a single instance of a survey project that contains the item information and properties and provides access to the underlying survey dataset. For more information on Survey Manager, see the API Reference for the ArcGIS API for Python." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "source": [ + "survey_manager = arcgis.apps.survey123.SurveyManager(gis)\r\n", + "surveyId = survey_manager.get(itemID)\r\n", + "surveyProp = surveyId.properties\r\n", + "print(surveyProp)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "{'id': 'c3318288b6254e1ebcd1f1dba70dc3e0', 'owner': 'NinjaGreen', 'created': 1612817471000, 'isOrgItem': True, 'modified': 1617393961000, 'guid': None, 'name': 'EmployeeForm.zip', 'title': 'EmployeeForm', 'type': 'Form', 'typeKeywords': ['Form', 'Survey123', 'Survey123 Connect', 'xForm'], 'description': 'This template includes all XLSForm features supported in ArcGIS Survey123.', 'tags': [], 'snippet': '', 'thumbnail': 'thumbnail/EmployeeForm.png', 'documentation': None, 'extent': [], 'categories': [], 'spatialReference': None, 'accessInformation': None, 'licenseInfo': None, 'culture': 'en-us', 'properties': None, 'url': None, 'proxyFilter': None, 'access': 'private', 'size': 51654, 'subInfo': 0, 'appCategories': [], 'industries': [], 'languages': [], 'largeThumbnail': None, 'banner': None, 'screenshots': [], 'listed': False, 'ownerFolder': '60384ef99a694dccb4fda1f2d9d46b47', 'protected': False, 'commentsEnabled': True, 'numComments': 0, 'numRatings': 0, 'avgRating': 0, 'numViews': 24, 'itemControl': 'admin', 'scoreCompleteness': 25, 'groupDesignations': None}\n" ] } ], - "source": [ - "survey_manager = arcgis.apps.survey123.SurveyManager(gis)\n", - "surveyId = survey_manager.get(itemID)\n", - "surveyProp = surveyId.properties\n", - "print(surveyProp)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Find the form item in the `gis` and download as a ZIP file to a temporary folder directory. Additional information on downloading content can be found here." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 4, - "metadata": {}, + "source": [ + "itm = arcgis.gis.Item(gis,itemID)\n", + "print(itm)\n", + "tmpdir = tempfile.TemporaryDirectory()\n", + "download_folder = tmpdir.name\n", + "savedZip = itm.download(save_path=download_folder)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "\n" ] } ], - "source": [ - "itm = arcgis.gis.Item(gis,itemID)\n", - "print(itm)\n", - "tmpdir = tempfile.TemporaryDirectory()\n", - "download_folder = tmpdir.name\n", - "savedZip = itm.download(save_path=download_folder)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Extract the ZIP file to an *`_extracted`* folder in the download location. This *`_extracted`* folder is where the updated media files will be copied and rezipped later on. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 5, - "metadata": {}, - "outputs": [], "source": [ "def extractZIP(filename,folder):\n", " zfile = zipfile.ZipFile(filename)\n", " zfile.extractall(folder)\n", "\n", "extractZIP(savedZip, os.path.join(download_folder + \"/_extracted/\"))\n" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Copy the updated file to the media folder, replacing the old file. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 6, - "metadata": {}, + "source": [ + "for file in updated_files:\n", + " source_file = os.path.join(source_loc, file)\n", + " dest_file = download_folder + \"/_extracted/esriinfo/media/\" + file\n", + " shutil.copyfile(source_file, dest_file)\n", + " print (file, \" updated to: \", download_folder + \"/_extracted/esriinfo/media/\")" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "employees.csv updated to: /tmp/tmpkxv1hpzo/_extracted/esriinfo/media/\n", "locations.csv updated to: /tmp/tmpkxv1hpzo/_extracted/esriinfo/media/\n" ] } ], - "source": [ - "for file in updated_files:\n", - " source_file = os.path.join(source_loc, file)\n", - " dest_file = download_folder + \"/_extracted/esriinfo/media/\" + file\n", - " shutil.copyfile(source_file, dest_file)\n", - " print (file, \" updated to: \", download_folder + \"/_extracted/esriinfo/media/\")" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Delete the ZIP file you downloaded previously. This will prevent any namespace issues and ensure the process of zipping and uploading the updated survey goes smoothly. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 7, - "metadata": {}, + "source": [ + "os.remove(savedZip)\n", + "print (\"Old ZIP file deleted from: \" + download_folder)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Old zip file deleted from: /tmp/tmpkxv1hpzo\n" ] } ], - "source": [ - "os.remove(savedZip)\n", - "print (\"Old ZIP file deleted from: \" + download_folder)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "## Upload updated survey\n", "\n", "Zip the updated survey and place it in the download folder you defined previously. \"The code below extracts the survey title from the survey properties and passes it into the file name for the ZIP file." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 8, - "metadata": {}, + "source": [ + "zipFileName = surveyProp['title']\n", + "updateZip = shutil.make_archive(zipFileName, 'zip', download_folder + '/_extracted/')\n", + "print (updateZip)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "/arcgis/EmployeeForm.zip\n" ] } ], - "source": [ - "zipFileName = surveyProp['title']\n", - "updateZip = shutil.make_archive(zipFileName, 'zip', download_folder + '/_extracted/')\n", - "print (updateZip)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Upload the new ZIP file and update the form item with the new media folder content. For more information on the update property of items please visit this link." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 9, - "metadata": {}, + "source": [ + "itm.update({},updateZip)" + ], "outputs": [ { + "output_type": "execute_result", "data": { "text/plain": [ "True" ] }, - "execution_count": 9, "metadata": {}, - "output_type": "execute_result" + "execution_count": 9 } ], - "source": [ - "itm.update({},updateZip)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Clean up the intermediate data. This process deletes the updated ZIP file as well as the extracted folder containing the unzipped survey content." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 10, - "metadata": {}, + "source": [ + "tmpdir.cleanup()\n", + "print (\"Temp folder deleted from: \" + download_folder)\n", + "# print (zipFileName + \" successfully updated with \" + source_file + \" and uploaded to your organization!\")\n", + "print (zipFileName + \" successfully updated with \" + ' & '.join(map(str, updated_files)) + \" and uploaded to your organization!\")" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Temp folder deleted from: /tmp/tmpkxv1hpzo\n", "EmployeeForm successfully updated with employees.csv & locations.csv and uploaded to your organization!\n" ] } ], - "source": [ - "tmpdir.cleanup()\n", - "print (\"Temp folder deleted from: \" + download_folder)\n", - "# print (zipFileName + \" successfully updated with \" + source_file + \" and uploaded to your organization!\")\n", - "print (zipFileName + \" successfully updated with \" + ' & '.join(map(str, updated_files)) + \" and uploaded to your organization!\")" - ] + "metadata": {} } ], "metadata": { @@ -310,4 +310,4 @@ }, "nbformat": 4, "nbformat_minor": 4 -} +} \ No newline at end of file diff --git a/Work with Reports/create_reports_using_the_arcgis_api_for_python.ipynb b/Work with Reports/create_reports_using_the_arcgis_api_for_python.ipynb index c0029e1..c98ff79 100644 --- a/Work with Reports/create_reports_using_the_arcgis_api_for_python.ipynb +++ b/Work with Reports/create_reports_using_the_arcgis_api_for_python.ipynb @@ -2,131 +2,130 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, "source": [ "# Create reports using the ArcGIS API for Python" - ] + ], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ - "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\n", - "\n", - "ArcGIS Survey123 reports are an effective way to quickly visualize, interpret, and share survey responses. Reports use a template you can personalize, that's applied to your survey results and shared as either a Microsoft Word or PDF document. \n", - "\n", - "It's common to deploy a webhook as an automated process to create and share reports with survey results. Although, using a third party webhook provider may not be a feasible workflow due to organizational or security policies. This is where the ArcGIS Survey123 module in the ArcGIS API for Python can be used to acheive a similar goal.\n", - "\n", - "This sample notebook demonstrates ways you can use the ArcGIS Survey123 module to work with reports and covers the following functionality: \n", - "\n", - "* Generate a default report template\n", - "* Identify report templates associated with a survey\n", - "* Generate a single report\n", - "* Generate multiple reports\n", - "* Generate multiple reports and save to your organization\n", - "* List saved reports\n", - "\n", - "
The Survey123 report service is an ArcGIS Online Premium service. It takes 0.5 credits per survey report generated. For more information, see the Understand credits documentation.
\n", - "\n", + "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\r\n", + "\r\n", + "ArcGIS Survey123 reports are an effective way to quickly visualize, interpret, and share survey responses. Reports use a template you can personalize, that's applied to your survey results and shared as either a Microsoft Word or PDF document. \r\n", + "\r\n", + "It's common to deploy a webhook as an automated process to create and share reports with survey results. Although, you may want to automate this process over a specified time frame. For example you may the past week, month or quarter. This is where the ArcGIS Survey123 module in the ArcGIS API for Python can be used to acheive a similar goal. \r\n", + "\r\n", + "This sample notebook demonstrates ways you can use the ArcGIS Survey123 module to work with reports and covers the following functionality: \r\n", + "\r\n", + "* Generate a default report template\r\n", + "* Identify report templates associated with a survey\r\n", + "* Generate a single report\r\n", + "* Generate multiple reports\r\n", + "* Generate multiple reports and save to your organization\r\n", + "* List saved reports\r\n", + "\r\n", + "
The Survey123 report service is an ArcGIS Online Premium service. It takes 0.5 credits per survey report generated. For more information, see the Understand credits documentation.
\r\n", + "\r\n", "The first step is to connect to your GIS." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 1, - "metadata": {}, - "outputs": [], "source": [ - "import arcgis\n", - "from arcgis.gis import GIS\n", + "import arcgis\r\n", + "from arcgis.gis import GIS\r\n", "gis = GIS(\"home\")" - ] + ], + "outputs": [], + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "Next, a Survey Manager is defined, and a survey is accessed using the form's item ID. A survey in the Survey Manager is a single instance of a survey project that contains the item information and properties and provides access to the underlying survey dataset. For more information on Survey Manager, see the API Reference for the ArcGIS API for Python." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 2, - "metadata": {}, + "source": [ + "survey_manager = arcgis.apps.survey123.SurveyManager(gis)\r\n", + "survey_by_id = survey_manager.get(\"9f01838a15594cfdbb2eb69fafb60d75\")\r\n", + "print(survey_by_id)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "\n" ] } ], - "source": [ - "survey_manager = arcgis.apps.survey123.SurveyManager(gis)\n", - "survey_by_id = survey_manager.get(\"9f01838a15594cfdbb2eb69fafb60d75\")\n", - "print(survey_by_id)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Identify report templates associated with a survey\n", "\n", "Using the `report_templates` method any report template already associated with the survey will be returned in a list. For each template in the list the title is printed using the print statement below. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 3, - "metadata": {}, + "source": [ + "templates = survey_by_id.report_templates\r\n", + "p = [print(t.title) for t in templates]" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Water Quality Inspection_sampleTemplate\n" ] } ], - "source": [ - "templates = survey_by_id.report_templates\n", - "p = [print(t.title) for t in templates]" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Generating a default report template\n", "\n", "In the case where there are not any report templates associated with the survey, a default template can be generated using the `create_report_template` method. The `create_report_template` method creates a simple default template automatically downloads it to the `Temp` folder. The ArcGIS Survey123 module does not support uploading a new report template." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 4, - "metadata": {}, + "source": [ + "new_template = survey_by_id.create_report_template()\r\n", + "print(new_template)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "/tmp/template.docx\n" ] } ], - "source": [ - "new_template = survey_by_id.create_report_template()\n", - "print(new_template)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Generate a single report\n", @@ -134,57 +133,57 @@ "Use the `generate_report` method to generate a single report by specifying a single object ID in the `where` parameter. Since there was only one report template associated with the survey the index of 0 is used. \n", "\n", "The `where` clause follows the same format as the `where` parameter when querying a feature service layer. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 5, - "metadata": {}, + "source": [ + "report = survey_by_id.generate_report(templates[0], where=\"objectid=122\")\r\n", + "print(report)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "/tmp/Water_Quality_Inspection_report_e68c44.docx\n" ] } ], - "source": [ - "report = survey_by_id.generate_report(templates[0], where=\"objectid=122\")\n", - "print(report)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Generate multiple reports\n", "\n", "If you are looking to generate reports for multiple object IDs in the survey you can specify `\"objectid=1 OR objectid=2\"` to generate reports for those two features. Alternatively, omitting the `where` clause will result in reports created for all features in the feature service. In either scenario the output is saved as a ZIP file, and will be stored locally on disk if run through a Jupyter Notebook. " - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 6, - "metadata": {}, + "source": [ + "batch_reports = survey_by_id.generate_report(templates[0])\r\n", + "print(batch_reports)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "/tmp/Water_Quality_Inspection_report_7a8ea1.zip\n" ] } ], - "source": [ - "batch_reports = survey_by_id.generate_report(templates[0])\n", - "print(batch_reports)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### Generate multiple reports and save to your organization as an item\n", @@ -194,46 +193,50 @@ "
This parameter is required in order to upload your report to your organization. However, the name of your report or code sample will default to survey_name_report_unique_ID.
\n", "\n", "The code sample can then be shared to a group or publicly so that others can download the ZIP and view the reports." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 7, - "metadata": {}, + "source": [ + "import datetime\r\n", + "nowstring = datetime.datetime.now().strftime(\"%Y%m%d%H%M%S\")\r\n", + "batch_reports = survey_by_id.generate_report(templates[0], report_title=\"Test_{0}\".format(nowstring))\r\n", + "print(batch_reports)" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "\n" ] } ], - "source": [ - "import datetime\n", - "nowstring = datetime.datetime.now().strftime(\"%Y%m%d%H%M%S\")\n", - "batch_reports = survey_by_id.generate_report(templates[0], report_title=\"Test_{0}\".format(nowstring))\n", - "print(batch_reports)" - ] + "metadata": {} }, { "cell_type": "markdown", - "metadata": {}, "source": [ "\n", "### List saved reports\n", "\n", "Finally, use the `reports` property to list the reports stored in the ArcGIS organizational account from the current GIS connection. This shows all reports that are saved in that named user's content." - ] + ], + "metadata": {} }, { "cell_type": "code", "execution_count": 8, - "metadata": {}, + "source": [ + "recentReports = survey_by_id.reports\r\n", + "p = [print(t.title) for t in recentReports]" + ], "outputs": [ { - "name": "stdout", "output_type": "stream", + "name": "stdout", "text": [ "Manhole_Inspection_report_7844e0\n", "Tree_Health_Survey_statistic.docx\n", @@ -244,10 +247,7 @@ ] } ], - "source": [ - "recentReports = survey_by_id.reports\n", - "p = [print(t.title) for t in recentReports]" - ] + "metadata": {} } ], "metadata": { @@ -271,4 +271,4 @@ }, "nbformat": 4, "nbformat_minor": 4 -} +} \ No newline at end of file From 56ddb330a3bc0fec4a41ec23db614e2225f0ca35 Mon Sep 17 00:00:00 2001 From: Zachary Sutherby Date: Fri, 23 Jul 2021 10:34:26 -0700 Subject: [PATCH 3/3] Added defect number --- ...eys_from_one_organization_to_another.ipynb | 336 +++++++++--------- 1 file changed, 173 insertions(+), 163 deletions(-) diff --git a/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb b/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb index c903f53..35e6bca 100644 --- a/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb +++ b/Clone Surveys/clone_surveys_from_one_organization_to_another.ipynb @@ -2,93 +2,86 @@ "cells": [ { "cell_type": "markdown", + "metadata": {}, "source": [ "# Clone surveys from one organization to another\n", "\n" - ], - "metadata": {} + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ - "### Introduction\r\n", - "\r\n", - "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\r\n", - "\r\n", - "A common question the Survey123 team has received from organization administrators is, \"What's the best way to clone my surveys from one organization to another?\"\r\n", - "\r\n", - "There are two common use cases for cloning surveys:\r\n", - "1. Create a copy of a survey in another ArcGIS organization. For example, a city's transportation and water departments have different ArcGIS Online organizations and the water department would benefit from having a copy of one of the transportation department's surveys as well as its associated web map and dashboard.\r\n", - "2. Clone a survey from a development organization in ArcGIS Enterprise to staging and production organizations.\r\n", - "\r\n", - "This sample Python notebook demonstrates how to clone surveys and associated content from one organization to another. This workflow can be used to clone surveys from ArcGIS Online to ArcGIS Online, ArcGIS Online to ArcGIS Enterprise, or ArcGIS Enterprise to ArcGIS Enterprise. The direction of cloning does not matter.\r\n", - "\r\n", - "This notebook demonstrates two cloning methods:\r\n", - "* Clone related items\r\n", - "* Clone survey folder\r\n", - "\r\n", - "The foundation of the workflow is the `clone_items()` method in the ArcGIS API for Python. This is the infrastructure that allows us to clone surveys from a source organization to a target organization. Given the different content and item types, possible ArcGIS Enterprise and ArcGIS Online configurations, security considerations, and item dependencies, the `clone_items()` method aims to produce an exact duplicate of an item that retains all of its functionality.\r\n", - "\r\n", - "Please note that cloning relies on the sharing model to determine the items a user can clone. The user specified in the source organization will need admin access to the content that will be cloned, and the user specified in the target organization will need the ability to create content in that organization.\r\n", - "\r\n", + "### Introduction\n", + "\n", + "
This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.
\n", + "\n", + "A common question the Survey123 team has received from organization administrators is, \"What's the best way to clone my surveys from one organization to another?\"\n", + "\n", + "There are two common use cases for cloning surveys:\n", + "1. Create a copy of a survey in another ArcGIS organization. For example, a city's transportation and water departments have different ArcGIS Online organizations and the water department would benefit from having a copy of one of the transportation department's surveys as well as its associated web map and dashboard.\n", + "2. Clone a survey from a development organization in ArcGIS Enterprise to staging and production organizations.\n", + "\n", + "This sample Python notebook demonstrates how to clone surveys and associated content from one organization to another. This workflow can be used to clone surveys from ArcGIS Online to ArcGIS Online, ArcGIS Online to ArcGIS Enterprise, or ArcGIS Enterprise to ArcGIS Enterprise. The direction of cloning does not matter.\n", + "\n", + "This notebook demonstrates two cloning methods:\n", + "* Clone related items\n", + "* Clone survey folder\n", + "\n", + "The foundation of the workflow is the `clone_items()` method in the ArcGIS API for Python. This is the infrastructure that allows us to clone surveys from a source organization to a target organization. Given the different content and item types, possible ArcGIS Enterprise and ArcGIS Online configurations, security considerations, and item dependencies, the `clone_items()` method aims to produce an exact duplicate of an item that retains all of its functionality.\n", + "\n", + "Please note that cloning relies on the sharing model to determine the items a user can clone. The user specified in the source organization will need admin access to the content that will be cloned, and the user specified in the target organization will need the ability to create content in that organization.\n", + "\n", "For more information on the `clone_items` method, see the ArcGIS API for Python Cloning content guide and API reference. " - ], - "metadata": {} + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "### Prepare to clone\n", "\n", "To start, we are going to need two GIS connections: one to our \"source\" organization, which is the organization in which the survey and content currently resides that we would like to clone; and another to a \"target\" organization, which is the organization that we would like to clone the survey and content to." - ], - "metadata": {} + ] }, { "cell_type": "code", "execution_count": 1, - "source": [ - "import arcgis\r\n", - "from arcgis.gis import GIS\r\n", - "import os\r\n", - "\r\n", - "# Connect to source and target organizations\r\n", - "source = GIS(profile=\"sourceOrg\")\r\n", - "target = GIS(profile=\"TargetOrg\")\r\n", - "print(\"Source GIS:\", source,\"\\nTarget GIS:\", target)" - ], + "metadata": {}, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Source GIS: GIS @ https://Survey123ninja.maps.arcgis.com version:8.4 \n", "Target GIS: GIS @ https://Survey123.maps.arcgis.com version:8.4\n" ] } ], - "metadata": {} + "source": [ + "import arcgis\n", + "from arcgis.gis import GIS\n", + "import os\n", + "\n", + "# Connect to source and target organizations\n", + "source = GIS(profile=\"sourceOrg\")\n", + "target = GIS(profile=\"TargetOrg\")\n", + "print(\"Source GIS:\", source,\"\\nTarget GIS:\", target)" + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "The first example highlights a workflow where you have a few surveys shared to a group that you would like to clone to a different organization. In order to work with your surveys a Survey Manager is defined. A survey in the Survey Manager is a single instance of a survey project that contains the item information and properties and provides access to the underlying survey dataset. In this example, four surveys are shared to a group. Using the group ID, a connection is made to the group and a list is created containing all form items in the group." - ], - "metadata": {} + ] }, { "cell_type": "code", "execution_count": 2, - "source": [ - "# Get surveys by group ID and then download each format supported\r\n", - "survey_manager = arcgis.apps.survey123.SurveyManager(source)\r\n", - "group = source.groups.get('0dd43178863545fbb5526278e1bdfc4c')\r\n", - "sourceForms = group.search('type:form')['results']\r\n", - "sourceForms" - ], + "metadata": {}, "outputs": [ { - "output_type": "execute_result", "data": { "text/plain": [ "[,\n", @@ -97,21 +90,29 @@ " ]" ] }, + "execution_count": 2, "metadata": {}, - "execution_count": 2 + "output_type": "execute_result" } ], - "metadata": {} + "source": [ + "# Get surveys by group ID and then download each format supported\n", + "survey_manager = arcgis.apps.survey123.SurveyManager(source)\n", + "group = source.groups.get('0dd43178863545fbb5526278e1bdfc4c')\n", + "sourceForms = group.search('type:form')['results']\n", + "sourceForms" + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "Now that we have your forms as a list, you are ready to clone the content from the source organization to the target organization. As previously noted, a use case for using the `clone_items()` method is to clone surveys between development, staging, and production organizations. This first example clones the surveys from an existing group (as defined above) located in the source organization to the target organization and shares the cloned surveys to a group with the same name in the target organization." - ], - "metadata": {} + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "\n", "### Clone related items\n", @@ -123,57 +124,16 @@ "Next, a new folder in the target organization is created based on the survey name, and the items in the list are cloned. Since the `copy_data` parameter is set to \"False\" in the `clone_items()` method, the resulting services created in the target organization will not contain any data. This is ideal when cloning from a development environment to a staging or production environment, as you might not wish to retain any test data.\n", "\n", "Finally, the form items are shared to the group in the target organization that was defined previously, including the feature service, and tags are added to each item." - ], - "metadata": {} + ] }, { "cell_type": "code", "execution_count": 3, - "source": [ - "# Create new group in the target environment\r\n", - "shared_group = target.groups.create(title=group.title, tags=group.tags)\r\n", - "\r\n", - "# Clone source items to the target environment\r\n", - "for form_item in sourceForms:\r\n", - "\r\n", - " # Obtain the feature service associated with the survey\r\n", - " related_items = form_item.related_items('Survey2Service','forward')\r\n", - " # Obtain the additional items that have a relationship with the survey\r\n", - " additional_items = form_item.related_items('Survey2Data','forward')\r\n", - " all_items = [form_item] + related_items + additional_items\r\n", - " print(\"Source items: \", *all_items, sep=\"\\n\")\r\n", - " \r\n", - " # Create new folder according to survey name \r\n", - " title = form_item.title.replace(\" \", \"_\")\r\n", - " folderName = \"Survey-\" + title\r\n", - " target.content.create_folder(folderName)\r\n", - " \r\n", - " # Clone all items to the new folder in target environment\r\n", - " cloned_items = target.content.clone_items(items=all_items, folder=folderName, copy_data=False)\r\n", - " \r\n", - " # Check the feature count in cloned feature layer. Feature count should 0 because existing data is not copied\r\n", - " print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\r\n", - " \r\n", - " for item in cloned_items:\r\n", - " if item.type == 'Form':\r\n", - " # Update the form item to ensure all resources are rebuilt\r\n", - " downloaded_item = item.download(file_name=item.id+'.zip')\r\n", - " item.update({}, downloaded_item)\r\n", - " os.remove(downloaded_item)\r\n", - " # Share the form item to the group\r\n", - " item.update(item_properties={'tags':'PrdEnv, PythonAPI'})\r\n", - " item.share(groups=shared_group.id)\r\n", - " # Share source feature service to group\r\n", - " source_fs = item.related_items('Survey2Service','forward')[0]\r\n", - " source_fs.update(item_properties={'tags':'PrdEnv, PythonAPI'})\r\n", - " source_fs.share(groups=shared_group.id)\r\n", - " else:\r\n", - " pass" - ], + "metadata": {}, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Source items: \n", "\n", @@ -196,29 +156,65 @@ ] } ], - "metadata": {} + "source": [ + "# Create new group in the target environment\n", + "shared_group = target.groups.create(title=group.title, tags=group.tags)\n", + "\n", + "# Clone source items to the target environment\n", + "for form_item in sourceForms:\n", + "\n", + " # Obtain the feature service associated with the survey\n", + " related_items = form_item.related_items('Survey2Service','forward')\n", + " # Obtain the additional items that have a relationship with the survey\n", + " additional_items = form_item.related_items('Survey2Data','forward')\n", + " all_items = [form_item] + related_items + additional_items\n", + " print(\"Source items: \", *all_items, sep=\"\\n\")\n", + " \n", + " # Create new folder according to survey name \n", + " title = form_item.title.replace(\" \", \"_\")\n", + " folderName = \"Survey-\" + title\n", + " target.content.create_folder(folderName)\n", + " \n", + " # Clone all items to the new folder in target environment\n", + " cloned_items = target.content.clone_items(items=all_items, folder=folderName, copy_data=False)\n", + " \n", + " # Check the feature count in cloned feature layer. Feature count should 0 because existing data is not copied\n", + " print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\n", + " \n", + " for item in cloned_items:\n", + " if item.type == 'Form':\n", + " # Update the form item to ensure all resources are rebuilt\n", + " downloaded_item = item.download(file_name=item.id+'.zip')\n", + " item.update({}, downloaded_item)\n", + " os.remove(downloaded_item)\n", + " # Share the form item to the group\n", + " item.update(item_properties={'tags':'PrdEnv, PythonAPI'})\n", + " item.share(groups=shared_group.id)\n", + " # Share source feature service to group\n", + " source_fs = item.related_items('Survey2Service','forward')[0]\n", + " source_fs.update(item_properties={'tags':'PrdEnv, PythonAPI'})\n", + " source_fs.share(groups=shared_group.id)\n", + " else:\n", + " pass" + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "As a confidence check, let's query the newly created group and confirm our four surveys have been shared. " - ], - "metadata": {} + ] }, { "cell_type": "code", "execution_count": 4, - "source": [ - "# Check items in the new group in target environment.\r\n", - "import time\r\n", - "time.sleep(30)\r\n", - "targetForms = target.content.search('type:form group:'+shared_group.id)\r\n", - "print(\"Cloned surveys: \", *targetForms, sep=\"\\n\")" - ], + "metadata": { + "scrolled": true + }, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Cloned surveys: \n", "\n", @@ -228,21 +224,26 @@ ] } ], - "metadata": { - "scrolled": true - } + "source": [ + "# Check items in the new group in target environment.\n", + "import time\n", + "time.sleep(30)\n", + "targetForms = target.content.search('type:form group:'+shared_group.id)\n", + "print(\"Cloned surveys: \", *targetForms, sep=\"\\n\")" + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "The example above is a useful workflow for cloning surveys between development, staging, and production environments and will only clone items that are linked to the form item, including: report templates, linked web maps, CSVs, and map packages. But what if you would like to clone not only the survey and its related items, but also the survey data that's already been collected? Additionally, what if you have web maps, web apps, or dashboards that use the survey data but aren't linked to the survey? \n", "\n", "If those items are stored in the survey's folder, a slightly different method can be used to clone all the content in this folder." - ], - "metadata": {} + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "\n", "### Clone survey folder\n", @@ -250,31 +251,16 @@ "The code below demonstrates connecting to one specific survey in your organization. Using the properties of the survey, the folder ID where the survey resides is assigned to a variable. Next, all the folders in the source organization for the source username are listed. Using list comprehension, the folder in the `full_folder` variable is matched with the folder ID obtained from the survey properties.\n", "\n", "Once the correct folder has been identified the contents of the folder are listed." - ], - "metadata": {} + ] }, { "cell_type": "code", "execution_count": 5, - "source": [ - "source_item_with_data = survey_manager.get(\"65b0ce4cfa2145eb8ce90122e54029e6\")\r\n", - "survey_folder = source_item_with_data.properties['ownerFolder']\r\n", - "\r\n", - "usr = arcgis.gis.User(source, source.users.me.username)\r\n", - "\r\n", - "full_folder = usr.folders\r\n", - "\r\n", - "# Identify the folder associated with the survey\r\n", - "fldr = next((f for f in full_folder if f['id'] == survey_folder), 0)\r\n", - "\r\n", - "#List all the items within that folder to be cloned later on\r\n", - "fldr_items = usr.items(folder=fldr)\r\n", - "print(\"Folder items:\", *fldr_items, sep=\"\\n\")" - ], + "metadata": {}, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "Folder items:\n", "\n", @@ -289,38 +275,37 @@ ] } ], - "metadata": {} + "source": [ + "source_item_with_data = survey_manager.get(\"65b0ce4cfa2145eb8ce90122e54029e6\")\n", + "survey_folder = source_item_with_data.properties['ownerFolder']\n", + "\n", + "usr = arcgis.gis.User(source, source.users.me.username)\n", + "\n", + "full_folder = usr.folders\n", + "\n", + "# Identify the folder associated with the survey\n", + "fldr = next((f for f in full_folder if f['id'] == survey_folder), 0)\n", + "\n", + "#List all the items within that folder to be cloned later on\n", + "fldr_items = usr.items(folder=fldr)\n", + "print(\"Folder items:\", *fldr_items, sep=\"\\n\")" + ] }, { "cell_type": "markdown", + "metadata": {}, "source": [ "Now that all items to be cloned are in a list, a new folder is created in the target organization to store the content. After the folder is created the content is cloned to the target environment. Since the `copy_data` parameter is not defined the default value for the parameter is \"True\", meaning all the underlying data will also be cloned. This means the resulting content in the target organization will be an identical clone of the original data. If you do not wish to retain the source data, setting the `copy_data` parameter to \"False\" will only clone the data schema and architecture to the target organization. The survey, web maps, web apps, dashboards, and other items will be configured as per their original items; the only difference is the feature layer will be empty." - ], - "metadata": {} + ] }, { "cell_type": "code", "execution_count": 6, - "source": [ - "# Create a new folder with the same name as the source folder to clone the contents to\r\n", - "target.content.create_folder(folder=fldr['title']+\"_Python\")\r\n", - "\r\n", - "# Clone items to the new folder\r\n", - "cloned_items = target.content.clone_items(items=fldr_items, folder=fldr['title']+\"_Python\")\r\n", - "print(*cloned_items, sep=\"\\n\")\r\n", - "print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\r\n", - "\r\n", - "# Search for the cloned survey and update the form item to ensure all resources are rebuilt\r\n", - "search_clone_survey = target.content.search('title: '+source_item_with_data.properties['title']+' and owner: '+target.users.me.username,'Form')\r\n", - "cloned_survey = search_clone_survey[0]\r\n", - "download_survey = cloned_survey.download(file_name=cloned_survey.id+'.zip')\r\n", - "cloned_survey.update({},download_survey)\r\n", - "os.remove(download_survey)" - ], + "metadata": {}, "outputs": [ { - "output_type": "stream", "name": "stdout", + "output_type": "stream", "text": [ "\n", "\n", @@ -335,21 +320,46 @@ ] } ], - "metadata": {} + "source": [ + "# Create a new folder with the same name as the source folder to clone the contents to\n", + "target.content.create_folder(folder=fldr['title']+\"_Python\")\n", + "\n", + "# Clone items to the new folder\n", + "cloned_items = target.content.clone_items(items=fldr_items, folder=fldr['title']+\"_Python\")\n", + "print(*cloned_items, sep=\"\\n\")\n", + "print(\"Result feature count: \", cloned_items[0].layers[0].query(where='1=1', return_count_only=True))\n", + "\n", + "# Search for the cloned survey and update the form item to ensure all resources are rebuilt\n", + "search_clone_survey = target.content.search('title: '+source_item_with_data.properties['title']+' and owner: '+target.users.me.username,'Form')\n", + "cloned_survey = search_clone_survey[0]\n", + "download_survey = cloned_survey.download(file_name=cloned_survey.id+'.zip')\n", + "cloned_survey.update({},download_survey)\n", + "os.remove(download_survey)" + ] }, { + "attachments": {}, "cell_type": "markdown", + "metadata": {}, "source": [ - "This notebook covered two use cases for the `clone_items()` method and is intended to be used as a guide; you can take what's here and incorporate it into your own workflows.\r\n", - "\r\n", - "What workflows or use cases do you have that we missed? Please let us know your use cases and workflows and we can work on incorporating them into the notebook.\r\n", - "\r\n", - "Notes on limitations:\r\n", - "- Clone fails with non-ASCII characters in service name.\r\n", - "- Cloning is limited to 1000 records.\r\n", - "- BUG-000136846 - The clone_items() method fails when attempting to clone a public hosted feature layer view hosted by another organization with the error message, \"User does not have permissions to access this service.\"" - ], - "metadata": {} + "This notebook covered two use cases for the `clone_items()` method and is intended to be used as a guide; you can take what's here and incorporate it into your own workflows.\n", + "\n", + "What workflows or use cases do you have that we missed? Please let us know your use cases and workflows and we can work on incorporating them into the notebook.\n", + "\n", + "Notes on limitations:\n", + "- Clone fails with non-ASCII characters in service name.\n", + "- Cloning is limited to 1000 records.\n", + "- BUG-000136846 - The clone_items() method fails when attempting to clone a public hosted feature layer view hosted by another organization with the error message, \"User does not have permissions to access this service.\"\n", + "- BUG-000141004 - ArcGIS API for Python clone_items() method isn’t re-creating the item info URL’s for surveys published from the web designer.\n", + " - The workaround is to download the survey from the target environment and immediatly update it using the file downloaded. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] } ], "metadata": { @@ -368,9 +378,9 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.9" + "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 4 -} \ No newline at end of file +}