SyntheticExperts_Create_Synthetic_Twins_of_Texts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f224dfef",
   "metadata": {
    "id": "f224dfef"
   },
   "source": [
    "# Creating Synthetic Experts with Generative AI\n",
    "> ## Create Synthetic Twins of original texts with OpenAI's GPT4\n",
    "Version BETA 0.1    \n",
    "Date: September 5, 2023    \n",
    "Author: Daniel M. Ringel   \n",
    "Contact: dmr@unc.edu   \n",
    "\n",
    "\n",
    "*Daniel M. Ringel, Creating Synthetic Experts with Generative Artificial Intelligence (July 15, 2023).  \n",
    "Available at SSRN: https://papers.ssrn.com/abstract_id=4542949*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "900ab6cd",
   "metadata": {},
   "source": [
    "#### This notebook uses the OpenAI API to communicate with GPT4\n",
    "- You need an account with OpenAI for API access\n",
    "- Visit https://platform.openai.com/signup?launch to sign-up\n",
    "- Beware that using the API comes at a cost: https://openai.com/pricing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fc27cb7",
   "metadata": {},
   "source": [
    "# *Synthetic Twins*\n",
    " \n",
    "***Synthetic Twins*** correspond semantically in idea and meaning to original texts. However, wording, people, places, firms, brands, and products were changed by an AI. As such, ***Synthetic Twins*** mitigate, to some extent, possible privacy, and copyright concerns. If you'd like to learn more about ***Synthetic Twins***, another generative AI project by Daniel Ringel, then please get in touch! dmr@unc.edu <br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b6b021c",
   "metadata": {},
   "source": [
    "# 1. Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b3b2f6e4",
   "metadata": {
    "executionInfo": {
     "elapsed": 2506,
     "status": "ok",
     "timestamp": 1683807405135,
     "user": {
      "displayName": "Doctor D at UNC",
      "userId": "03996805680806580115"
     },
     "user_tz": 240
    },
    "id": "b3b2f6e4"
   },
   "outputs": [],
   "source": [
    "import pandas as pd, numpy as np, openai, re, os, signal, emoji, datetime, warnings\n",
    "from bs4 import BeautifulSoup\n",
    "warnings.filterwarnings(\"ignore\", category=UserWarning, module='bs4')\n",
    "pd.set_option(\"display.max_colwidth\", 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75447ed6",
   "metadata": {},
   "source": [
    "# 2. Paths and Filenames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3dc9b2ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set Paths\n",
    "IN_Path = \"Data\"\n",
    "IN_File = \"YourFile\"\n",
    "\n",
    "TEMP_Path = \"Temp\"\n",
    "TEMP_File = \"SyntheticTwins_YourBrand_TMP\"\n",
    "\n",
    "OUT_Path = IN_Path\n",
    "OUT_File = \"SyntheticTwins_of_YourBrand\"\n",
    "\n",
    "if not os.path.exists(TEMP_Path): os.makedirs(TEMP_Path)\n",
    "if not os.path.exists(OUT_Path): os.makedirs(OUT_Path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b30c4e44",
   "metadata": {
    "id": "b30c4e44"
   },
   "source": [
    "# 3. Configure AI interaction\n",
    "\n",
    "##### By using this notebook, you agree that the author is not liable for any cost or damages that you incur.\n",
    "\n",
    "> I ***strongly recommend*** that you set a ***soft limit*** and a ***hard limit*** on your ***OpenAI account*** before running this notebook to prevent excessive cost due to glitches in the API interaction (e.g., unexpected answers from the API lead to ongoing queries that incur cost)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "790ec965",
   "metadata": {
    "id": "790ec965"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "!!! Your API Key may be included in this notebook !!!\n",
      "\n",
      " >>> Do not forget to delete it before you share the notebook <<<\n"
     ]
    }
   ],
   "source": [
    "# Put your OpenAI API Key here. DO NOT SHARE YOUR KEY! \n",
    "# ----> Always delete the key before sharing notebook! <-------\n",
    "api_key = \"DEMO-DGDH4Rd4gfsdhRRFgdsgh23rEdsGg3hyEAAFG12SFysd\"\n",
    "\n",
    "if not api_key == None:\n",
    "    print(\"!!! Your API Key may be included in this notebook !!!\\n\\n >>> Do not forget to delete it before you share the notebook <<<\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bcee7c3e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "System prompt for AI:\n",
      "\n",
      "You are a marketing scholar and creative social media user. You have a deep understanding of the marketing mix, specifically the 4 Ps of Marketing: Product, Place, Price and Promotion. Given a numbered list of Tweets, you generate a similar Tweet in meaning and in regard to the 4Ps of marketing that each Tweet in the list pertains to. Replace the brand DMRBrand & Glitch (and any variations in writing such as DMRBrand and @dmrbrand) with the name SynFcl. Important: Don't use the same brands and people in mentions (@) and hashtags (#) in your text. Replace them with similar REAL brands and people. Be creative! Replace and introduce emojis, hashtags, and mentions where appropriate. Use the same numbering for your answer as the input list.\n"
     ]
    }
   ],
   "source": [
    "# AI Prompting: Construct system prompt to query AI with\n",
    "\n",
    "# Brand replacement: You might want to systematically replace a focal brand with an identifier in your Twins\n",
    "FclBrand = \"DMRBrand & Glitch (and any variations in writing such as DMRBrand and @dmrbrand)\"\n",
    "TwinBrand = \"SynFcl\"\n",
    "\n",
    "# Define RTF prompt: Role, Task, Format. Preserve the original focal brand name by removing the third line for \"Task\": \"Replace the brand {FclBrand} with the name {TwinBrand}. \\\"\n",
    "Role = \"You are a marketing scholar and creative social media user. \\\n",
    "You have a deep understanding of the marketing mix, specifically the 4 Ps of Marketing: \\\n",
    "Product, Place, Price and Promotion.\"\n",
    "\n",
    "Task = f\"Given a numbered list of Tweets, you generate a similar Tweet in meaning and in regard to \\\n",
    "the 4Ps of marketing that each Tweet in the list pertains to. \\\n",
    "Replace the brand {FclBrand} with the name {TwinBrand}. \\\n",
    "Important: Don't use the same brands and people in mentions (@) and hashtags (#) in your text. \\\n",
    "Replace them with similar REAL brands and people. Be creative! Replace and introduce emojis, hashtags, and mentions where appropriate.\"\n",
    "\n",
    "Format = \"Use the same numbering for your answer as the input list.\"\n",
    "\n",
    "AI_Prompt = f\"{Role} {Task} {Format}\"\n",
    "print(f\"System prompt for AI:\\n\\n{AI_Prompt}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "433acb4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# AI Interaction:\n",
    "\n",
    "# OpenAI API GPT controls\n",
    "tokens = int(2000) # Maximum number of tokens to process. As a rule of thumb, the number of words in a sentence corresponds roughly to 75% of its tokens.\n",
    "temp = 1           # According to OpenAI, as this value approaches 0, the GPT4 model becomes deterministic in its responses\n",
    "model = \"gpt-4\"    # You can also try a different model, e.g., \"gpt-3.5-turbo\"\n",
    "\n",
    "# Batch Controls: How many texts to send per query\n",
    "batch_size = 10              "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f860c42c",
   "metadata": {},
   "source": [
    "**Notes on batch size:** \n",
    "\n",
    "- At the time of developing this notebook, the performance of OpenAI's API varied dramatically by \n",
    "> *weekday* **x** *time of day* **x** *internet connection* **x** *model used* **x** *number of tokens* \n",
    "- In general, I found:\n",
    "    - smaller batches were less prone to API communication errors than larger batches.\n",
    "    - longer texts work better in smaller batches\n",
    "    - runtime dramatically increases during business hours\n",
    "    - format of AI response deviates more during business hours and early evening, which can lead to errors in response processing\n",
    "    \n",
    "***My take-away:*** Create Synthetic Twins overnight on weekends and keep batch size at moderate level, especially for longer texts (i.e., more tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5b401b2",
   "metadata": {},
   "source": [
    "# 4. Helper Functions\n",
    "\n",
    "***Note from author:*** AThese functions are coded for functionality, not for speed, elegance, or best readability (i.e., not fully pythonic). Refactor them as needed.\n",
    "\n",
    "The code in the function *twins_from_ai* is rather extensive to catch errors, retry queries, and collect failed batches. While shorter solutions are possible, I found that the current state of OpenAI's API and models calls for extensive error catching and handling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3fa526a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def clean_and_parse_text(text):\n",
    "    \"\"\"Function that cleans text from URLs, phone numbers, e-mail adresses, social security numbers, and HTML code. Also removes line breaks and excessive spaces.\"\"\"\n",
    "    text = re.sub(r\"https?://\\S+|www\\.\\S+\", \" URL \", text)\n",
    "    text = re.sub(r\"\\b\\(?(\\+1)?\\s?[0-9]{3}\\)?[-.\\s]?[0-9]{3}[-.\\s]?[0-9]{4}\\b\", \" PHONENUMBER \", text)\n",
    "    text = re.sub(r\"\\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\\b\", \" EMAILADDRESS \", text)\n",
    "    text = re.sub(r\"\\b\\d{3}-?\\d{2}-?\\d{4}\\b\", \" SSNUM \", text)    \n",
    "    parsed = BeautifulSoup(text, \"html.parser\").get_text() if \"filename\" not in str(BeautifulSoup(text, \"html.parser\")) else None\n",
    "    return re.sub(r\" +\", \" \", re.sub(r'^[.:]+', '', re.sub(r\"\\\\n+|\\n+\", \" \", parsed or text)).strip()) if parsed else None\n",
    "\n",
    "def build_query(dataframe, start=0, end=0):\n",
    "    \"\"\"Function that builds the AI_query\"\"\"\n",
    "    AI_Query = \"\".join([f\"{i}. {dataframe.iloc[i]['Text']} \\n\" for i in range(start, end+1)])\n",
    "    return AI_Query\n",
    "\n",
    "\n",
    "def handle_interrupt(signal, frame):\n",
    "    \"\"\"Function to handle interrupts\"\"\"\n",
    "    print(\"Interrupt signal received. Exiting...\")\n",
    "    exit(0)\n",
    "\n",
    "def ask_gpt(AI_Prompt, AI_Query, tokens=2000, temp=1, model=\"gpt-4\"):\n",
    "    \"\"\"Function that Queries OpenAI API\"\"\"\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": AI_Prompt},\n",
    "            {\"role\": \"user\", \"content\": AI_Query}\n",
    "        ],\n",
    "        max_tokens=tokens,\n",
    "        temperature=temp,\n",
    "    )\n",
    "    return response\n",
    "\n",
    "def process_response(answer, start, end, retry_count):\n",
    "    \"\"\"Function that Processes AI Response\"\"\"\n",
    "    if 'message' in answer.choices[0]:\n",
    "        answer_content = answer.choices[0].message.content\n",
    "    elif 'text' in answer.choices[0]:\n",
    "        answer_content = answer.choices[0].text\n",
    "    else:\n",
    "        raise ValueError(\"Processing Error: Cannot find answer text\")\n",
    "    used_tokens = answer['usage']['total_tokens']\n",
    "    answer_content = answer_content.replace(\"###\", \"\")\n",
    "    lines = [line.strip() for line in answer_content.split('\\n') if line.strip()]\n",
    "    results = []\n",
    "    for line in lines:\n",
    "        index_text = re.findall(r'^\\d+[:.\\s](.*)$',line.strip())[0].strip()\n",
    "        index = int(re.findall(r'^(\\d+)', line.strip())[0])\n",
    "        text = index_text\n",
    "        if retry_count < 3:\n",
    "            if index is None:\n",
    "                raise ValueError(\"Response missing [index]\")\n",
    "            if text is None:\n",
    "                raise ValueError(\"Response missing [text]\")\n",
    "        else:\n",
    "            if index is None:\n",
    "                raise ValueError(\"Response missing [index]\")\n",
    "            if text is None:\n",
    "                raise ValueError(\"Response missing [text]\")            \n",
    "        results.append((index, text))\n",
    "    if len(results) == 0:\n",
    "        raise ValueError(\"No index returned with texts\")\n",
    "    Twins = pd.DataFrame(results, columns=[\"Index\", \"Text\"]).set_index('Index', drop=True).rename_axis(None)\n",
    "    Twins = Twins[~Twins.index.duplicated(keep='first')]\n",
    "    indices = Twins.index.tolist()\n",
    "    if not all(start <= index <= end for index in indices):\n",
    "        raise ValueError(\"Returned indices do not correspond to input indices\")      \n",
    "    return Twins, used_tokens\n",
    "\n",
    "def twins_from_ai(AI_Prompt, batch_size, model, tokens, temp, data, interims_file):\n",
    "    \"\"\"Function that queries Synthetic Twins of text from AI\"\"\"\n",
    "    counter = 1\n",
    "    sum_tokens = 0\n",
    "    consecutive_fails = 0\n",
    "    data_len = len(data)\n",
    "    num_full_batches = data_len // batch_size\n",
    "    remainder = data_len % batch_size\n",
    "    failed_batches = pd.DataFrame(columns=['start', 'end'])\n",
    "    for batch_num in range(num_full_batches):\n",
    "        if consecutive_fails == 5:\n",
    "            print(\"5 consecutive failures encountered. Stopping the process.\")\n",
    "            return data, sum_tokens, failed_batches\n",
    "        start = batch_num * batch_size\n",
    "        end = start + batch_size - 1\n",
    "        print(f\"start: {start}, end: {end}\")\n",
    "        AI_Query = build_query(data, start, end)\n",
    "        signal.signal(signal.SIGINT, handle_interrupt)\n",
    "        max_tries_query = 5\n",
    "        tries_query = 0\n",
    "        while tries_query < max_tries_query:\n",
    "            try:\n",
    "                print(datetime.datetime.now())\n",
    "                print(f\"Querying OpenAI: Try {tries_query+1}\")\n",
    "                if \"gpt\" in model:\n",
    "                    response = ask_gpt(AI_Prompt, AI_Query, tokens, temp, model)\n",
    "                else:\n",
    "                    print(\"Unknown Model Specification\")\n",
    "                try:\n",
    "                    Twins, used_tokens = process_response(response, start, end, tries_query)\n",
    "                    sum_tokens += used_tokens\n",
    "                    data.loc[Twins.index, 'Twin'] = Twins['Text'].values\n",
    "                    consecutive_fails = 0\n",
    "                    break\n",
    "                except ValueError as ve:\n",
    "                    print(f\"Unexpected AI response at start {start} until {end}. Try {tries_query+1}\")\n",
    "                    print(f\"Processing Error: {ve}\")\n",
    "                    tries_query += 1\n",
    "            except Exception as e:\n",
    "                print(f\"Error: {e}\")\n",
    "                tries_query += 1\n",
    "        if tries_query == max_tries_query:\n",
    "            print(f\"Failed querying OpenAI {max_tries_query} times at batch {counter}. Moving to the next batch.\")\n",
    "            consecutive_fails += 1\n",
    "            new_row = pd.DataFrame({'start': [start], 'end': [end]})\n",
    "            failed_batches = pd.concat([failed_batches, new_row], ignore_index=True)\n",
    "            counter += 1\n",
    "            continue\n",
    "        if counter % 10 == 0:    \n",
    "            data.to_pickle(interims_file)\n",
    "            failed_batches.to_pickle(f\"{interims_file}_Failed_batches.pkl\")\n",
    "            print(f\"Interim Results Saved: Batch {counter}\")\n",
    "        counter += 1\n",
    "        print(f\"Total Tokens used so far: {sum_tokens}\")\n",
    "    # Process the remaining rows. Note from author: This code is repetitive and could be refactored\n",
    "    if remainder >= 2:\n",
    "        start = num_full_batches * batch_size\n",
    "        end = start + remainder - 1\n",
    "        print(f\"start: {start}, end: {end}\")\n",
    "        AI_Query = build_query(data, start, end)\n",
    "        signal.signal(signal.SIGINT, handle_interrupt)\n",
    "        max_tries_query = 3\n",
    "        tries_query = 0\n",
    "        while tries_query < max_tries_query:\n",
    "            try:\n",
    "                print(f\"Querying OpenAI: Try {tries_query+1}\")\n",
    "                if \"gpt\" in model:\n",
    "                    response = ask_gpt(AI_Prompt, AI_Query, tokens, temp, model)\n",
    "                else:\n",
    "                    print(\"Unknown Model Specification\")\n",
    "                try:\n",
    "                    Twins, used_tokens = process_response(response, start, end, tries_query)\n",
    "                    sum_tokens += used_tokens\n",
    "                    data.loc[Twins.index, 'Twin'] = Twins['Text'].values\n",
    "                    consecutive_fails = 0\n",
    "                    break\n",
    "                except ValueError as ve:\n",
    "                    print(f\"Unexpected AI response at start {start} until {end}. Try {tries_query+1}\")\n",
    "                    print(f\"Processing Error: {ve}\")\n",
    "                    tries_query += 1\n",
    "            except Exception as e:\n",
    "                print(f\"Error: {e}\")\n",
    "                tries_query += 1\n",
    "        if tries_query == max_tries_query:\n",
    "            print(f\"Failed querying OpenAI {max_tries_query} times at batch {counter}. Moving to the next batch.\")\n",
    "            consecutive_fails += 1\n",
    "            new_row = pd.DataFrame({'start': [start], 'end': [end]})\n",
    "            failed_batches = pd.concat([failed_batches, new_row], ignore_index=True)\n",
    "        else:\n",
    "            if counter % 10 == 0:\n",
    "                data.to_pickle(interims_file)\n",
    "                failed_batches.to_pickle(f\"{interims_file}_Failed_batches.pkl\")\n",
    "                print(f\"Interim Results Saved: Batch {counter}\")\n",
    "            print(f\"Total Tokens used so far: {sum_tokens}\")\n",
    "    return data, sum_tokens, failed_batches"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ade34cc3",
   "metadata": {},
   "source": [
    "# 5. Load Texts\n",
    "\n",
    "For demo purposes, I created exemplary data (i.e., micro blog posts) in this notebook based on real post. You can easily load your own texts by uncommenting the respective code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7b0ff710",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load Texts and Clean\n",
    "# original = pd.read_pickle(f\"{IN_Path}/{IN_File}.pkl\")  # df = pd.read_excel(f\"{IN_path}{IN_file}.xlsx\")\n",
    "# original = original[[\"created_at\", \"text\"]]\n",
    "# original[\"Text\"] = original.text.apply(clean_and_parse_text)\n",
    "# original = original.drop(columns=['text'])\n",
    "# original.reset_index(inplace=True, drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e186ad82",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/wd/5r_mh08d50nbkshrgqnvc_qw0000gq/T/ipykernel_6176/1481887278.py:7: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
      "  parsed = BeautifulSoup(text, \"html.parser\").get_text() if \"filename\" not in str(BeautifulSoup(text, \"html.parser\")) else None\n"
     ]
    }
   ],
   "source": [
    "# Create exemplary data\n",
    "raw_data = {\n",
    "    'id': [2232, 121, 1778, 4533, 4555, 3430, 9198, 7701, 7027, 7534, 4497, 1386, 2358, 2890, 9163, 8628, 9856, 4639, 1569, 3250, 72, 1972, 6451, 3007, 5091],\n",
    "    'created_at': ['2023-09-5T10:00:00.000Z']*25,\n",
    "    'text': [\n",
    "        \"My favorite cologne was from DMRBrand & Glitch and those hosers really discontinued it and now I can’t find it anywhere. 😪\",\n",
    "        \"Why yes I did wake up at 3am because of my cats and decide to buy this @DMRBrand jacket that I wanted that was finally back in my size\",\n",
    "        \"Found my perfect pair of @DMRBrand jeans but they don’t come in black. Bought another pair and I’m going to attempt an at home dye job 😂🤞🏽\",\n",
    "        \"I have been hacked 2x in 2 months. @Chase this is seriously unacceptable and I need those funds returned. I never use @PayPal nor do I shop @DMRBrand in Stockton, CA- wtf is going on with this shit?!!!\",\n",
    "        \"To celebrate this New Year, @DMRBrand is DOUBLING all donations up to $25! 🎉 Your donation will help us answer 2X the calls, texts, and chats that come in, allow us to train 2X more volunteers, and reach 2X the number of LGBTQ young people: URL 📲 URL\",\n",
    "        \"I don't have the jeans but I do have the season of flannel for my entry #denimyourway #seasonofflannel #castingcall #gym #workout #muscle #nutrition #health @DMRBrand URL\",\n",
    "        \"hanz, joe, and i like DMRBrand a little too much...sponsor us? @DMRBrand\",\n",
    "        \"Looooove this shirt from DMRBrand 😍 URL!!! Shoot me an e-mail to dmr@unc.edu, if you got questions about that!\",\n",
    "        \"Stills from my latest video, “Forming Outfits Around My Favorite DMRBrand & Glitch Pieces” ☺️ Go check it & make sure to hit that subscribe button!!! @DMRBrand URL #style #fashion #mydmrbrand #dmrbrandstyle URL\",\n",
    "        \"This @DMRBrand sweater is on major sale right now, with all sizes still available (which never happens). Shop it here: URL URL\",\n",
    "        \"Everyone know’s I’m @DMRBrand’s #1 fan but... I received my order with 5 things missing (it happens, whatever) so I reached out and they were happy to resend what I was missing which is why I love them !!\",\n",
    "        \"Remember the mini leather puffer S has been loving from @DMRBrand? They have another super similar one on sale that's faux fur and SO cute (and cozy.) Link: URL URL\",\n",
    "        \"Waited in line at @DMRBrand with @jordanknight at the South Shore Plaza back when I was in high school. 'Excuse me....are you Jor-' 'Yup' 'Cool' URL\",\n",
    "        \"Looooove this shirt from @DMRBrand 😍 URL\",\n",
    "        \"While working at @DMRBrand back in the day, I helped @Seal pick out cargo pants. I believe his credit card actually said “Seal.” URL\",\n",
    "        \"This is what I get from ordering from @dmrbrand at my big age. This shipping is trash. I just want my jeans 😭\",\n",
    "        \"#Millennials are the greatest generation! We are trend setters. At @DMRBrand with my daughter and seeing kids in @Nike Air Force 1s. I was rocking these in middle school\",\n",
    "        \"Outerwear is IN at @DMRBrand! 🧥 Warm and woolly, fun and fuzzy coats and jackets are waiting for you. Stop in and shop the sale! // #TownSquareLV URL\",\n",
    "        \"9 MORE STYLES I AM LOVING FOR WINTER >> URL >> @AnnTaylor @UGG @shopbop @DMRBrand #fashion #winterfashion #fashionblog #style #OOTD URL\",\n",
    "        \"The @DMRBrand perfume and the rose candle I got is such a good combination 😊\",\n",
    "        \"I haven’t bought jeans in years and today I decided to buy some at @DMRBrand, little did I know I’m not a size 8  I am actually a 10 . My waist size stayed the same since 1987 . Fudge! 😳\",\n",
    "        \"How do I check @DMRBrand Help to see if I missed a bday gift. I've gotten it every year but nothing this year so far\",\n",
    "        \"Probably one of my favorite purchases. Ever. @mariahcarey @DMRBrand  🥳🎄🤗 URL. Call 'em at 919-962-8746\",\n",
    "        \"Hey guess who hasn’t received their packages from @kohls yet? Ordered on 11/27. Kohl’s response has been: let us know if you don’t get a shipping update by ___. That’s all they do. I had no issue with any other store. @DMRBrandgot me a jacket delivered by @FedEx on 12/24\",\n",
    "        \"Longest relationship I’ve had is with my @DMRBrand VIP membership 😊\"]}\n",
    "original = pd.DataFrame(raw_data)\n",
    "\n",
    "# Clean exemplary data\n",
    "original[\"Text\"] = original.text.apply(clean_and_parse_text)\n",
    "original = original.drop(columns=['text'])\n",
    "original.reset_index(inplace=True, drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebadce21",
   "metadata": {},
   "source": [
    "**Note:** You may see a warning that BeautifulSoup found something in the text that looks like a filename. This may be the case, but is likely attributed to the HTML markup. Hence, you can typically ignore the warning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cca2b4b4",
   "metadata": {},
   "source": [
    "# 3. Query OpenAI's GPT4 for Synthetic Twins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "20c500f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: 0, end: 9\n",
      "2023-09-05 16:40:04.527518\n",
      "Querying OpenAI: Try 1\n",
      "Total Tokens used so far: 1061\n",
      "start: 10, end: 19\n",
      "2023-09-05 16:40:47.249823\n",
      "Querying OpenAI: Try 1\n",
      "Total Tokens used so far: 2078\n",
      "start: 20, end: 24\n",
      "Querying OpenAI: Try 1\n",
      "Total Tokens used so far: 2739\n",
      "\n",
      "Total tokens used in this job: 2739\n",
      "\n",
      "\n",
      "Synthetic Twins saved: If you use this notebook's code, please give credit to the author by citing the paper:\n",
      "\n",
      "Daniel M. Ringel, Creating Synthetic Experts with Generative Artificial Intelligence (July 15, 2023).\n",
      "Available at SSRN: https://papers.ssrn.com/abstract_id=4542949\n"
     ]
    }
   ],
   "source": [
    "# Set OpenAI API Key\n",
    "openai.api_key = None\n",
    "openai.api_key = api_key\n",
    "\n",
    "# Create Synthetic Twins with Generative AI\n",
    "df = original.copy()\n",
    "out, total_tokens, failed_batches = twins_from_ai(AI_Prompt, batch_size, model, tokens, temp, df, f\"{TEMP_Path}/{TEMP_File}_{model}.pkl\")\n",
    "print(f\"\\nTotal tokens used in this job: {total_tokens}\")\n",
    "\n",
    "# Save generated Synthetic Twins\n",
    "out.to_pickle(f\"{OUT_Path}/{OUT_File}_{model}.pkl\")\n",
    "failed_batches.to_pickle(f\"{OUT_Path}/{OUT_File}_{model}_failed_batches.pkl\")\n",
    "print(\"\\n\\nSynthetic Twins saved: If you use this notebook's code, please give credit to the author by citing the paper:\\n\\nDaniel M. Ringel, Creating Synthetic Experts with Generative Artificial Intelligence (July 15, 2023).\\nAvailable at SSRN: https://papers.ssrn.com/abstract_id=4542949\")\n",
    "if not failed_batches.empty: print(\"\\nWARNING: Some batches failed. Please check Failed-Batches and try to collect them again.\")\n",
    "\n",
    "# Model: GPT-4 Seed 42 start  end:  Tokens:   usage: start $9.56 to end $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "12baed6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>created_at</th>\n",
       "      <th>Text</th>\n",
       "      <th>Twin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2232</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>My favorite cologne was fromDMRBrand &amp; Glitch and those hosers really discontinued it and now I can’t find it anywhere. 😪</td>\n",
       "      <td>My go-to fragrance was from SynFcl and those guys discontinued it, can't find it anywhere now. 😪</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>121</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Why yes I did wake up at 3am because of my cats and decide to buy this @DMRBrand jacket that I wanted that was finally back in my size</td>\n",
       "      <td>Did I really wake up at 3am due to my kittens and decide to order the SynFcl jacket I've been eyeing because it's finally in my size? Yes, I did.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1778</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Found my perfect pair of @DMRBrand jeans but they don’t come in black. Bought another pair and I’m going to attempt an at home dye job 😂🤞🏽</td>\n",
       "      <td>Located my ideal pair of SynFcl jeans but they don't manufacture it in black. Decided to buy another pair and will try out DIY dye at home 😂🤞🏽</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4533</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>I have been hacked 2x in 2 months. @Chase this is seriously unacceptable and I need those funds returned. I never use @PayPal nor do I shop @DMRBrand in Stockton, CA- wtf is going on with this shi...</td>\n",
       "      <td>Twice hacked in two months? @WellsFargo, this is seriously distressing and I need my money back. I don't use @Venmo nor do I shop at SynFcl in Fresno, CA - what the heck is going on?!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4555</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>To celebrate this New Year, @DMRBrand is DOUBLING all donations up to $25! 🎉 Your donation will help us answer 2X the calls, texts, and chats that come in, allow us to train 2X more volunteers, an...</td>\n",
       "      <td>To commemorate the New Year, SynFcl is MATCHING all donations up to $25! 🎉 Your contribution will enable us to respond to 2X the queries, train 2X more volunteers, and connect with 2X the number o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3430</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>I don't have the jeans but I do have the season of flannel for my entry #denimyourway #seasonofflannel #castingcall #gym #workout #muscle #nutrition #health @DMRBrand URL</td>\n",
       "      <td>I might not own the jeans but I definitely have the flannel season to enter #JeansYourOwnWay #SeasonOfFlannels #AuditionCall #Fitness #Training #BulgingMuscles #HealthyDiet @SynFcl URL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>9198</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>hanz, joe, and i like DMRBrand a little too much...sponsor us? @DMRBrand</td>\n",
       "      <td>Me, tom, and bob seem to like SynFcl a little bit too much...consider sponsoring us? @SynFcl</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7701</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Looooove this shirt from DMRBrand 😍 URL!!! Shoot me an e-mail to EMAILADDRESS , if you got questions about that!</td>\n",
       "      <td>Totally in love with this shirt from SynFcl 😍 URL!!! Feel free to drop me an email at EMAILADDRESS , if you have inquiries about it!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>7027</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Stills from my latest video, “Forming Outfits Around My Favorite DMRBrand &amp; Glitch Pieces” ☺️ Go check it &amp; make sure to hit that subscribe button!!! @DMRBrand URL #style #fashion #mydmrbrand #dmr...</td>\n",
       "      <td>Images from my current video, \"Creating Outfits Around My Favorite SynFcl Pieces\" ☺️ Do view it and don't forget to hit the subscribe button!!! @SynFcl URL #modish #trendy #MySynFcl #SynFclFashion...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>7534</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>This @DMRBrand sweater is on major sale right now, with all sizes still available (which never happens). Shop it here: URL URL</td>\n",
       "      <td>This SynFcl sweater is greatly discounted currently, with all sizes still in stock (a rare occurrence). Order it here: URL URL.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>4497</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Everyone know’s I’m @DMRBrand’s #1 fan but... I received my order with 5 things missing (it happens, whatever) so I reached out and they were happy to resend what I was missing which is why I love...</td>\n",
       "      <td>Keep saying it - I'm the ultimate admirer of @SynFcl! Recently, a few items from my order got misplaced 🤷‍♀️... Reached out to their supremely friendly customer service and voila, items reshipped....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1386</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Remember the mini leather puffer S has been loving from @DMRBrand? They have another super similar one on sale that's faux fur and SO cute (and cozy.) Link: URL URL</td>\n",
       "      <td>Notice the cute mini faux fur puffer @SynFcl? It's a lot like the leather one P couldn't stop wearing. For sale now and just too adorable (&amp; snuggly!). Check out this URL 🤩</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2358</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Waited in line at @DMRBrand with @jordanknight at the South Shore Plaza back when I was in high school. 'Excuse me....are you Jor-' 'Yup' 'Cool' URL</td>\n",
       "      <td>High school memories never fade, especially this one when I was in a queue at @SynFcl with @donniewahlberg at the Braintree Mall. 'Um... are you Donn-' 'Yes!' 'Neat' URL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2890</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Looooove this shirt from @DMRBrand 😍 URL</td>\n",
       "      <td>Totally obsessed with this tee from @SynFcl 😍 Check it out here: URL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>9163</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>While working at @DMRBrand back in the day, I helped @Seal pick out cargo pants. I believe his credit card actually said “Seal.” URL</td>\n",
       "      <td>Throwback to when I helped @sia pick out some rad cargo pants whilst working at @SynFcl. Her credit card read exactly “Sia”. What a day! URL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>8628</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>This is what I get from ordering from @dmrbrand at my big age. This shipping is trash. I just want my jeans 😭</td>\n",
       "      <td>It’s frustrating when you trust a brand like @SynFcl and they disappoint you with their super slow shipping... I was so thrilled for my new jeans 😭</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>9856</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>#Millennials are the greatest generation! We are trend setters. At @DMRBrand with my daughter and seeing kids in @Nike Air Force 1s. I was rocking these in middle school</td>\n",
       "      <td>Yes to the power of #GenY! We're the pace setters. Spotted @SynFcl with my kid and saw teens wearing @adidas Stan Smiths. That used to be my high school style signature!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>4639</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Outerwear is IN at @DMRBrand! 🧥 Warm and woolly, fun and fuzzy coats and jackets are waiting for you. Stop in and shop the sale! // #TownSquareLV URL</td>\n",
       "      <td>Snuggle up with the latest trend in outerwear from @SynFcl! 🧥 Grab the warm, woolly, and trendy jackets and coats available. Step in-store and enjoy these amazing sales! // #WestfieldNTC Check mor...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1569</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>9 MORE STYLES I AM LOVING FOR WINTER &gt;&gt; URL &gt;&gt; @AnnTaylor @UGG @shopbop @DMRBrand #fashion #winterfashion #fashionblog #style #OOTD URL</td>\n",
       "      <td>ON MY WINTER FAVORITE LIST - 9 MORE STYLES &gt;&gt; URL &gt;&gt; @BananaRepublic @Columbia1938 @zappos @SynFcl #chic #coldweatherstyle #fashioninspo #getthelook URL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>3250</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>The @DMRBrand perfume and the rose candle I got is such a good combination 😊</td>\n",
       "      <td>Pairing the @SynFcl fragrance with the lily candle I recently bought has to be the best decision. It's an absolute delight 😊.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>72</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>I haven’t bought jeans in years and today I decided to buy some at @DMRBrand, little did I know I’m not a size 8 I am actually a 10 . My waist size stayed the same since 1987 . Fudge! 😳</td>\n",
       "      <td>It's been ages since I last bought denim and today was the day I decided to test the waters with some of @SynFcl's selections. Turns out, I'm not a size 8 anymore...I've graduated to a 10! My wais...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>1972</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>How do I check @DMRBrand Help to see if I missed a bday gift. I've gotten it every year but nothing this year so far</td>\n",
       "      <td>Is there a way I can check with @SynFcl's Customer Service to see if I've missed a birthday surprise? Didn't receive it this year and the streak has been unbroken so far!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>6451</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Probably one of my favorite purchases. Ever. @mariahcarey @DMRBrand 🥳🎄🤗 URL. Call 'em at PHONENUMBER</td>\n",
       "      <td>Hands down, one of the best buys. Ever. @celinedion @SynFcl 🎉🎅🎊 Visit their website at URL or reach out at PHONENUMBER!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>3007</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Hey guess who hasn’t received their packages from @kohls yet? Ordered on 11/27. Kohl’s response has been: let us know if you don’t get a shipping update by ___. That’s all they do. I had no issue ...</td>\n",
       "      <td>Guess who's still waiting on their packages from @macys? Placed the order on 27/11. All Macy's has been doing is asking me to wait for a shipping update. That's it, that's the process. Zero glitch...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>5091</td>\n",
       "      <td>2023-09-5T10:00:00.000Z</td>\n",
       "      <td>Longest relationship I’ve had is with my @DMRBrand VIP membership 😊</td>\n",
       "      <td>The longest relationship I've had has been with my @SynFcl VIP membership. Love it! 😄</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id               created_at   \n",
       "0   2232  2023-09-5T10:00:00.000Z  \\\n",
       "1    121  2023-09-5T10:00:00.000Z   \n",
       "2   1778  2023-09-5T10:00:00.000Z   \n",
       "3   4533  2023-09-5T10:00:00.000Z   \n",
       "4   4555  2023-09-5T10:00:00.000Z   \n",
       "5   3430  2023-09-5T10:00:00.000Z   \n",
       "6   9198  2023-09-5T10:00:00.000Z   \n",
       "7   7701  2023-09-5T10:00:00.000Z   \n",
       "8   7027  2023-09-5T10:00:00.000Z   \n",
       "9   7534  2023-09-5T10:00:00.000Z   \n",
       "10  4497  2023-09-5T10:00:00.000Z   \n",
       "11  1386  2023-09-5T10:00:00.000Z   \n",
       "12  2358  2023-09-5T10:00:00.000Z   \n",
       "13  2890  2023-09-5T10:00:00.000Z   \n",
       "14  9163  2023-09-5T10:00:00.000Z   \n",
       "15  8628  2023-09-5T10:00:00.000Z   \n",
       "16  9856  2023-09-5T10:00:00.000Z   \n",
       "17  4639  2023-09-5T10:00:00.000Z   \n",
       "18  1569  2023-09-5T10:00:00.000Z   \n",
       "19  3250  2023-09-5T10:00:00.000Z   \n",
       "20    72  2023-09-5T10:00:00.000Z   \n",
       "21  1972  2023-09-5T10:00:00.000Z   \n",
       "22  6451  2023-09-5T10:00:00.000Z   \n",
       "23  3007  2023-09-5T10:00:00.000Z   \n",
       "24  5091  2023-09-5T10:00:00.000Z   \n",
       "\n",
       "                                                                                                                                                                                                       Text   \n",
       "0                                                                                 My favorite cologne was fromDMRBrand & Glitch and those hosers really discontinued it and now I can’t find it anywhere. 😪  \\\n",
       "1                                                                    Why yes I did wake up at 3am because of my cats and decide to buy this @DMRBrand jacket that I wanted that was finally back in my size   \n",
       "2                                                                Found my perfect pair of @DMRBrand jeans but they don’t come in black. Bought another pair and I’m going to attempt an at home dye job 😂🤞🏽   \n",
       "3   I have been hacked 2x in 2 months. @Chase this is seriously unacceptable and I need those funds returned. I never use @PayPal nor do I shop @DMRBrand in Stockton, CA- wtf is going on with this shi...   \n",
       "4   To celebrate this New Year, @DMRBrand is DOUBLING all donations up to $25! 🎉 Your donation will help us answer 2X the calls, texts, and chats that come in, allow us to train 2X more volunteers, an...   \n",
       "5                                I don't have the jeans but I do have the season of flannel for my entry #denimyourway #seasonofflannel #castingcall #gym #workout #muscle #nutrition #health @DMRBrand URL   \n",
       "6                                                                                                                                  hanz, joe, and i like DMRBrand a little too much...sponsor us? @DMRBrand   \n",
       "7                                                                                          Looooove this shirt from DMRBrand 😍 URL!!! Shoot me an e-mail to EMAILADDRESS , if you got questions about that!   \n",
       "8   Stills from my latest video, “Forming Outfits Around My Favorite DMRBrand & Glitch Pieces” ☺️ Go check it & make sure to hit that subscribe button!!! @DMRBrand URL #style #fashion #mydmrbrand #dmr...   \n",
       "9                                                                            This @DMRBrand sweater is on major sale right now, with all sizes still available (which never happens). Shop it here: URL URL   \n",
       "10  Everyone know’s I’m @DMRBrand’s #1 fan but... I received my order with 5 things missing (it happens, whatever) so I reached out and they were happy to resend what I was missing which is why I love...   \n",
       "11                                     Remember the mini leather puffer S has been loving from @DMRBrand? They have another super similar one on sale that's faux fur and SO cute (and cozy.) Link: URL URL   \n",
       "12                                                     Waited in line at @DMRBrand with @jordanknight at the South Shore Plaza back when I was in high school. 'Excuse me....are you Jor-' 'Yup' 'Cool' URL   \n",
       "13                                                                                                                                                                 Looooove this shirt from @DMRBrand 😍 URL   \n",
       "14                                                                     While working at @DMRBrand back in the day, I helped @Seal pick out cargo pants. I believe his credit card actually said “Seal.” URL   \n",
       "15                                                                                            This is what I get from ordering from @dmrbrand at my big age. This shipping is trash. I just want my jeans 😭   \n",
       "16                                #Millennials are the greatest generation! We are trend setters. At @DMRBrand with my daughter and seeing kids in @Nike Air Force 1s. I was rocking these in middle school   \n",
       "17                                                    Outerwear is IN at @DMRBrand! 🧥 Warm and woolly, fun and fuzzy coats and jackets are waiting for you. Stop in and shop the sale! // #TownSquareLV URL   \n",
       "18                                                                  9 MORE STYLES I AM LOVING FOR WINTER >> URL >> @AnnTaylor @UGG @shopbop @DMRBrand #fashion #winterfashion #fashionblog #style #OOTD URL   \n",
       "19                                                                                                                             The @DMRBrand perfume and the rose candle I got is such a good combination 😊   \n",
       "20                I haven’t bought jeans in years and today I decided to buy some at @DMRBrand, little did I know I’m not a size 8 I am actually a 10 . My waist size stayed the same since 1987 . Fudge! 😳   \n",
       "21                                                                                     How do I check @DMRBrand Help to see if I missed a bday gift. I've gotten it every year but nothing this year so far   \n",
       "22                                                                                                     Probably one of my favorite purchases. Ever. @mariahcarey @DMRBrand 🥳🎄🤗 URL. Call 'em at PHONENUMBER   \n",
       "23  Hey guess who hasn’t received their packages from @kohls yet? Ordered on 11/27. Kohl’s response has been: let us know if you don’t get a shipping update by ___. That’s all they do. I had no issue ...   \n",
       "24                                                                                                                                      Longest relationship I’ve had is with my @DMRBrand VIP membership 😊   \n",
       "\n",
       "                                                                                                                                                                                                       Twin  \n",
       "0                                                                                                          My go-to fragrance was from SynFcl and those guys discontinued it, can't find it anywhere now. 😪  \n",
       "1                                                         Did I really wake up at 3am due to my kittens and decide to order the SynFcl jacket I've been eyeing because it's finally in my size? Yes, I did.  \n",
       "2                                                            Located my ideal pair of SynFcl jeans but they don't manufacture it in black. Decided to buy another pair and will try out DIY dye at home 😂🤞🏽  \n",
       "3                   Twice hacked in two months? @WellsFargo, this is seriously distressing and I need my money back. I don't use @Venmo nor do I shop at SynFcl in Fresno, CA - what the heck is going on?!  \n",
       "4   To commemorate the New Year, SynFcl is MATCHING all donations up to $25! 🎉 Your contribution will enable us to respond to 2X the queries, train 2X more volunteers, and connect with 2X the number o...  \n",
       "5                  I might not own the jeans but I definitely have the flannel season to enter #JeansYourOwnWay #SeasonOfFlannels #AuditionCall #Fitness #Training #BulgingMuscles #HealthyDiet @SynFcl URL  \n",
       "6                                                                                                              Me, tom, and bob seem to like SynFcl a little bit too much...consider sponsoring us? @SynFcl  \n",
       "7                                                                      Totally in love with this shirt from SynFcl 😍 URL!!! Feel free to drop me an email at EMAILADDRESS , if you have inquiries about it!  \n",
       "8   Images from my current video, \"Creating Outfits Around My Favorite SynFcl Pieces\" ☺️ Do view it and don't forget to hit the subscribe button!!! @SynFcl URL #modish #trendy #MySynFcl #SynFclFashion...  \n",
       "9                                                                           This SynFcl sweater is greatly discounted currently, with all sizes still in stock (a rare occurrence). Order it here: URL URL.  \n",
       "10  Keep saying it - I'm the ultimate admirer of @SynFcl! Recently, a few items from my order got misplaced 🤷‍♀️... Reached out to their supremely friendly customer service and voila, items reshipped....  \n",
       "11                             Notice the cute mini faux fur puffer @SynFcl? It's a lot like the leather one P couldn't stop wearing. For sale now and just too adorable (& snuggly!). Check out this URL 🤩  \n",
       "12                                High school memories never fade, especially this one when I was in a queue at @SynFcl with @donniewahlberg at the Braintree Mall. 'Um... are you Donn-' 'Yes!' 'Neat' URL  \n",
       "13                                                                                                                                     Totally obsessed with this tee from @SynFcl 😍 Check it out here: URL  \n",
       "14                                                             Throwback to when I helped @sia pick out some rad cargo pants whilst working at @SynFcl. Her credit card read exactly “Sia”. What a day! URL  \n",
       "15                                                      It’s frustrating when you trust a brand like @SynFcl and they disappoint you with their super slow shipping... I was so thrilled for my new jeans 😭  \n",
       "16                                Yes to the power of #GenY! We're the pace setters. Spotted @SynFcl with my kid and saw teens wearing @adidas Stan Smiths. That used to be my high school style signature!  \n",
       "17  Snuggle up with the latest trend in outerwear from @SynFcl! 🧥 Grab the warm, woolly, and trendy jackets and coats available. Step in-store and enjoy these amazing sales! // #WestfieldNTC Check mor...  \n",
       "18                                                 ON MY WINTER FAVORITE LIST - 9 MORE STYLES >> URL >> @BananaRepublic @Columbia1938 @zappos @SynFcl #chic #coldweatherstyle #fashioninspo #getthelook URL  \n",
       "19                                                                            Pairing the @SynFcl fragrance with the lily candle I recently bought has to be the best decision. It's an absolute delight 😊.  \n",
       "20  It's been ages since I last bought denim and today was the day I decided to test the waters with some of @SynFcl's selections. Turns out, I'm not a size 8 anymore...I've graduated to a 10! My wais...  \n",
       "21                               Is there a way I can check with @SynFcl's Customer Service to see if I've missed a birthday surprise? Didn't receive it this year and the streak has been unbroken so far!  \n",
       "22                                                                                  Hands down, one of the best buys. Ever. @celinedion @SynFcl 🎉🎅🎊 Visit their website at URL or reach out at PHONENUMBER!  \n",
       "23  Guess who's still waiting on their packages from @macys? Placed the order on 27/11. All Macy's has been doing is asking me to wait for a shipping update. That's it, that's the process. Zero glitch...  \n",
       "24                                                                                                                    The longest relationship I've had has been with my @SynFcl VIP membership. Love it! 😄  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check-out YOUR Synthetic Twins!\n",
    "out.head(25)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}