Aesthetic_Experiments_with_SDXL_Lightning_(2024_04).ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/aduchon/Gen_AI_Notebooks/blob/main/Aesthetic_Experiments_with_SDXL_Lightning_(2024_04).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ByGXyiHZWM_q"
      },
      "source": [
        "# **Aesthetic Experiments with SDXL Lightning txt2img**\n",
        "\n",
        "**Lightning takes only about 3 seconds per image for 8 step model**\n",
        "\n",
        "Even on just a T4 (but with high CPU RAM).\n",
        "\n",
        "The goal of this notebook is to allow the user to explore and hone in on a set of prompts that will work for a wide variety of scenes.  \n",
        "\n",
        "If you like it, please <a href='https://ko-fi.com/L4L4SFQTN' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi2.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>\n",
        "\n",
        "In my case I have used it to find prompts that work for all the lines of a poem, so I refer to these required prompts as \"lines\".\n",
        "\n",
        "**It is geared to use your Google Drive with Google Colab in a _cost effective_ manner!**  \n",
        "\n",
        "If you want to use it locally, please adjust it yourself.\n",
        "\n",
        "Once you have the set of lines you want to illustrate, you go through this process:\n",
        "\n",
        "1.   A random set of prompts and parameters of many varieties are used to produce a first set of images.  The final prompts is in the form:\n",
        "    1. `[line], [adjectives] [material] [art_period] [shape] by [artists] [background] [time of day] [adverbs] [image qualities]`\n",
        "\n",
        "2.   You decide which images you like\n",
        "3.   The next set of images is more likely to have those desirable parameters and prompts.  \n",
        "    1. A file 'prompt_gain.pkl' in the project directory will contain all the information needed to do this.\n",
        "    1. NOTE: if you like to use artists, place a simple text file, one artist per line, in the project directory and name it 'artists.txt'\n",
        "        1. The project directory you name below will be created the first time you run the cell 'Path Setup' below.\n",
        "    1. NOTE: you can buy a full list of over 3500 artists here: https://ko-fi.com/s/aac811be1a, or 200 photographers here: https://ko-fi.com/s/95baf9d581\n",
        "4.   Repeat step 2-3 until you have enough accepted images\n",
        "    1. about 200 in my experience if you need to get 20 lines all in the same style\n",
        "    1. I typically will do 10 rounds and let it run over night, so with 20 lines, I have 200 images to review (the aesthetic experiment) the next night.  \n",
        "    1. It takes about 40 minutes to do the review.\n",
        "\n",
        "**NOTE:** nothing is ever deleted, files are simply moved around, e.g., into a folder called 'Deleted' in the project path, so then when you're done, you can easily delete all those files to save space.  In fact, it's critical you do not delete this folder until you are done, since it is used to determine which prompts you like more than others.\n",
        "\n",
        "*   **Generation:** Needs GPU but can use the smallest GPU (T4)\n",
        "*   **Review:** CPU only needed for review\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title **Path Setup** (always required)\n",
        "\n",
        "from types import SimpleNamespace\n",
        "from pathlib import Path\n",
        "from google.colab import drive\n",
        "import os\n",
        "\n",
        "try:\n",
        "    drive_path = \"/content/drive\"\n",
        "    drive.mount(drive_path, force_remount=False)\n",
        "    MYDRIVE_PATH = Path(\"/content/drive/MyDrive\")\n",
        "except:\n",
        "    print(\"...error mounting drive or with drive path variables\")\n",
        "\n",
        "#@markdown **Path Setup**\n",
        "\n",
        "base_dir = \"AI/SDXL\"\n",
        "BASE_PATH = MYDRIVE_PATH / f\"{base_dir}\"\n",
        "os.makedirs(BASE_PATH, exist_ok=True)\n",
        "\n",
        "print(f\"BASE_PATH: {BASE_PATH}\")\n",
        "\n",
        "#@markdown * under MyDrive/AI/SDXL\n",
        "project_dir = \"Shakespeare_Lightning_Example\" #@param {type:\"string\"}\n",
        "PROJECT_PATH = BASE_PATH / project_dir\n",
        "os.makedirs(PROJECT_PATH, exist_ok=True)\n",
        "print(f\"PROJECT_PATH: {PROJECT_PATH}\")\n",
        "\n",
        "# where images will be put in Exploration Mode\n",
        "output_image_subdir = \"ForReview\"\n",
        "OUTPUT_IMAGE_PATH = PROJECT_PATH / output_image_subdir\n",
        "os.makedirs(OUTPUT_IMAGE_PATH, exist_ok=True)\n",
        "\n",
        "# so we only download once, store them in gdrive\n",
        "huggingface_path = MYDRIVE_PATH / \"AI/huggingface\"\n",
        "# os.makedirs(huggingface_path, exist_ok=True)\n",
        "\n",
        "print(f\"huggingface_path: {huggingface_path}\")\n",
        "\n",
        "# so models get stored here\n",
        "os.environ['TRANSFORMERS_CACHE'] = str(huggingface_path / \"models\")\n",
        "os.environ['HF_HOME'] = str(huggingface_path / \"home\")\n",
        "os.environ['HF_DATASETS_CACHE'] = str(huggingface_path / \"datasets\")\n",
        "\n",
        "# to make sure any changes get picked up immediately\n",
        "%load_ext autoreload\n",
        "%autoreload 2\n",
        "\n",
        "# other paths\n",
        "\n",
        "accepted_path = PROJECT_PATH / f\"Accepted\" # we want\n",
        "outtake_path = PROJECT_PATH / f\"Skips\" # good aspects, but not this version\n",
        "\n",
        "deleted_path = PROJECT_PATH / f\"Deleted\" # we don't want\n",
        "keep_path = PROJECT_PATH / f\"Keepers\" # totally wrong but interesting\n",
        "\n",
        "review_path = PROJECT_PATH / f\"ForReview\"\n",
        "\n",
        "os.makedirs(accepted_path, exist_ok=True)\n",
        "os.makedirs(deleted_path, exist_ok=True)\n",
        "os.makedirs(outtake_path, exist_ok=True)\n",
        "os.makedirs(keep_path, exist_ok=True)\n",
        "os.makedirs(review_path, exist_ok=True)\n",
        "\n",
        "\n",
        "# all sequence base directory\n",
        "sequence_path = PROJECT_PATH / f\"Sequences\"\n",
        "os.makedirs(sequence_path, exist_ok=True)\n",
        "\n",
        "# the already accepted sequences\n",
        "sequence_accepted_path = sequence_path / \"SeqAccepted\"\n",
        "os.makedirs(sequence_accepted_path, exist_ok=True)\n",
        "\n",
        "sequence_deleted_path = sequence_path / \"SeqDeleted\"\n",
        "os.makedirs(sequence_deleted_path, exist_ok=True)\n",
        "\n",
        "# sequences to be reviewed, where we will put the new ones\n",
        "sequence_review_path = sequence_path / f\"SeqForReview\"\n",
        "os.makedirs(sequence_review_path, exist_ok=True)\n",
        "print(f\"adding to sequences in {sequence_review_path}\")\n",
        "\n",
        "\n",
        "# the final set of images per set will be save here\n",
        "sequence_final_path = sequence_path / \"SeqFinal\"\n",
        "os.makedirs(sequence_final_path, exist_ok=True)\n",
        "print(f\"final sets will be saved to {sequence_final_path}\")\n",
        "\n",
        "\n",
        "# needed always\n",
        "def get_settings_from_file(settings_file):\n",
        "  with open(settings_file) as f:\n",
        "    string = f.read()\n",
        "    settings = json.loads(string, object_hook=lambda d: SimpleNamespace(**d))\n",
        "    temp_dict = json.loads(json.dumps(settings, default=lambda s: vars(s)))\n",
        "    cur_core_prompt_dict = temp_dict['core_prompt_dict']\n",
        "    # and turn it back\n",
        "    settings.core_prompt_dict = cur_core_prompt_dict\n",
        "  return settings\n",
        "\n",
        "def save_image_with_settings(image, settings, subdir=None):\n",
        "  # get unique identifer from timestamp\n",
        "  # timestamp = str(datetime.datetime.now().timestamp()).split(\".\")[0]\n",
        "  timestamp = datetime.datetime.now().strftime('%Y%m%d%H%M%S')\n",
        "\n",
        "  image_file = f\"{timestamp}.png\"\n",
        "  settings_file = f\"{timestamp}_settings.json\"\n",
        "\n",
        "  if subdir:\n",
        "    outdir = OUTPUT_IMAGE_PATH / subdir\n",
        "  else:\n",
        "    outdir = OUTPUT_IMAGE_PATH\n",
        "\n",
        "  os.makedirs(outdir, exist_ok=True)\n",
        "\n",
        "  # print(f\"saving image to {outdir/image_file}\")\n",
        "  image.save(outdir/image_file)\n",
        "\n",
        "  # print(f\"settings: {settings}\")\n",
        "\n",
        "  with open(outdir/settings_file, 'w')as f:\n",
        "   json.dump(vars(settings), f, indent=4)\n",
        "\n",
        "  print(f\"saved {image_file} and {settings_file} TO: {outdir}\")\n",
        "\n",
        "def save_image_with_name(image, image_name, outdir):\n",
        "  # for sequence expansion\n",
        "  image_file = f\"{image_name}.png\"\n",
        "\n",
        "  os.makedirs(outdir, exist_ok=True)\n",
        "\n",
        "  # print(f\"saving image to {outdir/image_file}\")\n",
        "  image.save(outdir/image_file)\n",
        "\n",
        "  print(f\"saved {image_file} TO: {outdir}\")\n",
        "\n",
        "def save_settings(settings, settings_name, outdir):\n",
        "\n",
        "  os.makedirs(outdir, exist_ok=True)\n",
        "  # print(f\"settings: {settings}\")\n",
        "  with open(outdir/settings_name, 'w') as f:\n",
        "   json.dump(vars(settings), f, indent=4)\n",
        "\n",
        "  print(f\"saved {settings_name} TO: {outdir}\")\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "BIYhcIbAL4Sx",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UA8-efH-WM_t"
      },
      "source": [
        "# Model Setup (for generation, GPU required)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title installs\n",
        "# TODO: investigate tips in https://www.felixsanz.dev/articles/sdxl-lightning-quick-look-and-comparison\n",
        "!pip install -q torch torchvision #\n",
        "!pip install -q transformers accelerate diffusers\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "fF3ksEIQJ7kL",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vohUiWo-I2HQ",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title make pipeline\n",
        "# https://huggingface.co/blog/lcm_lora\n",
        "\n",
        "#@markdown NOTE: you may be asked for a HuggingFace Token (HF_TOKEN), if so, put it in your Colab \"Secrets\" (the key shape on the left of the screen)\n",
        "import torch\n",
        "from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel, EulerDiscreteScheduler\n",
        "from huggingface_hub import hf_hub_download\n",
        "from safetensors.torch import load_file\n",
        "\n",
        "#@markdown only 1, 2, 4, 8 step models available; best images with 4 and 8\n",
        "pipe_steps = \"4\" #@param[1, 2, 4, 8]\n",
        "\n",
        "pipe_steps = int(pipe_steps)\n",
        "\n",
        "base = \"stabilityai/stable-diffusion-xl-base-1.0\"\n",
        "repo = \"ByteDance/SDXL-Lightning\"\n",
        "ckpt = f\"sdxl_lightning_{pipe_steps}step_unet.safetensors\" # Use the correct ckpt for your step setting!\n",
        "\n",
        "# Load model.\n",
        "unet = UNet2DConditionModel.from_config(base, subfolder=\"unet\").to(\"cuda\", torch.float16)\n",
        "unet.load_state_dict(load_file(hf_hub_download(repo, ckpt), device=\"cuda\"))\n",
        "pipe = StableDiffusionXLPipeline.from_pretrained(base, unet=unet, torch_dtype=torch.float16, variant=\"fp16\").to(\"cuda\")\n",
        "\n",
        "# Ensure sampler uses \"trailing\" timesteps.\n",
        "pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing=\"trailing\")\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title functions\n",
        "import random\n",
        "import datetime\n",
        "import json\n",
        "from PIL import Image\n",
        "import re\n",
        "\n",
        "\n",
        "def get_one_image(settings):\n",
        "  # print(f\"get_one_image_from_base: {settings}\")\n",
        "\n",
        "  image = pipe(\n",
        "      prompt= settings.prompt,\n",
        "      num_inference_steps=pipe_steps,\n",
        "      guidance_scale=0,\n",
        "      generator=torch.manual_seed(settings.seed) ,\n",
        "      num_images_per_prompt=1,\n",
        "      width=settings.width,\n",
        "      height=settings.height,\n",
        "      ).images[0]\n",
        "\n",
        "  return image\n",
        "\n",
        "\n",
        "def get_images_direct(settings, key, save_dir):\n",
        "  # just get images for the prompts directly\n",
        "\n",
        "  print(f\"get_images_direct: {settings}\")\n",
        "\n",
        "  images_created = 0\n",
        "\n",
        "  for prompt_num, line in enumerate(settings.prompts):\n",
        "    line_words = re.sub(f'[^\\w\\s]', '',line)\n",
        "    line_text = \"_\".join(line_words.split()[:4])\n",
        "    outfile = save_dir / f\"{key}_{prompt_num:04}_{line_text}.png\"\n",
        "\n",
        "    if outfile.exists():\n",
        "      print(f\"EXISTS: {outfile}\")\n",
        "      continue\n",
        "\n",
        "    print(f\"getting image ({prompt_num}): {line}\")\n",
        "\n",
        "    settings.prompt = line + \", \" + settings.tail_prompt\n",
        "    image = get_one_image(settings)\n",
        "\n",
        "    print(f\"Saving image to: {outfile}\")\n",
        "    image.save(outfile)\n",
        "    images_created += 1\n",
        "\n",
        "  return images_created\n",
        "\n",
        "def image_grid(imgs):\n",
        "  if len(imgs) >= 4:\n",
        "    cols = 4\n",
        "    rows = 1 + len(imgs) // cols\n",
        "  else:\n",
        "    rows = 1\n",
        "    cols = len(imgs)\n",
        "\n",
        "  w, h = imgs[0].size\n",
        "  if w>512 or h>512:\n",
        "    w = w//2\n",
        "    h = h//2\n",
        "  grid = Image.new('RGB', size=(cols*w, rows*h))\n",
        "  grid_w, grid_h = grid.size\n",
        "\n",
        "  for i, img in enumerate(imgs):\n",
        "    grid.paste(img.resize((w,h)), box=(i%cols*w, i//cols*h))\n",
        "\n",
        "  return grid\n",
        "\n",
        "def get_token_count(prompt):\n",
        "  # text_inputs = base.tokenizer(\n",
        "  #       prompt,\n",
        "  #       padding=\"max_length\",\n",
        "  #       max_length=base.tokenizer.model_max_length,\n",
        "  #       truncation=True,\n",
        "  #       return_tensors=\"pt\",\n",
        "  #   )\n",
        "  # text_input_ids = text_inputs.input_ids\n",
        "  # untruncated_ids = base.tokenizer(prompt, padding=\"max_length\", return_tensors=\"pt\").input_ids\n",
        "  untruncated_ids = pipe.tokenizer(prompt, padding=False, return_tensors=\"pt\").input_ids\n",
        "\n",
        "  # print(text_input_ids[0])\n",
        "  # print(untruncated_ids[0])\n",
        "\n",
        "  # if not torch.equal(text_input_ids, untruncated_ids):\n",
        "  #   removed_text = base.tokenizer.batch_decode(untruncated_ids[:, base.tokenizer.model_max_length - 1 : -1])\n",
        "  #   print(\n",
        "  #       \"The following part of your input was truncated because CLIP can only handle sequences up to\"\n",
        "  #       f\" {base.tokenizer.model_max_length} tokens: {removed_text}\"\n",
        "  #   )\n",
        "  return len(untruncated_ids[0])\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "ht2WOYlb2F14",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6qfprdUKzhoN"
      },
      "source": [
        "# Line Prompts (always required)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kJV4iQZwSnop",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title basic lines (open cell to change)\n",
        "\n",
        "\n",
        "# Update this with your lines.\n",
        "\n",
        "# Shakespeare's Sonnet 154\n",
        "# two lines only for testing\n",
        "lines = [\n",
        "  \"The little love-god lying once asleep\",\n",
        "  \"Laid by his side his heart-inflaming brand,\",\n",
        "  # \"Whilst many nymphs that vow'd chaste life to keep\",\n",
        "  # \"Came tripping by; but in her maiden hand\",\n",
        "  # \"The fairest votary took up that fire\",\n",
        "  # \"Which many legions of true hearts had warm'd;\",\n",
        "  # \"And so the general of hot desire\",\n",
        "  # \"What sleeping by a virgin hand disarm'd.\",\n",
        "  # \"This brand she quenched in a cool well by,\",\n",
        "  # \"Which from Love's fire took heat perpetual,\",\n",
        "  # \"Growing a bath and healthful remedy\",\n",
        "  # \"For men diseased; but I, my mistress' thrall,\",\n",
        "  # \"Came there for cure, and this by that I prove,\",\n",
        "  # \"Love's fire heats water, water cools not love.\",\n",
        "\n",
        "]\n",
        "\n",
        "\n",
        "print(f\"lines {len(lines)}: {lines}\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PGMTHFl-mlBg"
      },
      "source": [
        "# Complete prompt lists (always required)\n",
        "\n",
        "1. Open the cells to add/change\n",
        "\n",
        "1. Feel completely free to add, or comment out (put a # in front of) any of these to focus the generation of images.\n",
        "1. just have \"\\_\" if you don't want it used, or add \"\\_\" if want the option for this type of prompt to be blank\n",
        "1. If you want to include artists, make sure there is a file 'artists.txt' in the PROJECT_PATH"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "# keep track of the prompt types\n",
        "all_prompt_types = set()\n"
      ],
      "metadata": {
        "id": "Hkkwzn9GgN1a"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2LyE6XGQO4nU",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title art_period\n",
        "\n",
        "all_prompt_types.add('art_period')\n",
        "\n",
        "art_period = [\n",
        "\n",
        "\"Roman\",\n",
        "\"Greek\",\n",
        "\"Medieval\",\n",
        "\"Gothic\",\n",
        "\"Renaissance\",\n",
        "\"Cretan School\",\n",
        "\"Mannerism\",\n",
        "\"Baroque\",\n",
        "\"Rococo\",\n",
        "\"Neoclassicism\",\n",
        "\"Romanticism\",\n",
        "\"Academic art\",\n",
        "\"Realism\",\n",
        "\"Macchiaioli\",\n",
        "\"PreRaphaelite\",\n",
        "\"Naturalism\",\n",
        "\"Art Nouveau\",\n",
        "\"Art Deco\",\n",
        "\"Impressionism\",\n",
        "\"Post-Impressionism\",\n",
        "\"Neo-Impressionism\",\n",
        "\"Neo-Expressionism\",\n",
        "\"Fauvism\",\n",
        "\"Expressionism\",\n",
        "\"Tonalism\",\n",
        "\"Cubism\",\n",
        "\"Surrealism\",\n",
        "\"Futurism\",\n",
        "\"Abstract Expressionism\",\n",
        "\"Avantgarde\",\n",
        "\"Bauhaus\",\n",
        "\"Op Art\",\n",
        "\"Pop Art\",\n",
        "\"Constructivism\",\n",
        "\"Suprematism\",\n",
        "\"New Objectivity\",\n",
        "\"Symbolism\",\n",
        "\"Vorticism\",\n",
        "\"Biomorphism\",\n",
        "\"De Stijl\",\n",
        "\"Socialist\",\n",
        "\"Dadaism\",\n",
        "\"Kinetic Art\",\n",
        "\"Futurism\",\n",
        "\"Harlem Renaissance\",\n",
        "\"Arte Povera\",\n",
        "\"Zero Group\",\n",
        "\"Minimalism\",\n",
        "\"Conceptual Art\",\n",
        "\"Contemporary Art\",\n",
        "\"Lowbrow\",\n",
        "\"Modernism\",\n",
        "\"Deconstuctionism\",\n",
        "\"Post-Modern\",\n",
        "\"Maximalist\",\n",
        "\"Massurealism\",\n",
        "\"Stuckism\",\n",
        "\"Remoderism\",\n",
        "\"Excessivism\",\n",
        "\"art\", #\n",
        "\"modern art\",\n",
        "\"digital art\",\n",
        "\n",
        "\"_\",\n",
        "\n",
        " \"classic painting\",\n",
        " \"historical painting\",\n",
        "\"jazz age\",\n",
        " \"atomic era\",\n",
        "\n",
        "\"abstract minimalism\",\n",
        "\n",
        "\n",
        "\n",
        "]\n",
        "\n",
        "print(f\"art_period {len(art_period)}: {art_period}\")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lH9nmFJrljqo",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title materials\n",
        "# use https://huggingface.co/spaces/pharma/CLIP-Interrogator\n",
        "\n",
        "all_prompt_types.add('material')\n",
        "\n",
        "material = [\n",
        "\"acrylic painting\",\n",
        "\"airbrush painting\",\n",
        "\"aquatint\",\n",
        "\"ballpoint pen art\",\n",
        "\"banhua\",\n",
        "\"black velvet\",\n",
        "\"bokashi\",\n",
        "\"brayer painting\",\n",
        "\"carborundum print\",\n",
        "\"cartoon\",\n",
        "\"casein painting\",\n",
        "\"catchpenny print\",\n",
        "\"cel animation\",\n",
        "\"chalk\",\n",
        "\"character animation\",\n",
        "\"charcoal drawing\",\n",
        "\"chibi art\",\n",
        "\"children’s story book\",\n",
        "\"chine-collé\",\n",
        "\"chromolithography\",\n",
        "\"chromoxylography\",\n",
        "\"cliché verre\",\n",
        "\"collage\",\n",
        "\"collagraphy\",\n",
        "\"colored pencil\",\n",
        "\"comic\",\n",
        "\"cordel literature\",\n",
        "\"crayon\",\n",
        "\"decoupage\",\n",
        "\"disney\",\n",
        "\"doodle\",\n",
        "\"drypoint\",\n",
        "\"e-hon\",\n",
        "\"encaustic\",\n",
        "\"engraving\",\n",
        "\"epinal print\",\n",
        "\"etching\",\n",
        "\"fan art\",\n",
        "\"finger painting\",\n",
        "\"frescoe painting\",\n",
        "\"frescography\",\n",
        "\"fudezaishiki\",\n",
        "\"geomontography\",\n",
        "\"giclée\",\n",
        "\"glitter painting\",\n",
        "\"gond painting\",\n",
        "\"gouache painting\",\n",
        "\"graphite drawing\",\n",
        "\"grease pencil\",\n",
        "\"grisaille\",\n",
        "\"heures de charles d'angoulême\",\n",
        "\"illuminated manuscript\",\n",
        "\"illustrated book\",\n",
        "\"illustrated childrens book\",\n",
        "\"illustration for children\",\n",
        "\"illustration\",\n",
        "\"impasto\",\n",
        "\"ink-wash\",\n",
        "\"japonisme\",\n",
        "\"kappazuri\",\n",
        "\"kuchi-e\",\n",
        "\"limning\",\n",
        "\"linocut\",\n",
        "\"lithography\",\n",
        "\"looney tunes\",\n",
        "\"madhubani painting\",\n",
        "\"metalpoint\",\n",
        "\"mixed media\",\n",
        "\"mosaic\",\n",
        "\"mural\",\n",
        "\"nib painting\",\n",
        "\"oil painting\",\n",
        "\"origami\",\n",
        "\"pabalat\",\n",
        "             \"painting\",\n",
        "\"palette knife\",\n",
        "\"papel picado\",\n",
        "\"pastel\",\n",
        "\"pen drawing\",\n",
        "\"pencil drawing\",\n",
        "\"permanent marker\",\n",
        "\"phad painting\",\n",
        "\"picture book\",\n",
        "\"pixar\",\n",
        "\"pop-up book\",\n",
        "\"puppet film\",\n",
        "\"relief printing\",\n",
        "\"saturday morning cartoon\",\n",
        "\"screen print\",\n",
        "\"screengrab\",\n",
        "\"sfumato\",\n",
        "\"silverpoint\",\n",
        "\"sketch\",\n",
        "\"spray paint\",\n",
        "\"stele rubbing\",\n",
        "\"stencil art\",\n",
        "\"stereochromy\",\n",
        "\"stereoscopic image\",\n",
        "\"stick figure\",\n",
        "\"stop motion\",\n",
        "\"tempera\",\n",
        "\"trois crayons\",\n",
        "\"ukiyo-e\",\n",
        "\"vitreography\",\n",
        "\"vytynanky\",\n",
        "\"warli painting\",\n",
        "\"watercolor\",\n",
        "\"whiteboard art\",\n",
        "\"wimmelbilderbuch\",\n",
        "\"wycinanki\",\n",
        "\n",
        "    \"_\",\n",
        "]\n",
        "print(f\"material ({len(material)}): {material}\")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title adjectives\n",
        "\n",
        "# https://github.com/WASasquatch/noodle-soup-prompts/blob/main/nsp_pantry.json\n",
        "\n",
        "# can be only single words\n",
        "# positive for childrens books\n",
        "all_prompt_types.add('adjectives')\n",
        "\n",
        "adjectives = [\n",
        "\"abandoned\",\n",
        "\"abhorrent\",\n",
        "\"absurdist\",\n",
        "\"acclaimed\",\n",
        "\"accomplished\",\n",
        "\"adroit\",\n",
        "\"aesthetic\",\n",
        "\"alien\",\n",
        "\"alluring\",\n",
        "\"aloof\",\n",
        "\"amazing \",\n",
        "\"amber\",\n",
        "\"amused\",\n",
        "\"angry\",\n",
        "\"anxious\",\n",
        "\"apalling\",\n",
        "\"apocalyptic\",\n",
        "\"appealing\",\n",
        "\"artistic\",\n",
        "\"arty\",\n",
        "\"astonishing\",\n",
        "\"atmospheric\",\n",
        "\"attractive\",\n",
        "\"authentic\",\n",
        "\"avant-garde\",\n",
        "\"award-winning\",\n",
        "\"awe-inspiring\",\n",
        "\"awe-struck\",\n",
        "\"awry\",\n",
        "\"baffled\",\n",
        "\"balanced\",\n",
        "\"basic\",\n",
        "\"beauteous\",\n",
        "\"beautiful\",\n",
        "\"bewildered\",\n",
        "\"bleak\",\n",
        "\"bold\",\n",
        "\"boundless\",\n",
        "\"bright\",\n",
        "\"brilliant\",\n",
        "\n",
        "\"calming\",\n",
        "\"camp\",\n",
        "\"candid\",\n",
        "\"catastrophic\",\n",
        "\"ceramic\",\n",
        "\"chaotic\",\n",
        "\"characteristic\",\n",
        "\"charming\",\n",
        "\"classic\",\n",
        "\"clever\",\n",
        "\"collectable\",\n",
        "\"colorful\",\n",
        "\"colossal\",\n",
        "\"comical\",\n",
        "\"complex\",\n",
        "\"confident\",\n",
        "\"contemplative\",\n",
        "\"contemporary\",\n",
        "\"content\",\n",
        "\n",
        "\"crafty\",\n",
        "\"creative\",\n",
        "\"crippling\",\n",
        "\"cultured\",\n",
        "\"curious\",\n",
        "\"cursed\",\n",
        "\"cute\",\n",
        "\"daring\",\n",
        "\"dazzling\",\n",
        "\"decaying\",\n",
        "\"decorative\",\n",
        "\"delicate\",\n",
        "\"desolate\",\n",
        "\"desperate\",\n",
        "\"detailed\",\n",
        "\"determined\",\n",
        "\"devastated\",\n",
        "\"devastating\",\n",
        "\"devoured\",\n",
        "\"disappointed\",\n",
        "\"disastrous\",\n",
        "\"disciplined\",\n",
        "\"disgusted\",\n",
        "\"disheartening\",\n",
        "\"dismal\",\n",
        "\"distinctive\",\n",
        "\"disturbing\",\n",
        "\"divine\",\n",
        "\"doomed\",\n",
        "\"dramatic\",\n",
        "\"dreamlike\",\n",
        "\"dreamy\",\n",
        "\"dreary\",\n",
        "\"dynamic\",\n",
        "\"eclectic\",\n",
        "\"eerie\",\n",
        "\"elated\",\n",
        "\"elegant\",\n",
        "\"elevated\",\n",
        "\"emotional\",\n",
        "\"enchanted\",\n",
        "\"enchanting\",\n",
        "\"energetic\",\n",
        "\"engaging\",\n",
        "\"engrossing\",\n",
        "\"enigmatic\",\n",
        "\"enthusiastic\",\n",
        "\"enticing\",\n",
        "\"envious\",\n",
        "\"esthetical\",\n",
        "\"ethereal\",\n",
        "\"evocative\",\n",
        "\"exceptional\",\n",
        "\"excited\",\n",
        "\"expressive\",\n",
        "\"exquisite\",\n",
        "\"extreme\",\n",
        "\"eye-catching\",\n",
        "\"fanciful\",\n",
        "\"fascinating\",\n",
        "\"fashionable\",\n",
        "\"fearful\",\n",
        "\"figural\",\n",
        "\"figurative\",\n",
        "\"flawless\",\n",
        "\"fluid\",\n",
        "\"folk\",\n",
        "\"folksy\",\n",
        "\"forlorn\",\n",
        "\"formal\",\n",
        "\"freelance\",\n",
        "\"fresh\",\n",
        "\"frightening\",\n",
        "\"frightful\",\n",
        "\"frustrated\",\n",
        "\"fun\",\n",
        "\"funny\",\n",
        "\"gaudy\",\n",
        "\"genius\",\n",
        "\"ghastly\",\n",
        "\"gifted\",\n",
        "\"gigantic\",\n",
        "\"glamorous\",\n",
        "\"gloomy\",\n",
        "\n",
        "\"gorgeous\",\n",
        "\"gory\",\n",
        "\"graceful\",\n",
        "\"grand\",\n",
        "\"grandiose\",\n",
        "\"grim\",\n",
        "\"gruesome\",\n",
        "\"guilty\",\n",
        "\"handsome\",\n",
        "\"happy\",\n",
        "\"harmonious\",\n",
        "\"harrowing\",\n",
        "\"haunting\",\n",
        "\"heart-wrenching\",\n",
        "\"honest\",\n",
        "\"hopeful\",\n",
        "\"hopeless\",\n",
        "\"horrendous\",\n",
        "\"horrifying\",\n",
        "\"hued\",\n",
        "\"humorous\",\n",
        "\"hyper\",\n",
        "\"hyper-creative\",\n",
        "\"imaginative\",\n",
        "\"immense\",\n",
        "\"impassioned\",\n",
        "\"impatient\",\n",
        "\"impeccable\",\n",
        "\"impossible\",\n",
        "\"infused\",\n",
        "\"inspirational\",\n",
        "\"inspired\",\n",
        "\"inspiring\",\n",
        "\"instinctive\",\n",
        "\"intellectual\",\n",
        "\"intense\",\n",
        "\"interesting\",\n",
        "\"interpretive\",\n",
        "\"intuitive\",\n",
        "\"inventive\",\n",
        "\"joyful\",\n",
        "\"knockout\",\n",
        "\"labyrinthine\",\n",
        "\n",
        "\"layered\",\n",
        "\"light\",\n",
        "\"liquid\",\n",
        "\"literary\",\n",
        "\"luminous\",\n",
        "\"lyrical\",\n",
        "\"macabre\",\n",
        "\"magical\",\n",
        "\"magisterial\",\n",
        "\n",
        "\"massive\",\n",
        "\"melancholic\",\n",
        "\"memorable\",\n",
        "\"miraculous\",\n",
        "\"miserable\",\n",
        "\"monstrous\",\n",
        "\"monumental\",\n",
        "\"mournful\",\n",
        "\"moving\",\n",
        "\"mundane\",\n",
        "\"musical\",\n",
        "\"mysterious\",\n",
        "\"mystical\",\n",
        "\"narrative\",\n",
        "\"naturalistic\",\n",
        "\"nauseating\",\n",
        "\"nervous\",\n",
        "\"nonchalant\",\n",
        "\"nubile\",\n",
        "\n",
        "\"oppressive\",\n",
        "\n",
        "\n",
        "\"organic\",\n",
        "\"original\",\n",
        "\"pained\",\n",
        "\"paradoxical\",\n",
        "\"passionate\",\n",
        "\"patina\",\n",
        "\"peaceful\",\n",
        "\n",
        "\"pensive\",\n",
        "\"perfect\",\n",
        "\"personable\",\n",
        "\"petrifying\",\n",
        "\"phenomenal\",\n",
        "\"philosophical\",\n",
        "\"picturesque\",\n",
        "\"playful\",\n",
        "\"pleasant\",\n",
        "\"poetic\",\n",
        "\"pretty\",\n",
        "\"pure\",\n",
        "\"questionable\",\n",
        "\"radiant\",\n",
        "\"ravishing\",\n",
        "\"regretful\",\n",
        "\"relieved\",\n",
        "\"religious\",\n",
        "\"remarkable\",\n",
        "\"rhythmical\",\n",
        "\"rich\",\n",
        "\"romantic\",\n",
        "\n",
        "\"ruined\",\n",
        "\"sad\",\n",
        "\n",
        "\"satire\",\n",
        "\"saturated\",\n",
        "\"sensual\",\n",
        "\"sensuous\",\n",
        "\"serene\",\n",
        "\"shocking\",\n",
        "\"showstopping\",\n",
        "\"shy\",\n",
        "\"simple\",\n",
        "\"smart\",\n",
        "\"soft\",\n",
        "\"sorrowful\",\n",
        "\"spacey\",\n",
        "\"sparse\",\n",
        "\"spiritual\",\n",
        "\"statuesque\",\n",
        "\"stimulating\",\n",
        "\"stirring\",\n",
        "\"studied\",\n",
        "\"stunning\",\n",
        "\"stylish\",\n",
        "\"stylized\",\n",
        "\"sublime\",\n",
        "\"substantive\",\n",
        "\"superb\",\n",
        "\"supernatural\",\n",
        "\"supple\",\n",
        "\"surprised\",\n",
        "\"surreal\",\n",
        "\"symbolic\",\n",
        "\"tasteful\",\n",
        "\"telegenic\",\n",
        "\"traditional\",\n",
        "\"tragic\",\n",
        "\"tranquil\",\n",
        "\"trendy\",\n",
        "\"turquoise\",\n",
        "\"unconventional\",\n",
        "\"unexpected\",\n",
        "\"unimaginable\",\n",
        "\"unique\",\n",
        "\"universal\",\n",
        "\"unpredictable\",\n",
        "\"unpretentious\",\n",
        "\"vibrant\",\n",
        "\"visionary\",\n",
        "\"vivid \",\n",
        "\"weird\",\n",
        "\"whimsical\",\n",
        "\"wretched\",\n",
        "\"zingy\",\n",
        "\n",
        "               \"and more and more and more\",\n",
        "               \"etc.\",\n",
        "\n",
        "# # Colors\n",
        "# \"aqua blue\",\n",
        "# \"aquamarine\",\n",
        "# \"beige\",\n",
        "# \"blue\",\n",
        "# \"bronze colored\",\n",
        "# \"burgundy red\",\n",
        "# \"champagne yellow\",\n",
        "# \"chartreuse\",\n",
        "# \"copper colored\",\n",
        "# \"coral orange\",\n",
        "# \"crimson\",\n",
        "# \"cyan\",\n",
        "# \"emerald green\",\n",
        "# \"fuchsia\",\n",
        "# \"gold colored\",\n",
        "# \"green\",\n",
        "# \"indigo\",\n",
        "# \"ivory white\",\n",
        "# \"lavender purple\",\n",
        "# \"lime green\",\n",
        "# \"magenta\",\n",
        "# \"maroon\",\n",
        "# \"mint green\",\n",
        "# \"mauve\",\n",
        "# \"mustard yellow\",\n",
        "# \"navy blue\",\n",
        "# \"olive green\",\n",
        "# \"orange\",\n",
        "              #  \"orchid\",\n",
        "# \"peach red\",\n",
        "# \"pearl white\",\n",
        "# \"periwinkle\",\n",
        "# \"pink\",\n",
        "# \"plum red\",\n",
        "# \"purple\",\n",
        "# \"red\",\n",
        "# \"rose red\",\n",
        "# \"ruby red\",\n",
        "# \"salmon orange\",\n",
        "# \"sapphire red\",\n",
        "# \"scarlet red\",\n",
        "              #  \"sepia\",\n",
        "\n",
        "# \"silver colored\",\n",
        "# \"tan brown\",\n",
        "              #  \"teal\",\n",
        "\n",
        "# \"violet\",\n",
        "# \"vermillion\",\n",
        "# \"yellow\",\n",
        "\n",
        "\"_\",\n",
        "]\n",
        "\n",
        "\n",
        "print(f\"adjectives ({len(adjectives)}): {adjectives}\")\n"
      ],
      "metadata": {
        "id": "gspdJaa4JI-a",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title background\n",
        "all_prompt_types.add('background')\n",
        "\n",
        "background = [\n",
        "\n",
        "  \"_\",\n",
        "\n",
        "  # \"forest\",\n",
        "  # \"city\",\n",
        "  # \"beach\",\n",
        "  # \"mountains\",\n",
        "# \"huddled masses of the poor\",\n",
        "\n",
        "\n",
        "]\n",
        "\n",
        "print(f\"background ({len(background)}): {background}\")\n",
        "\n"
      ],
      "metadata": {
        "id": "NV58GJzZJNbb",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title time of day\n",
        "all_prompt_types.add('time_of_day')\n",
        "\n",
        "time_of_day = [\n",
        "\"_\",\n",
        "# \"sunrise\",\n",
        "#                 \"sunset\",\n",
        "#                 \"golden hour\",\n",
        "\n",
        "]\n",
        "\n",
        "print(f\"time_of_day ({len(time_of_day)}): {time_of_day}\")\n",
        "\n"
      ],
      "metadata": {
        "id": "RRRcFq0bJR70",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title image qualities\n",
        "all_prompt_types.add('image_qualities')\n",
        "\n",
        "image_qualities = [\n",
        "\"2019\",\n",
        "\"21:9\",\n",
        "                    \"32k UHD\",\n",
        "\"35mm\",\n",
        "\"4k\",\n",
        "\"8k\",\n",
        "\"artstation\",\n",
        "\"atmospheric\",\n",
        "\"award-winning\",\n",
        "\"cinematic lighting\",\n",
        "\"deep depth of field\",\n",
        "                    \"detailed\",\n",
        "\"deviant art\",\n",
        "\"dramatic lighting\",\n",
        "\"dslr camera\",\n",
        "\"exquisite details\",\n",
        "\"exquisite textures\",\n",
        "\"extreme long shot\",\n",
        "\"extreme wide shot\",\n",
        "\"extremely detailed\",\n",
        "\"f/22\",\n",
        "\"fantastic backlight\",\n",
        "\"full shot\",\n",
        "\"hdr\",\n",
        "\"highly detailed\",\n",
        "\"high resolution\",\n",
        "\"high definition\",\n",
        "\"hyper detailed\",\n",
        "\"hyper realistic\",\n",
        "\"hyperrealism\",\n",
        "\"image in center\",\n",
        "\"instagram contest winner\",\n",
        "\"intricate details\",\n",
        "\"kodak pro gold\",\n",
        "\"large format\",\n",
        "\"ldsr camera\",\n",
        "\"long shot\",\n",
        "\"maximum texture\",\n",
        "\"national geographic\",\n",
        "\"medium format\",\n",
        "\"perfect lighting\",\n",
        "\"perfect composition\",\n",
        "\"photo courtesy museum of art\",\n",
        "\"photography\",\n",
        "\"rim lighting\",\n",
        "\"rule of thirds\",\n",
        "\"sharp features\",\n",
        "\"sharp focus\",\n",
        "\"sharp\",\n",
        "\"anamorphic lenses\",\n",
        "\"shutterstock contest winner\",\n",
        "\"smooth\",\n",
        "\"soft lighting\",\n",
        "\"studio light\",\n",
        "\"theatrical\",\n",
        "\"trending on artstation\",\n",
        "\"ultra detailed\",\n",
        "\"ultra photoreal\",\n",
        "\"ultra realistic\",\n",
        "\"unreal engine\",\n",
        "\"very crisp\",\n",
        "\"very detailed\",\n",
        "\"volumetric\",\n",
        "\"wide angle lens\",\n",
        "\"wide shot\",\n",
        "\"widescreen shot\",\n",
        "\n",
        "#lenses:\n",
        "# \"Sigma 50mm T1.5 FF High-Speed Prime\",\n",
        "\"OM-D E-M5 Mark III\",\n",
        "# \"M.Zuiko Digital ED 12–40mm F2.8 PRO\",\n",
        "\"1/50sec\",\n",
        "\"ISO64\",\n",
        "\"OM system 12–40mm PRO\",\n",
        "# \"Samyang/Rokinon Xeen 50mm T1.5\",\n",
        "\"Sigma 40mm f/1.4 DG HSM\",\n",
        "\"landscape\",\n",
        "\n",
        "# new lenses\n",
        "\"Nikon Z9 200 mm lens\",\n",
        "\n",
        "\"clean detailed faces\",\n",
        "\"intracate clothing\",\n",
        "\"analogous colors\",\n",
        "\"glowing shadows\",\n",
        "\"beautiful gradient\",\n",
        "\"depth of field\",\n",
        "\"clean image\",\n",
        "\n",
        "\n",
        " # film types\n",
        " \"Kodak Ektachrome E100\",\n",
        " \"Fujifilm Pro 400H\",\n",
        " \"Agfa Vista 400\",\n",
        " \"Kodak Gold 100\",\n",
        " \"Kodak Gold 200\",\n",
        " \"Kodak Portra 400\",\n",
        " \"Kodak Ektar 100\",\n",
        " \"Fujichrome Velvia 50\",\n",
        " \"CineStill 800T\",\n",
        " \"Fujifilm Superia 400\",\n",
        "\"AgfaColor Neu\",\n",
        "  \"Anscochrome\",\n",
        "  \"Fujifilm Velvia\",\n",
        "  \"Agfa CT Precisa\",\n",
        "                    \"Lomography Color Negative 800\",\n",
        "\"Fujifilm Provia film\",\n",
        "\"ISO 400\",\n",
        "\"Kodak film\",\n",
        "\n",
        "  # TODO: https://aesthetics.fandom.com/wiki/List_of_Aesthetics\n",
        "\n",
        "  \"associated press photo\",\n",
        "  \"AP photo\",\n",
        "  \"UHD\",\n",
        "\n",
        "\n",
        "  # from clip interegator\n",
        "\n",
        "  \"awesome composition\",\n",
        "  \"trending on cgstation\",\n",
        "  \"high detailed official artwork\",\n",
        "  \"9k\",\n",
        "  \"official artwork\",\n",
        "\n",
        "  \"mesmerizing shot\",\n",
        "  # \"\",\n",
        "  # \"\",\n",
        "# \"shot in a realistic and cinematic style with RED Weapon Monstro 8K VV\",\n",
        "\"shot on hasselblad\",\n",
        "\"shot with Canon EF on Kodak Portra 800 film\",\n",
        "\"shot with a Hasselblad camera\",\n",
        "                    \"the image boasts incredible realism\",\n",
        "# \"tokina at-x 11-16mm f/2.8 pro dx ii\",\n",
        "                    \"85mm lens\",\n",
        "# \"Arri Alexa Mini LF with a Cooke S7/i 35mm T2.0 lens\",\n",
        "\"Contax T3 SLR\",\n",
        "\n",
        "\"Leica 85mm lens\",\n",
        "\"Leica Summilux-C Prime Lenses.\",\n",
        "\"Matte\",\n",
        "# \"Sigma 150-600mm f/5-6.3 DG OS HSM Sports lens at 600mm\",\n",
        "\"Sony a7s III\",\n",
        "                    \"aesthetic photostrip from a photobooth\",\n",
        "\"cartridge-loaded\",\n",
        "\"chemical-processed\",\n",
        "                    \"contrasting color\",\n",
        "\"contrasting softness\",\n",
        "                    \"digital art smooth\",\n",
        "\"dreamlike aura\",\n",
        "\"natural lighting\",\n",
        "                    \"nikon d850\",\n",
        "\"non-digital\",\n",
        "\"pentax 645n\",\n",
        "                    \"stunning color grading\",\n",
        "\"light-sensitive\",\n",
        "\"point-and-shoot\",\n",
        "\"perfect color graded\",\n",
        "\"beautifully color-coded\",\n",
        "\"cinematic look\",\n",
        "\"cinematic shot\",\n",
        "                    \"contest winner\",\n",
        "\"fixed-aperture\",\n",
        "\"fixed-focus\",\n",
        "\"flash-enabled\",\n",
        "      \"global illumination\",\n",
        "\"glossy\",\n",
        "\"grainy\",\n",
        "\"handheld\",\n",
        "                    \"cartridge-loaded\",\n",
        "\"chemical-processed\",\n",
        "                    \"contrasting color\",\n",
        "\"contrasting softness\",\n",
        "                    \"cinematic dark atmosphere\",\n",
        "                    \"gloomy and foggy atmosphere\",\n",
        "     \"night cinematic lighting\",\n",
        "       \"supersharp\",\n",
        "                          \"sharp edges\",\n",
        "                    \"jagged edges\",\n",
        "\n",
        "\n",
        "\n",
        "]\n",
        "\n",
        "print(f\"image_qualities ({len(image_qualities)}): {image_qualities}\")\n"
      ],
      "metadata": {
        "id": "snL9GnfAJWhN",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title adverbs\n",
        "\n",
        "all_prompt_types.add('adverbs')\n",
        "\n",
        "adverbs = [\n",
        "\n",
        "\"admirably\",\n",
        "\"aesthetic joys of dance \",\n",
        "\"affectionately \",\n",
        "\"amazing contrast of tenderness and cruelty\",\n",
        "\"anatomical\",\n",
        "\"anatomically\",\n",
        "\"badly\",\n",
        "\"benevolently \",\n",
        "\"boastfully\",\n",
        "\"boldly\",\n",
        "\"bravely\",\n",
        "\"breathtakingly\",\n",
        "\"brightly\",\n",
        "\"brilliantly\",\n",
        "\"busily\",\n",
        "\"calmly\",\n",
        "\"carefully\",\n",
        "\"caringly\",\n",
        "\"celebrated works\",\n",
        "\"cheerfully\",\n",
        "\"choreographed\",\n",
        "\"classic model\",\n",
        "# \"clearly\",\n",
        "\"complex and ever-changing emotions\",\n",
        "\"courageously\",\n",
        "\"cruelly\",\n",
        "\"daily\",\n",
        "\"dramatically\",\n",
        "\"dripping \",\n",
        "\"easily\",\n",
        "\"ecstatically\",\n",
        "\"elegantly\",\n",
        "\"emotion-packed\",\n",
        "\"enormously\",\n",
        "\"enthusiastically\",\n",
        "\"exactly\",\n",
        "\"expressive faces\",\n",
        "\"exultation\",\n",
        "\"faithfully\",\n",
        "\"fiercely\",\n",
        "\"finesse\",\n",
        "\"fondly\",\n",
        "\"foolishly\",\n",
        "\"fortunately\",\n",
        "\"gently\",\n",
        "\"gladly\",\n",
        "\"gracefully\",\n",
        "\"graciousness\",\n",
        "\"greedily\",\n",
        "\"happily\",\n",
        "\"honestly\",\n",
        "\"innocently\",\n",
        "\"intricately\",\n",
        "\"inventive\",\n",
        "\"joyfully\",\n",
        "\"joyously\",\n",
        "\"kindly\",\n",
        "\"laughingly\",\n",
        "\"lyrically\",\n",
        "\"manifest ability\",\n",
        "\"merrily\",\n",
        "\"neatly\",\n",
        "\"physical interpretation\",\n",
        "\n",
        "\"reimagined\",\n",
        "\n",
        "\"shimmering \",\n",
        "\"smooth skin\",\n",
        "\n",
        "\"strong emotion\",\n",
        "\"strong dynamic pose\",\n",
        "\"swiftly\",\n",
        "\"tenderly\",\n",
        "\"warmly\",\n",
        "\"wheeling\",\n",
        " \"tense and dynamic atmosphere\",\n",
        "\"dynamic image immerses viewers\",\n",
        "\"exaggerated facial features\",\n",
        "\"juxtaposition of ethereal beauty and unsettling transformation\",\n",
        "\"reminiscent of a dreamlike\",\n",
        "\"emotive body language\",\n",
        "\n",
        "\n",
        " \"_\",\n",
        "\n",
        "\n",
        "]\n",
        "print(f\"adverbs ({len(adverbs)}): {adverbs}\")\n",
        "\n"
      ],
      "metadata": {
        "id": "ky4nU_-aJewj",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title shape\n",
        "all_prompt_types.add('shape')\n",
        "\n",
        "shape = [\n",
        "\"_\",\n",
        "\"photograph\",\n",
        "\"painting\",\n",
        "\n",
        "\n",
        "]\n",
        "print(f\"shape ({len(shape)}): {shape}\")\n"
      ],
      "metadata": {
        "id": "DhnkUV6rgVYH",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9DPcgVl5BBdA",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title artists from files\n",
        "all_prompt_types.add('artists')\n",
        "\n",
        "from pathlib import Path\n",
        "!pip install unidecode\n",
        "from unidecode import unidecode\n",
        "\n",
        "artists = set()\n",
        "\n",
        "artists_file = PROJECT_PATH / \"artists.txt\"\n",
        "\n",
        "print(f\"getting artists from : {artists_file}\")\n",
        "\n",
        "try:\n",
        "  with open(artists_file, 'r', encoding='UTF-8') as file:\n",
        "    for line in file:\n",
        "      # print(line.strip())\n",
        "      artists.add(unidecode(line.strip()))\n",
        "\n",
        "  # turn set into a list\n",
        "  artists = list(artists)\n",
        "  print(f\"Found {len(artists)} in {artists_file}\")\n",
        "except Exception as e:\n",
        "  print(f\"NO file: {artists_file}\")\n",
        "  artists = [\"_\"]\n",
        "\n",
        "print(f\"Found {len(artists)}\")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zgQGyRlY9cBN"
      },
      "outputs": [],
      "source": [
        "#@title numeric options\n",
        "\n",
        "all_prompt_types.add('steps')\n",
        "all_prompt_types.add('guidance_scale')\n",
        "\n",
        "# what was chosen at the top\n",
        "steps = [pipe_steps] #\n",
        "\n",
        "# they suggest just 0.0, and no negative prompt\n",
        "guidance_scale = [0.0] #\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EY_quiwAY457"
      },
      "source": [
        "# Core Prompt Functions (always required). Also, rerun after getting final prompt votes (Step 1c)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5aoKRsib052V",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title core_prompt function\n",
        "\n",
        "import random\n",
        "import re\n",
        "import pickle\n",
        "from collections import OrderedDict\n",
        "import numpy as np\n",
        "import pprint\n",
        "\n",
        "\n",
        "# global\n",
        "cur_core_prompt_dict = {}\n",
        "\n",
        "test_weights = False\n",
        "\n",
        "# load the prompt gain\n",
        "gain_path = PROJECT_PATH / \"prompt_gain.pkl\"\n",
        "print(f\"using gain_path: {gain_path}\")\n",
        "\n",
        "\n",
        "def update_prompt_type(prompt_type):\n",
        "  # to convert from sd2.1 to SDXL styles\n",
        "\n",
        "  if prompt_type == 'art_periods':\n",
        "    prompt_type_out = 'art_period'\n",
        "  elif prompt_type == 'materials':\n",
        "    prompt_type_out = 'material'\n",
        "  elif prompt_type == 'scale':\n",
        "    prompt_type_out = 'guidance_scale'\n",
        "  else:\n",
        "    prompt_type_out = prompt_type\n",
        "\n",
        "  return prompt_type_out\n",
        "\n",
        "\n",
        "def set_prompt_probs(prompt_type):\n",
        "\n",
        "  # converting from old dict keys to new:\n",
        "  new_prompt_type = update_prompt_type(prompt_type)\n",
        "\n",
        "  if new_prompt_type not in all_prompt_types:\n",
        "    print(f\"prompt_type deprecated: {prompt_type}\")\n",
        "    return None, None\n",
        "\n",
        "  # create an ordered dict\n",
        "  ordered_option2weight = OrderedDict(prompt_gain[prompt_type])\n",
        "\n",
        "\n",
        "  print(f\"{prompt_type}: current weights ({len(ordered_option2weight)}): {ordered_option2weight}\")\n",
        "\n",
        "  # will only include items not currently excluded\n",
        "  cur_items = [(update_prompt_type(i), ordered_option2weight[i]) for i in ordered_option2weight.keys() if i in eval(new_prompt_type)]\n",
        "\n",
        "  # add in any new items with a wt of 1\n",
        "  cur_keys = [i for i,w in cur_items]\n",
        "  for item in eval(new_prompt_type):\n",
        "    if item not in cur_keys:\n",
        "      # print(f\"adding to {k}: {item}\")\n",
        "      cur_items.append((item, 1))\n",
        "\n",
        "\n",
        "  total_cnt = sum([wt for i, wt in cur_items])\n",
        "  options = [i for i, wt in cur_items]\n",
        "  probs = [wt / total_cnt for i, wt in cur_items]\n",
        "  return options, probs\n",
        "\n",
        "# TODO: if check for any new prompt types give them even probs\n",
        "\n",
        "can_use_weights = False\n",
        "# for each type, set the list of options and probs once\n",
        "prompt_type2options = {}\n",
        "\n",
        "if gain_path.exists():\n",
        "  # this path won't exist until the first voting which will make sure every\n",
        "  # known prompt gets a value of 1\n",
        "  can_use_weights = True\n",
        "\n",
        "  print(f\"Loading gain from {gain_path}\")\n",
        "  with open(gain_path, 'rb') as handle:\n",
        "      prompt_gain = pickle.load(handle)\n",
        "\n",
        "  print(f\"prompt_gain: keys={prompt_gain.keys()}\")\n",
        "\n",
        "  # set the dictionary\n",
        "  for k in prompt_gain.keys():\n",
        "\n",
        "    options, probs = set_prompt_probs(k)\n",
        "    if not options:\n",
        "      continue\n",
        "\n",
        "\n",
        "    k = update_prompt_type(k)\n",
        "    prompt_type2options[k] = {}\n",
        "    prompt_type2options[k]['options'] = options\n",
        "    prompt_type2options[k]['probs'] = probs\n",
        "\n",
        "  # get any new prompts\n",
        "  for k in all_prompt_types:\n",
        "    if k not in prompt_type2options:\n",
        "      print(f\"new prompt_type: {k}\")\n",
        "      total_cnt = len(eval(k)) # the number of items\n",
        "      options = eval(k)\n",
        "      probs = [1 / total_cnt] * total_cnt\n",
        "      prompt_type2options[k] = {}\n",
        "      prompt_type2options[k]['options'] = options\n",
        "      prompt_type2options[k]['probs'] = probs\n",
        "\n",
        "\n",
        "  # print(f\"final:\")\n",
        "  # pprint.pprint(prompt_type2options)\n",
        "\n",
        "\n",
        "def get_sample(prompt_type, option_cnt):\n",
        "  if not prompt_type in prompt_type2options:\n",
        "    return [\"_\"]\n",
        "  if not prompt_type2options[prompt_type]['options']:\n",
        "    return [\"_\"]\n",
        "\n",
        "  options_to_use = np.random.choice(prompt_type2options[prompt_type]['options'],\n",
        "                    replace=False,\n",
        "                    p=prompt_type2options[prompt_type]['probs'],\n",
        "                    size=option_cnt, )\n",
        "\n",
        "  # print(f\"{prompt_type}: options_to_use ({len(options_to_use)}): {options_to_use}\")\n",
        "  return options_to_use.tolist()\n",
        "\n",
        "\n",
        "def get_core_prompt():\n",
        "  prompt_type2cnt = {}\n",
        "  for prompt_type in all_prompt_types:\n",
        "    # most types get just 1\n",
        "    prompt_type2cnt[prompt_type] = 1\n",
        "\n",
        "  prompt_type2cnt['artists'] = random.randint(0, min(6, len(artists)))\n",
        "  prompt_type2cnt['adjectives'] = random.randint(1, min(3, len(adjectives)))\n",
        "  prompt_type2cnt['adverbs'] = random.randint(2, min(4, len(adverbs)))\n",
        "  prompt_type2cnt['image_qualities'] = random.randint(2, min(4, len(image_qualities)))\n",
        "\n",
        "  cur_dict = {}\n",
        "\n",
        "  # use the weight values\n",
        "  if can_use_weights:\n",
        "    for prompt_type in all_prompt_types:\n",
        "      cur_dict[prompt_type] = get_sample(prompt_type, prompt_type2cnt[prompt_type])\n",
        "\n",
        "  else:\n",
        "    for prompt_type in all_prompt_types:\n",
        "      cur_dict[prompt_type] = random.sample(eval(prompt_type), prompt_type2cnt[prompt_type])\n",
        "\n",
        "  print(f\"cur_core_prompt: {cur_dict}\")\n",
        "  return cur_dict\n",
        "\n",
        "def get_prompt_for_line(line_num):\n",
        "  #\n",
        "  #  [line], [color] [art_period] [material] [shape] by [artists]\n",
        "\n",
        "  # need to check token count of entire prompt, must be less than 77\n",
        "\n",
        "  token_count = 100\n",
        "  while token_count > 77:\n",
        "    out_prompt = lines[line_num]\n",
        "\n",
        "    # print(f\"line prompt: {out_prompt}\")\n",
        "\n",
        "    # everything is a list, so have to concatenate each type\n",
        "    out_prompt = out_prompt \\\n",
        "                  + \", \" \\\n",
        "                  + \", \".join(cur_core_prompt_dict['adjectives'])  \\\n",
        "                  + \" \" \\\n",
        "                  + \" \" + \", \".join(cur_core_prompt_dict['material'])  \\\n",
        "                  + \" \" \\\n",
        "                  + \" \" + \", \".join(cur_core_prompt_dict['art_period'])  \\\n",
        "                  + \" \" \\\n",
        "                  + \" \" + \", \".join(cur_core_prompt_dict['shape'])  \\\n",
        "                  + \" \" \\\n",
        "                  + \" by the artists \" + \" and \".join(cur_core_prompt_dict['artists']) \\\n",
        "                  + \" \" \\\n",
        "                  + \", \" + \", \".join(cur_core_prompt_dict['background'])  \\\n",
        "                  + \" \" \\\n",
        "                  + \", \" + \", \".join(cur_core_prompt_dict['time_of_day'])  \\\n",
        "                  + \" \" \\\n",
        "                  + \", \" + \", \".join(cur_core_prompt_dict['adverbs']) \\\n",
        "                  + \" \" \\\n",
        "                  + \", \" + \", \".join(cur_core_prompt_dict['image_qualities'])\n",
        "\n",
        "    out_prompt = re.sub(\"_\", \"\", out_prompt)\n",
        "    out_prompt = re.sub(\"\\s+\", \" \", out_prompt)\n",
        "    out_prompt = re.sub(\"(\\,\\s){2,}\", \", \", out_prompt)\n",
        "\n",
        "    token_count = get_token_count(out_prompt)\n",
        "    print(f\"prompt token_count ({token_count}): {out_prompt}\")\n",
        "\n",
        "    if token_count > 77:\n",
        "      # remove from the last item in image_qualities\n",
        "      if len(cur_core_prompt_dict['image_qualities']) > 0:\n",
        "        # get rid of the last item\n",
        "        removed = cur_core_prompt_dict['image_qualities'].pop()\n",
        "        print(f\"{token_count}: removed from image_qualities: {removed}\")\n",
        "      elif len(cur_core_prompt_dict['adverbs']) > 0:\n",
        "        # do these second if still too long\n",
        "        # get rid of the last item\n",
        "        removed = cur_core_prompt_dict['adverbs'].pop()\n",
        "        print(f\"{token_count}: removed from adverbs: {removed}\")\n",
        "      elif len(cur_core_prompt_dict['artists']) > 0:\n",
        "        # do these third if still too long\n",
        "        # get rid of the last item\n",
        "        removed = cur_core_prompt_dict['artists'].pop()\n",
        "        print(f\"{token_count}: removed from artists: {removed}\")\n",
        "      else:\n",
        "        raise ValueError(f\"token_count {token_count} is still too long\")\n",
        "\n",
        "  return out_prompt\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jcs_5ocgaoBM"
      },
      "outputs": [],
      "source": [
        "# testing\n",
        "\n",
        "try:\n",
        "    cur_core_prompt_dict = get_core_prompt()\n",
        "    print(f\"cur_core_prompt_dict: {cur_core_prompt_dict}\")\n",
        "    prompt = get_prompt_for_line(0)\n",
        "    print(f\"prompt: {prompt}\")\n",
        "\n",
        "    # get weighted options\n",
        "    if can_use_weights:\n",
        "      print(\"\\nfrom weights:\")\n",
        "      print(f\"scale: {get_sample('guidance_scale', 1)[0]}\")\n",
        "      print(f\"steps: {get_sample('steps', 1)[0]}\")\n",
        "\n",
        "  # TODO: get the new scheduler name\n",
        "except Exception as e:\n",
        "  print(e)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5DvIRhptuEP_"
      },
      "source": [
        "#Sample Exploration (GPU required)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kyxSN-zLk8mb",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title Sample Exploration\n",
        "\n",
        "from IPython import display\n",
        "\n",
        "#@title Sample Exploration Meta-Run (to narrow down core_prompt)\n",
        "\n",
        "# get the available core prompts\n",
        "# get a random set of the core prompts\n",
        "# combine and run the meta as many times as given\n",
        "# go through each line\n",
        "\n",
        "import random\n",
        "import time\n",
        "import gc\n",
        "import glob\n",
        "from pathlib import Path\n",
        "import re\n",
        "\n",
        "attempts = 2  #@param\n",
        "kill_when_done = False #@param{type:\"boolean\"}\n",
        "\n",
        "#@markdown much faster if you don't show images\n",
        "show_images = False #@param{type:\"boolean\"}\n",
        "\n",
        "#@markdown if greater than zero will only do the first max_lines\n",
        "max_lines = 0 #@param{type:\"integer\"}\n",
        "\n",
        "#@markdown if greater than zero will start at min_line\n",
        "min_lines = 0 #@param{type:\"integer\"}\n",
        "\n",
        "#@markdown https://stablediffusionxl.com/sdxl-resolutions-and-aspect-ratios/\n",
        "\n",
        "width = 1344 #@param{type:\"integer\"}\n",
        "height = 768 #@param{type:\"integer\"}\n",
        "\n",
        "\n",
        "def get_line_name(line_number):\n",
        "  # get a short version of the line\n",
        "  out_name = lines[line_number]\n",
        "  # Remove all non-word characters (everything except numbers and letters)\n",
        "  out_name = re.sub(r\"[^\\w\\s]\", '', out_name)\n",
        "  # Replace all runs of whitespace with a single dash\n",
        "  out_name = re.sub(r\"\\s+\", '-', out_name)\n",
        "  # get the first 3 words as the outname\n",
        "  out_name = \"-\".join(out_name.split(\"-\")[:3])\n",
        "  return out_name\n",
        "\n",
        "\n",
        "for a in range(attempts):\n",
        "  display.clear_output(wait=True)\n",
        "\n",
        "  for line_number in range(len(lines)):\n",
        "    if max_lines > 0 and line_number >= max_lines:\n",
        "      continue\n",
        "    if min_lines > 0 and line_number < min_lines:\n",
        "      continue\n",
        "\n",
        "    image_settings = SimpleNamespace()\n",
        "    # get a random instance of the core prompt dict, global\n",
        "    cur_core_prompt_dict = get_core_prompt()\n",
        "\n",
        "    # these are likely fixed\n",
        "    image_settings.width = width\n",
        "    image_settings.height = height\n",
        "    image_settings.line_number = line_number\n",
        "\n",
        "    # where will put the out image\n",
        "    image_settings.line_name = get_line_name(line_number)\n",
        "    subdir = f\"line_{line_number:02}_{image_settings.line_name}\"\n",
        "    print(f\"round: {a+1}; batch_name: {subdir}\")\n",
        "\n",
        "    image_settings.prompt = get_prompt_for_line(line_number)\n",
        "    # getting the prompt for the specific line, may change image_qualities in cur_core_prompt_dict\n",
        "    # so set this after getting the prompt\n",
        "    image_settings.core_prompt_dict = cur_core_prompt_dict\n",
        "\n",
        "    # to show all the settings:\n",
        "    # pprint.pprint(image_settings.core_prompt_dict)\n",
        "\n",
        "    # print(f\"line {line_number+1:2}/{len(lines):2}: attempt {a+1:2} /{attempts:2}: {image_settings.prompt}\")\n",
        "    print(f\"\\tprompt  : {image_settings.prompt}\")\n",
        "\n",
        "    # number values\n",
        "    image_settings.guidance_scale = cur_core_prompt_dict['guidance_scale'][0]\n",
        "    image_settings.steps = cur_core_prompt_dict['steps'][0]\n",
        "    print(f\"scale: {image_settings.guidance_scale}; steps:{image_settings.steps}\")\n",
        "\n",
        "    image_settings.seed = random.randint(1,2147483647)\n",
        "    output_image = get_one_image(image_settings)\n",
        "\n",
        "    save_image_with_settings(output_image, image_settings, subdir=subdir)\n",
        "\n",
        "    if show_images:\n",
        "      display.display(output_image)\n",
        "\n",
        "    # clean up unused memory\n",
        "    gc.collect()\n",
        "    torch.cuda.empty_cache()\n",
        "\n",
        "\n",
        "if kill_when_done:\n",
        "  # kill switch when done\n",
        "  from google.colab import runtime\n",
        "  runtime.unassign()\n",
        "  print(\"DONE!\")\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GWkU2lGo69n6"
      },
      "source": [
        "# Prompt Voting (no GPU required)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MBZrGJQcgcS8",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title Prompt Voting\n",
        "\n",
        "# go through the directories with a prefix\n",
        "# show the image to the user\n",
        "# (a)dd or (s)kip; add=good style, skip=bad style\n",
        "# if an add, then read the json for that image, pull out the prompts\n",
        "# add a counter dictionary for each prompt, at the end or (q)uit, show the counts\n",
        "from pathlib import Path\n",
        "import os, re\n",
        "import glob\n",
        "from google.colab.patches import cv2_imshow\n",
        "import cv2\n",
        "import time\n",
        "import json\n",
        "from collections import defaultdict\n",
        "from tqdm import tqdm\n",
        "\n",
        "from PIL import Image\n",
        "\n",
        "def remove_empty_dir(dir):\n",
        "  #remove directory if empty\n",
        "  if len(os.listdir(dir)) == 0:\n",
        "    try:\n",
        "      print(f\"trying to remove empty dir: {dir}\")\n",
        "      os.rmdir(dir)\n",
        "      print(f\"\\tremoved empty dir: {dir}\")\n",
        "    except Exception as e:\n",
        "      print(e)\n",
        "      pass\n",
        "\n",
        "def move_files_to_path(files, path):\n",
        "\n",
        "  print(f\"moving {len(files)} to {path}\")\n",
        "  # move the skipped files\n",
        "  for f in files:\n",
        "    file_path = Path(f)\n",
        "    # and the settings file\n",
        "    core_file_str = str(file_path.stem)\n",
        "    timestamp = core_file_str.split(\"_\")[0]\n",
        "    json_file = file_path.parent / f\"{timestamp}_settings.json\"\n",
        "    # move them\n",
        "    try:\n",
        "      file_path.rename(path / file_path.name)\n",
        "      json_file.rename(path / json_file.name)\n",
        "      #print(f\"move {f} TO {outtake_path / file_path.name}\")\n",
        "    except Exception as e:\n",
        "      print(f\"ERROR moving: {f}\")\n",
        "      print(e)\n",
        "\n",
        "    remove_empty_dir(file_path.parent)\n",
        "\n",
        "\n",
        "input_files = [f\n",
        "               for f in map(Path, sorted(glob.glob(f\"{review_path}/*/*\")))\n",
        "               if f.is_file() and f.suffix in [\".jpg\", \".png\"]\n",
        "               ]\n",
        "\n",
        "total_file_cnt = len(input_files)\n",
        "\n",
        "print(f\"input_files {total_file_cnt}: {input_files}\")\n",
        "\n",
        "accepted_files = []\n",
        "delete_files = []\n",
        "skip_files = []\n",
        "keep_files = []\n",
        "\n",
        "print(\"Loading images\")\n",
        "file2image = {}\n",
        "for i in tqdm(range(total_file_cnt)):\n",
        "  file2image[str(input_files[i])] = cv2.imread(str(input_files[i]))\n",
        "\n",
        "\n",
        "quit = False\n",
        "for i in range(total_file_cnt):\n",
        "  if quit:\n",
        "    break\n",
        "\n",
        "  if file2image[str(input_files[i])].shape[0] > 1000 or file2image[str(input_files[i])].shape[1] > 1000:\n",
        "    small = cv2.resize(file2image[str(input_files[i])], (0,0), fx=0.5, fy=0.5)\n",
        "    cv2_imshow(small)\n",
        "  else:\n",
        "    cv2_imshow(str(file2image[input_files[i]]))\n",
        "\n",
        "  # get the full line:\n",
        "  file_name = input_files[i]\n",
        "  line_num = int(file_name.parent.stem.split(\"_\")[1])\n",
        "  print(lines[line_num])\n",
        "\n",
        "  accepted = False\n",
        "  time.sleep(1)\n",
        "\n",
        "  # have to have this sleep for the input box to show up\n",
        "  while not accepted:\n",
        "    try:\n",
        "      print(f\"img : {input_files[i]}\")\n",
        "      someInput = input(\"Accept: \")\n",
        "      if someInput == \"a\":\n",
        "        print(f\"\\tAccepted: {len(accepted_files):4} / {len(skip_files):4} / {len(delete_files):4} / {len(keep_files):4} / {total_file_cnt:4} \")\n",
        "        accepted = True\n",
        "        accepted_files.append(input_files[i])\n",
        "      elif someInput == \"d\":\n",
        "        print(f\"\\tDeleting: {len(accepted_files):4} / {len(skip_files):4} / {len(delete_files):4} / {len(keep_files):4} / {total_file_cnt:4} \")\n",
        "        delete_files.append(input_files[i])\n",
        "        accepted = True\n",
        "      elif someInput == \"s\":\n",
        "        print(f\"\\tSkipping: {len(accepted_files):4} / {len(skip_files):4} / {len(delete_files):4} / {len(keep_files):4} / {total_file_cnt:4} \")\n",
        "        skip_files.append(input_files[i])\n",
        "        accepted = True\n",
        "      elif someInput == \"k\":\n",
        "        print(f\"\\tKeeping: {len(accepted_files):4} / {len(skip_files):4} / {len(delete_files):4} / {len(keep_files):4} / {total_file_cnt:4} \")\n",
        "        keep_files.append(input_files[i])\n",
        "        accepted = True\n",
        "      elif someInput == \"q\":\n",
        "        print(\"Quitting\")\n",
        "        accepted = True\n",
        "        quit = True\n",
        "      elif someInput == \"u\":\n",
        "        print(\"Upscaling\")\n",
        "        accepted = False\n",
        "        quit = False\n",
        "        cv2_imshow(file2image[input_files[i]])\n",
        "    except Exception:\n",
        "      pass\n",
        "\n",
        "print(f\"accepted_files {len(accepted_files)}\")\n",
        "print(f\"delete_files {len(delete_files)}\")\n",
        "print(f\"skip_files {len(skip_files)}\")\n",
        "print(f\"keep_files {len(keep_files)}\")\n",
        "\n",
        "move_files_to_path(accepted_files, accepted_path)\n",
        "move_files_to_path(delete_files, deleted_path)\n",
        "move_files_to_path(skip_files, outtake_path)\n",
        "move_files_to_path(keep_files, keep_path)\n",
        "\n",
        "print(os.listdir(PROJECT_PATH))\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tqeYc-AfqeKF",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title get the final prompt votes\n",
        "\n",
        "\n",
        "from pathlib import Path\n",
        "import os\n",
        "import glob\n",
        "import json\n",
        "from collections import defaultdict\n",
        "import time\n",
        "\n",
        "!pip install unidecode\n",
        "\n",
        "from unidecode import unidecode\n",
        "\n",
        "min_count_to_show = 4 #@param\n",
        "\n",
        "accepted_path = PROJECT_PATH / f\"Accepted\"\n",
        "deleted_path = PROJECT_PATH / f\"Deleted\"\n",
        "skipped_path = PROJECT_PATH / f\"Skips\"\n",
        "kept_path = PROJECT_PATH / f\"Keepers\"\n",
        "\n",
        "\n",
        "prompt_gain = {}\n",
        "\n",
        "def prompt_counter_from_path(path, prompt_type):\n",
        "  print(f\"getting prompt counts for ({prompt_type}) from {path}\")\n",
        "  files = [str(f)\n",
        "               for f in map(Path, sorted(glob.glob(f\"{path}/*\")))\n",
        "               if f.is_file() and f.suffix in [\".jpg\", \".png\"]\n",
        "               ]\n",
        "  print(f\"got {len(files)} from {path}\")\n",
        "\n",
        "  prompt_counter = defaultdict(int)\n",
        "  prompt_rank = defaultdict(int)\n",
        "\n",
        "  for f in files:\n",
        "    # get the json file of the image\n",
        "    file_path = Path(f)\n",
        "    core_file_str = str(file_path.stem)\n",
        "    timestamp = core_file_str.split(\"_\")[0]\n",
        "    json_file = file_path.parent / f\"{timestamp}_settings.json\"\n",
        "    #print(f\"json: {json_file}\")\n",
        "    try:\n",
        "      with open(json_file) as json_file_open:\n",
        "        params = json.load(json_file_open)\n",
        "        # everything is a list now\n",
        "        for k in params['core_prompt_dict'].keys():\n",
        "          if k != prompt_type:\n",
        "            continue\n",
        "          # print(f\"k: {k}\")\n",
        "          for r, p in enumerate(params['core_prompt_dict'][k]):\n",
        "            if type(p) == str:\n",
        "              p = unidecode(p.strip())\n",
        "            else:\n",
        "              pass\n",
        "            # print(f\"p: {p} at {r}\")\n",
        "            if p:\n",
        "              prompt_counter[p] += 1.0\n",
        "              prompt_rank[p] += float(r)\n",
        "    except Exception as e:\n",
        "          print(e)\n",
        "          time.sleep(10)\n",
        "\n",
        "  return prompt_counter, prompt_rank, prompt_counter\n",
        "\n",
        "def param_counter_from_path(path, param):\n",
        "  # get counts of the param\n",
        "  print(f\"getting sampler counts for {param} from {path}\")\n",
        "  files = [str(f)\n",
        "               for f in map(Path, sorted(glob.glob(f\"{path}/*\")))\n",
        "               if f.is_file() and f.suffix in [\".jpg\", \".png\"]\n",
        "               ]\n",
        "  print(f\"param_counter_from_path: got {len(files)} from {path}\")\n",
        "\n",
        "  sampler_counter = defaultdict(int)\n",
        "\n",
        "  for f in files:\n",
        "    # get the json file of the image\n",
        "    file_path = Path(f)\n",
        "    core_file_str = str(file_path.stem)\n",
        "    timestamp = core_file_str.split(\"_\")[0]\n",
        "    json_file = file_path.parent / f\"{timestamp}_settings.json\"\n",
        "    #print(f\"json: {json_file}\")\n",
        "    try:\n",
        "      with open(json_file) as json_file_open:\n",
        "        params = json.load(json_file_open)\n",
        "        try:\n",
        "          s = params[param]\n",
        "          sampler_counter[s] += 1.0\n",
        "        except KeyError as e2:\n",
        "          pass\n",
        "    except Exception as e:\n",
        "          print(e)\n",
        "          time.sleep(10)\n",
        "\n",
        "  return sampler_counter\n",
        "\n",
        "\n",
        "def line_counter_from_path(path):\n",
        "  # get counts of the line\n",
        "  print(f\"getting line counts from {path}\")\n",
        "  files = [str(f)\n",
        "               for f in map(Path, sorted(glob.glob(f\"{path}/*\")))\n",
        "               if f.is_file() and f.suffix in [\".jpg\", \".png\"]\n",
        "               ]\n",
        "  print(f\"got {len(files)} from {path}\")\n",
        "\n",
        "  line_counter = defaultdict(int)\n",
        "\n",
        "  for f in files:\n",
        "    # get the json file of the image\n",
        "    file_path = Path(f)\n",
        "    core_file_str = str(file_path.stem)\n",
        "    timestamp = core_file_str.split(\"_\")[0]\n",
        "    json_file = file_path.parent / f\"{timestamp}_settings.json\"\n",
        "    #print(f\"json: {json_file}\")\n",
        "    try:\n",
        "      with open(json_file) as json_file_open:\n",
        "        params = json.load(json_file_open)\n",
        "        # print(params)\n",
        "        if 'line_number' in params:\n",
        "          s = params['line_number'] # for here, just one\n",
        "          line_counter[s] += 1.0\n",
        "        else:\n",
        "          line_counter[\"_\"] += 1.0\n",
        "          continue\n",
        "    except Exception as e:\n",
        "      print(f\"line_counter_path {json_file}\\n{e}\")\n",
        "      time.sleep(10)\n",
        "\n",
        "  return line_counter\n",
        "\n",
        "# get the line counts\n",
        "print(f\"\\n\\nLINES\")\n",
        "\n",
        "line_pos = {}\n",
        "line_cnt = {}\n",
        "neg_line = line_counter_from_path(deleted_path)\n",
        "pos_line = line_counter_from_path(accepted_path)\n",
        "skip_line = line_counter_from_path(skipped_path)\n",
        "keep_line = line_counter_from_path(kept_path)\n",
        "\n",
        "all_lines = set(list(neg_line.keys()) +\n",
        "                list(pos_line.keys()) +\n",
        "                list(skip_line.keys()) +\n",
        "                list(keep_line.keys())\n",
        "                )\n",
        "\n",
        "# get the sorted proportionate samplers\n",
        "print(f\"all_lines: {all_lines} \")\n",
        "sampler_pos = {}\n",
        "for k in all_lines:\n",
        "\n",
        "  try:\n",
        "    vpos = pos_line[k] if k in pos_line else 0.0\n",
        "    vneg = neg_line[k] if k in neg_line else 0.0\n",
        "    spos = skip_line[k] if k in skip_line else 0.0\n",
        "    kneg = keep_line[k] if k in keep_line else 0.0\n",
        "    line_cnt[k] = vpos + vneg + spos + kneg\n",
        "    # positive is accepted + skipped\n",
        "    line_pos[k] = 1.0*(vpos + spos) / (vpos + vneg + spos + kneg)\n",
        "  except:\n",
        "    line_pos[k] = 1.0\n",
        "    print(f\"line_pos: no neg {k} \")\n",
        "\n",
        "#print(sampler_pos)\n",
        "\n",
        "sorted_line_pos = list((k,v) for k, v\n",
        "                      in sorted(line_pos.items(),\n",
        "                                key=lambda x: x[1], reverse=True)\n",
        ")\n",
        "print(f\"sorted_line_pos: {sorted_line_pos}\")\n",
        "for k, v in sorted(sorted_line_pos,\n",
        "                    key=lambda x: x[1], reverse=False):\n",
        "  if type(k) == int:\n",
        "    # have to ignore the unmatchable\n",
        "    # print(f\"k: {k}\")\n",
        "    # print(f\"{line_pos[k]}\")\n",
        "    # print(f\"{line_cnt[k]}\")\n",
        "    print(f\"{k:2}: {line_pos[k]:4.3f} / {int(line_cnt[k]):4} :: {lines[k]}\")\n",
        "  else:\n",
        "    print(f\"{k:2}: {line_pos[k]:4.3f} / {int(line_cnt[k]):4} :: {k}\")\n",
        "\n",
        "\n",
        "##############################\n",
        "print(f\"\\n\\nPROMPTS\")\n",
        "for prompt_type in all_prompt_types:\n",
        "  print(f\"\\n\\nPROMPT: {prompt_type}\")\n",
        "\n",
        "  neg_counter, _, _ = prompt_counter_from_path(deleted_path, prompt_type)\n",
        "  pos_counter, pos_rank, pos_count = prompt_counter_from_path(accepted_path, prompt_type)\n",
        "\n",
        "  skip_counter, skip_rank, skip_count = prompt_counter_from_path(skipped_path, prompt_type)\n",
        "  keep_counter, keep_rank, keep_count = prompt_counter_from_path(kept_path, prompt_type)\n",
        "\n",
        "  # TODO: deal with integer, non-str prompts\n",
        "  sorted_neg_prompts = list((k,v) for k, v in sorted(neg_counter.items(), key=lambda x: x[1], reverse=True))\n",
        "  print(f\"neg prompts: {sorted_neg_prompts}\")\n",
        "  neg_prompt_string = \", \".join([str(k) for k,v in sorted_neg_prompts[:15]])\n",
        "  print(f\"neg: {neg_prompt_string}\")\n",
        "\n",
        "  sorted_pos_prompts = list((k,v) for k, v in sorted(pos_counter.items(), key=lambda x: x[1], reverse=True))\n",
        "  print(f\"pos prompts: {sorted_pos_prompts}\")\n",
        "  pos_prompt_string = \", \".join([str(k) for k,v in sorted_pos_prompts[:15]])\n",
        "  print(f\"pos: {pos_prompt_string}\")\n",
        "\n",
        "  # get the highest and lowest proportionate prompts\n",
        "  prop_pos = {}\n",
        "  total_cnt = {}\n",
        "\n",
        "  max_count = 0\n",
        "\n",
        "  all_keys = set(list(neg_counter.keys()) +\n",
        "                 list(pos_counter.keys()) +\n",
        "                 list(skip_counter.keys()) +\n",
        "                 list(keep_counter.keys())\n",
        "                 )\n",
        "\n",
        "  for k in all_keys:\n",
        "    try:\n",
        "      vneg = neg_counter[k] if k in neg_counter else 0.0\n",
        "      vpos = pos_counter[k] if k in pos_counter else 0.0\n",
        "      spos = skip_counter[k] if k in skip_counter else 0.0\n",
        "      kneg = keep_counter[k] if k in keep_counter else 0.0\n",
        "      total_cnt[k] = vpos + vneg + spos + kneg\n",
        "      if total_cnt[k] > max_count:\n",
        "        max_count =  total_cnt[k]\n",
        "\n",
        "      # new 6/2023: use skips in numerator\n",
        "      prop_pos[k] = 1.0*(vpos + spos) / total_cnt[k]\n",
        "    except:\n",
        "      prop_pos[k] = 1.0\n",
        "      print(f\"prop_pos: no neg {k} \")\n",
        "\n",
        "  pt = prompt_type\n",
        "  cur_prompt_gain = {}\n",
        "\n",
        "  # make sure all the known ones at least get 1\n",
        "  # so there's data at the beginning when not everything has been shown yet\n",
        "  for k in eval(pt):\n",
        "    cur_prompt_gain[k] = 1\n",
        "\n",
        "  for k, v in sorted(prop_pos.items(), key=lambda x: (x[1], total_cnt[x[0]], x[0]), reverse=False):\n",
        "    if k not in eval(pt):\n",
        "      continue\n",
        "\n",
        "    cur_prompt_gain[k] = 1 + int(prop_pos[k] * max_count)\n",
        "\n",
        "    if total_cnt[k] >= min_count_to_show:\n",
        "      print(f\"{k:50}: gain={cur_prompt_gain[k]}, {prop_pos[k]:4.3f} / {int(total_cnt[k]):4}\")\n",
        "\n",
        "  # add to overall gain\n",
        "  prompt_gain[pt] = cur_prompt_gain\n",
        "  print(f\"{pt}: total in prompt_gain: {len(prompt_gain[pt])}\")\n",
        "\n",
        "# save the prompt gain\n",
        "gain_path = PROJECT_PATH / \"prompt_gain.pkl\"\n",
        "import pickle\n",
        "\n",
        "print(f\"Saving gain to {gain_path}\")\n",
        "with open(gain_path, 'wb') as handle:\n",
        "    pickle.dump(prompt_gain, handle, protocol=pickle.HIGHEST_PROTOCOL)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zYAlIJviW4ps"
      },
      "source": [
        "# Sequence exploration from accepted images (GPU required)\n",
        "**Use the 8 step model at this stage and beyond**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5owwhP6kOUEf"
      },
      "outputs": [],
      "source": [
        "#@title Extend a sequence from accepted images (adjust the order of the line numbers below)\n",
        "\n",
        "#@markdown NOTE: you must open this cell and replace `lines_in_difficult_order` with the numbers in the `LINES` output from the final prompt votes in the cell above.\n",
        "\n",
        "\n",
        "# Go through the accepted images\n",
        "# See if the sequence folder already exists\n",
        "# if not, create the folder and read the parameters\n",
        "# go through each line\n",
        "\n",
        "from pathlib import Path\n",
        "import os, re, gc\n",
        "import glob\n",
        "import time\n",
        "import json\n",
        "from types import SimpleNamespace\n",
        "from IPython import display\n",
        "\n",
        "#@markdown if true, start from SeqAccepted\n",
        "second_round_or_more = False #@param{type:\"boolean\"}\n",
        "\n",
        "kill_when_done = False #@param{type:\"boolean\"}\n",
        "\n",
        "#@markdown much faster if you don't show images\n",
        "show_images = False #@param{type:\"boolean\"}\n",
        "\n",
        "\n",
        "# move the accepted directories to the review path\n",
        "if second_round_or_more:\n",
        "  old_accepted_dirs = [f\n",
        "                for f in map(Path, sorted(glob.glob(f\"{sequence_accepted_path}/*\")))\n",
        "                if f.is_dir()\n",
        "                ]\n",
        "  print(f\"Moving {len(old_accepted_dirs)} : {old_accepted_dirs}\")\n",
        "\n",
        "  # move these to the review path\n",
        "  for d in old_accepted_dirs:\n",
        "    print(f\"moving: {d}\")\n",
        "    d.rename(sequence_review_path / d.name)\n",
        "\n",
        "  # get the settings files from within those directories with an underscore\n",
        "  # get the dirs\n",
        "  accepted_dirs = [f\n",
        "                for f in map(Path, sorted(glob.glob(f\"{sequence_review_path}/*\")))\n",
        "                if f.is_dir()\n",
        "                ]\n",
        "  print(f\"found {len(accepted_dirs)} review_paths\")\n",
        "\n",
        "  # get the most recent settings file from each dir\n",
        "  accepted_files = []\n",
        "\n",
        "  for d in accepted_dirs:\n",
        "    settings_file = list(sorted(d.glob(\"*_settings.json\")))[-1]\n",
        "    print(f\"adding settings {len(accepted_files)}: {settings_file}\")\n",
        "    accepted_files.append(settings_file)\n",
        "  check_subdirs_only = True\n",
        "\n",
        "else:\n",
        "  # get the settings from the original accepted images\n",
        "  accepted_files = [f\n",
        "                for f in map(Path, sorted(glob.glob(f\"{accepted_path}/*\")))\n",
        "                if f.is_file() and f.suffix in [\".json\"]\n",
        "                ]\n",
        "  check_subdirs_only = False\n",
        "\n",
        "print(f\"accepted files ({len(accepted_files)}): {accepted_files}\")\n",
        "\n",
        "# to minimize sequence testing\n",
        "# take these from the final prompt votes\n",
        "\n",
        "lines_in_difficult_order = [\n",
        " 0,1\n",
        "\n",
        "\n",
        "]\n",
        "\n",
        "#@markdown how many lines to test, go: 2, 4, 8, 16, -1 (all)\n",
        "max_lines_out = -1 #@param{type:\"integer\"}\n",
        "\n",
        "if max_lines_out == -1:\n",
        "  max_lines_out = len(lines_in_difficult_order)\n",
        "\n",
        "lines_to_use = lines_in_difficult_order[:max_lines_out]\n",
        "print(f\"lines_to_use ({len(lines_to_use)}): {lines_to_use}\")\n",
        "\n",
        "max_accepted_files = -1 #@param{type:\"integer\"}\n",
        "\n",
        "if max_accepted_files == -1:\n",
        "  max_accepted_files = len(accepted_files)\n",
        "\n",
        "accepted_files = accepted_files[:max_accepted_files]\n",
        "print(f\"accepted_files ({len(accepted_files)}): {accepted_files}\")\n",
        "\n",
        "# TODO: test different aspect ratios\n",
        "# if set, the clear the line numbers and redo\n",
        "\n",
        "\n",
        "\n",
        "def sequence_exists(sequence_dir):\n",
        "  # check the possible paths\n",
        "\n",
        "  in_review_path_already = False\n",
        "  # TODO: need to double check this\n",
        "  outdir = sequence_review_path / sequence_dir.name\n",
        "  # print(f\"Checking: {outdir}\")\n",
        "  if outdir.is_dir():\n",
        "    in_review_path_already = True\n",
        "\n",
        "  # TODO: need to double check this\n",
        "  outdir = sequence_accepted_path / sequence_dir.name\n",
        "  # print(f\"Checking: {outdir}\")\n",
        "  if outdir.is_dir():\n",
        "    if in_review_path_already:\n",
        "      print(f\"WARNING: accepted IN REVIEW: {outdir}\")\n",
        "    print(f\"WARNING: sequence in accepted: {outdir}\")\n",
        "    return True\n",
        "\n",
        "  # TODO: need to double check this\n",
        "  outdir = sequence_deleted_path / sequence_dir.name\n",
        "  # print(f\"Checking: {outdir}\")\n",
        "  if outdir.is_dir():\n",
        "    if in_review_path_already:\n",
        "      print(f\"WARNING: deleted IN REVIEW: {outdir}\")\n",
        "    print(f\"WARNING: sequence in deleted: {outdir}\")\n",
        "    return True\n",
        "\n",
        "  # TODO: outtakes are now just files\n",
        "  # tODO: does this work?\n",
        "  outdir = outtake_path / f\"{sequence_dir.name.split('_')[0]}_settings.txt\"\n",
        "  # print(f\"Checking: {outdir}\")\n",
        "  if outdir.is_file():\n",
        "    if in_review_path_already:\n",
        "      print(f\"WARNING: outtake IN REVIEW: {outdir}\")\n",
        "    print(f\"WARNING: sequence in outtake_path: {outdir}\")\n",
        "    return True\n",
        "\n",
        "  return in_review_path_already\n",
        "\n",
        "exceptions = []\n",
        "\n",
        "for a in range(len(accepted_files)):\n",
        "# for a in range(4):\n",
        "\n",
        "  if second_round_or_more:\n",
        "    # the directory is the parent of the file,\n",
        "\n",
        "    settings_file = accepted_files[a]\n",
        "    try:\n",
        "      image_settings = get_settings_from_file(settings_file)\n",
        "    except Exception as e:\n",
        "      print(e)\n",
        "      exceptions.append(e)\n",
        "      continue\n",
        "\n",
        "    outdir = accepted_files[a].parent\n",
        "    cur_seed = image_settings.seed\n",
        "    cur_timestamp = outdir.name.split(\"_\")[0]\n",
        "    cur_line_numbers = image_settings.line_numbers\n",
        "\n",
        "  else:\n",
        "    # the directory must be created in the for_review\n",
        "    settings_file = accepted_files[a]\n",
        "    image_settings = get_settings_from_file(settings_file)\n",
        "    cur_seed = image_settings.seed\n",
        "    cur_timestamp = settings_file.name.split(\"_\")[0]\n",
        "\n",
        "    # a new directory in review if starting from single images\n",
        "    outdir = sequence_review_path / f\"{cur_timestamp}_{cur_seed}\"\n",
        "    if sequence_exists(sequence_dir=outdir):\n",
        "      print (f\"EXISTS: {outdir}\")\n",
        "      continue\n",
        "    cur_line_numbers = []\n",
        "\n",
        "  print(f\"\\n\\n({a:4}/{len(accepted_files):4}) :: dir: {outdir}\")\n",
        "  print(f\"settings_file: {settings_file}\")\n",
        "\n",
        "  print(f\"time: {cur_timestamp}\")\n",
        "  print(f\"seed: {cur_seed}\")\n",
        "\n",
        "  print(f\"cur_line_numbers: {cur_line_numbers}\")\n",
        "\n",
        "  # get an actual dict from the sn\n",
        "  temp_dict = json.loads(json.dumps(image_settings, default=lambda s: vars(s)))\n",
        "\n",
        "  # sets the global used by get_prompt_for_line below\n",
        "  cur_core_prompt_dict = temp_dict['core_prompt_dict']\n",
        "\n",
        "  # and turn it back\n",
        "  image_settings.core_prompt_dict = cur_core_prompt_dict\n",
        "\n",
        "  line_numbers = cur_line_numbers\n",
        "  new_line_numbers = []\n",
        "\n",
        "  # so the images show up in order, even if lines skipped\n",
        "  for line_number in range(len(lines)):\n",
        "\n",
        "    if line_number not in lines_to_use:\n",
        "      continue\n",
        "    if line_number in cur_line_numbers:\n",
        "      # print(f\"already run: {line_number}\")\n",
        "      continue\n",
        "    new_line_numbers.append(line_number)\n",
        "    line_numbers.append(line_number)\n",
        "\n",
        "  if not new_line_numbers:\n",
        "    print(f\"already extended: {settings_file}\")\n",
        "    continue\n",
        "\n",
        "  print(f\"round: {a}; outdir: {outdir}; \")\n",
        "  print(f\"new_line_numbers: {new_line_numbers}\")\n",
        "\n",
        "\n",
        "  # need to make just one new settings file for all the images\n",
        "  # so add prompts, image_files (plural) to the settings\n",
        "\n",
        "  image_settings.line_numbers = line_numbers # all of them\n",
        "\n",
        "  # uses cur_core_prompt_dict to create the actual prompt\n",
        "  image_settings.prompts = [get_prompt_for_line(n) for n in line_numbers]\n",
        "\n",
        "  print(f\"scale: {image_settings.guidance_scale}; steps:{image_settings.steps}\")\n",
        "  print(f\"final image_settings: {image_settings}\")\n",
        "\n",
        "  # save one setting for the whole directory\n",
        "  save_settings(image_settings, settings_file.name, outdir)\n",
        "\n",
        "  # go throught the prompts and files\n",
        "  cur_image_settings = image_settings\n",
        "  for n, line_num in enumerate(line_numbers):\n",
        "    if line_num not in new_line_numbers:\n",
        "      continue\n",
        "\n",
        "    print(f\"line {line_num}: \")\n",
        "    # commment out for testing\n",
        "    cur_image_settings.prompt = image_settings.prompts[n]\n",
        "    print(f\"cur_image_setting: {cur_image_settings}\")\n",
        "    output_image = get_one_image(cur_image_settings)\n",
        "\n",
        "    if show_images:\n",
        "      display.display(output_image)\n",
        "    save_image_with_name(output_image, f\"line_{line_num:02}\", outdir)\n",
        "\n",
        "  # clean up unused memory\n",
        "  gc.collect()\n",
        "  torch.cuda.empty_cache()\n",
        "\n",
        "print(f\"Exceptions: {exceptions}\")\n",
        "\n",
        "# kill switch when done\n",
        "if kill_when_done:\n",
        "  print(\"DONE!  Killing runtime\")\n",
        "  from google.colab import runtime\n",
        "  runtime.unassign()\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Azxj1uWqfoEJ"
      },
      "source": [
        "\n",
        "#**Sequence Voting (no GPU required)**\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title sequence voting functions\n",
        "\n",
        "# show them to the user, if not in the file\n",
        "# set A, S, D (accept, skip, delete) assessments\n",
        "from IPython.core.interactiveshell import StrDispatch\n",
        "\n",
        "from pathlib import Path\n",
        "import glob\n",
        "import time\n",
        "import os\n",
        "import json\n",
        "from IPython.core.interactiveshell import InteractiveShell\n",
        "InteractiveShell.ast_node_interactivity = \"all\"\n",
        "from IPython import display\n",
        "from collections import defaultdict\n",
        "from PIL import Image, ImageFont, ImageDraw\n",
        "\n",
        "def get_lines_from_path(path):\n",
        "  # lines_to_use = [14, 2, 15, 11]\n",
        "  settings_file = list(sorted(path.glob(\"*_settings.json\")))[-1]\n",
        "  print(f\"settings_file: {settings_file}\")\n",
        "  settings = get_settings_from_file(settings_file)\n",
        "\n",
        "  # lines_to_use = settings.line_numbers\n",
        "  # image files are more informative\n",
        "  lines_to_use = settings.prompts\n",
        "  line_numbers = settings.line_numbers\n",
        "\n",
        "  # the images are shown in original line order\n",
        "  # so we need to reorder them\n",
        "\n",
        "  ordered_lines = sorted(line_numbers)\n",
        "  out_titles = []\n",
        "  for n in ordered_lines:\n",
        "    pos = line_numbers.index(n)\n",
        "    out_titles.append(lines[pos])\n",
        "\n",
        "  print(f\"get_image_titles_from_path ({len(out_titles)}): {out_titles}\")\n",
        "  return out_titles\n",
        "\n",
        "\n",
        "def image_grid_list(file_list, titles, max_height=960, title_replacements=[]):\n",
        "\n",
        "  row2imgs = defaultdict(list)\n",
        "\n",
        "  max_rows = 1\n",
        "  # shoot for 3 rows max\n",
        "  if len(file_list) <= 4:\n",
        "    max_cols = len(file_list)\n",
        "  else:\n",
        "    max_cols = 4 if len(file_list) <= 12 else len(file_list) // 4\n",
        "  max_w = 0\n",
        "  max_h = 0\n",
        "\n",
        "\n",
        "  for fnum, f in enumerate(file_list):\n",
        "    img = Image.open(f)\n",
        "    print(f\"plot_images on row {max_rows}: fnum {fnum} file: {f}\")\n",
        "\n",
        "    max_w = max(max_w, img.size[0])\n",
        "    max_h = max(max_h, img.size[1])\n",
        "\n",
        "    row2imgs[max_rows].append(img)\n",
        "    if fnum > 0 and (fnum+1) % max_cols == 0:\n",
        "      max_rows += 1\n",
        "\n",
        "  if len(row2imgs[max_rows]) == 0:\n",
        "    max_rows -= 1\n",
        "\n",
        "  font_size = 35\n",
        "  max_title_length = int(max_w / int(font_size / 1.5))\n",
        "\n",
        "  # print(f\"row2imgs ({max_rows}, {max_cols}): {row2imgs}\")\n",
        "  print(f\"row2imgs ({max_rows}, {max_cols})\")\n",
        "  print(f\"max_w: {max_w}, max_h: {max_h}\")\n",
        "\n",
        "  grid = Image.new('RGB', size=(max_cols*max_w, max_rows*max_h))\n",
        "  grid_w, grid_h = grid.size\n",
        "  print(f\"grid_size: {grid.size}\")\n",
        "\n",
        "  # use this if it the other cannot be found\n",
        "  # font = ImageFont.load_default()\n",
        "  font = ImageFont.truetype(r'/usr/share/fonts/truetype/liberation/LiberationMono-Regular.ttf', font_size)\n",
        "\n",
        "  # add the line and position. inupper left\n",
        "  draw = ImageDraw.Draw(grid)\n",
        "\n",
        "  if len(file_list) == len(titles):\n",
        "    add_title = True\n",
        "  else:\n",
        "    add_title = False\n",
        "\n",
        "  i = 0\n",
        "  for row, r in enumerate(row2imgs.keys()):\n",
        "    for col, img in enumerate(row2imgs[r]):\n",
        "      # print(f\"{row}, {col} pasting at: {row*max_w} , {col*max_h}\")\n",
        "      grid.paste(img, box=(col*max_w, row*max_h))\n",
        "      cur_title = \"\"\n",
        "      if add_title:\n",
        "        if title_replacements:\n",
        "          cur_title = titles[i]\n",
        "          for r in title_replacements:\n",
        "            cur_title = cur_title.replace(r,'')\n",
        "            cur_title = cur_title.strip()[:max_title_length]\n",
        "        else:\n",
        "          cur_title = titles[i][:max_title_length]\n",
        "      i += 1\n",
        "\n",
        "      draw.text((col*max_w +5, row*max_h+5), f\"{cur_title}\", fill='white', font=font)\n",
        "\n",
        "  if grid_h > max_height:\n",
        "    # shrink down so all the images can be seen at once\n",
        "    ratio = max_height / grid_h\n",
        "    new_h = max_height\n",
        "    new_w = int(ratio * grid_w)\n",
        "    print(f\"resizing to ({new_w}, {new_h})\")\n",
        "    grid = grid.resize((new_w, new_h), Image.Resampling.LANCZOS)\n",
        "\n",
        "  return grid\n",
        "\n",
        "\n",
        "def remove_empty_dir(dir):\n",
        "  #remove directory if empty\n",
        "  if len(os.listdir(dir)) == 0:\n",
        "    try:\n",
        "      print(f\"trying to remove empty dir: {dir}\")\n",
        "      os.rmdir(dir)\n",
        "      print(f\"\\tremoved empty dir: {dir}\")\n",
        "    except Exception as e:\n",
        "      print(e)\n",
        "      pass\n",
        "\n",
        "def move_dir_files_to_path(in_path, out_path):\n",
        "  input_files = [f\n",
        "               for f in map(Path, sorted(glob.glob(f\"{in_path}/*\")))\n",
        "               if f.is_file()\n",
        "               ]\n",
        "\n",
        "  print(f\"moving {len(input_files)} to {out_path}\")\n",
        "  # move the skipped files\n",
        "  for file_path in input_files:\n",
        "    try:\n",
        "      file_path.rename(out_path / file_path.name)\n",
        "      #print(f\"move {f} TO {outtake_path / file_path.name}\")\n",
        "    except:\n",
        "      print(f\"ERROR moving: {f}\")\n",
        "      exit()\n",
        "\n",
        "  remove_empty_dir(in_path)\n"
      ],
      "metadata": {
        "id": "OUMm75g7zrgh",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Sequence Voting v2\n",
        "\n",
        "\n",
        "###########################\n",
        "review_dirs = [\n",
        "          f for f in map(Path, sorted(glob.glob(f\"{sequence_review_path}/*\")))\n",
        "               if f.is_dir()  and \"_\" in str(f)\n",
        "    ]\n",
        "\n",
        "print(f\"review_dirs ({len(review_dirs)}): {review_dirs}\")\n",
        "\n",
        "quit = False\n",
        "\n",
        "total_dirs = len(review_dirs)\n",
        "\n",
        "accept_cnt = 0\n",
        "skip_cnt = 0\n",
        "delete_cnt = 0\n",
        "\n",
        "# if there's some repetition among the lines\n",
        "title_replacements = []\n",
        "\n",
        "for i, sd in enumerate(review_dirs):\n",
        "  if quit:\n",
        "    break\n",
        "\n",
        "  if i % 10 == 0:\n",
        "    display.clear_output(wait=True)\n",
        "\n",
        "  print(f\"\\n\\n ({i+1:2} / {total_dirs:3}) {accept_cnt:3}/{skip_cnt:3}/{delete_cnt:3}\\n{sd.name}\")\n",
        "  # lines_to_use = get_lines_from_path(sd)\n",
        "  # titles_to_use = [lines[i][0] if type(lines[i]) == list else lines[i]  for i in lines_to_use  ]\n",
        "  # titles_to_use = lines_to_use\n",
        "\n",
        "  files = [\n",
        "      f\n",
        "      for f in map(Path, sorted(glob.glob(f\"{sd}/*\")))\n",
        "      if f.is_file() and f.suffix in [\".jpg\", \".png\"]\n",
        "    ]\n",
        "\n",
        "  titles_to_use = []\n",
        "  for f in files:\n",
        "    line_num = int(f.stem.split(\"_\")[1])\n",
        "    titles_to_use.append(lines[line_num])\n",
        "\n",
        "  grid = image_grid_list(files, titles=titles_to_use, title_replacements=title_replacements)\n",
        "  display.display(grid)\n",
        "\n",
        "  accepted = False\n",
        "  time.sleep(6) # 8 needs at\n",
        "\n",
        "  # have to have this sleep for the input box to show up\n",
        "  while not accepted:\n",
        "    try:\n",
        "      someInput = input(\"Accept: \")\n",
        "      new_path = None\n",
        "      if someInput == \"a\":\n",
        "        accepted = True\n",
        "        new_path = sequence_accepted_path / sd.name\n",
        "        accept_cnt += 1\n",
        "      elif someInput == \"d\":\n",
        "        accepted = True\n",
        "        new_path = sequence_deleted_path / sd.name\n",
        "        delete_cnt += 1\n",
        "      elif someInput == \"s\":\n",
        "        accepted = True\n",
        "        # TODO: need to move these to Keepers and individual images\n",
        "        new_path = outtake_path / sd.name\n",
        "        skip_cnt += 1\n",
        "      elif someInput == \"q\":\n",
        "        print(\"Quitting\")\n",
        "        accepted = True\n",
        "        quit = True\n",
        "      if new_path:\n",
        "        print(f\"moving {sd}  to {new_path}\")\n",
        "        sd.rename(new_path)\n",
        "\n",
        "    except Exception as e:\n",
        "      print(e)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "psx6EZgXG7Hj",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title adjust image settings in sequences from skips to accepted (CPU)\n",
        "\n",
        "# go through the skip sequences (folders in skips)\n",
        "# decide if they can be recovered by adding to negative prompts\n",
        "# if so, then add (from input) to the negative prompt list, remove the line numbers\n",
        "# put the settings file in to SeqAccepted for the next round\n",
        "import os\n",
        "from datetime import datetime\n",
        "\n",
        "#@markdown to restrict what to review: \"%Y-%m-%d %H:%M:%S\", e.g., 2024-01-26 12:00:00\n",
        "start_time_text = \"\" #@param{\"type\":\"string\"}\n",
        "\n",
        "if start_time_text:\n",
        "  start_time = datetime.strptime(start_time_text,\"%Y-%m-%d %H:%M:%S\")\n",
        "else:\n",
        "  start_time = None\n",
        "\n",
        "print(f\"start_time: {start_time}\")\n",
        "\n",
        "###########################\n",
        "# get skipped sequences\n",
        "skip_dirs = [\n",
        "          f for f in map(Path, sorted(glob.glob(f\"{outtake_path}/*\")))\n",
        "               if f.is_dir()  # and \"_\" in str(f)\n",
        "    ]\n",
        "\n",
        "print(f\"skip_dirs ({len(skip_dirs)}): {skip_dirs}\")\n",
        "\n",
        "quit = False\n",
        "\n",
        "total_dirs = len(skip_dirs)\n",
        "\n",
        "accept_cnt = 0\n",
        "\n",
        "# if there's some repetition among the lines\n",
        "title_replacements = []\n",
        "\n",
        "for i, sd in enumerate(skip_dirs):\n",
        "  if quit:\n",
        "    break\n",
        "\n",
        "  dir_time = datetime.fromtimestamp(os.path.getmtime(sd))\n",
        "  print(f\"dir_time: {dir_time}\")\n",
        "  if start_time and dir_time < start_time:\n",
        "    print(f\"APPROVED from {os.path.getctime(sd)}: {sd}\")\n",
        "    continue\n",
        "\n",
        "\n",
        "  if i % 10 == 0:\n",
        "    display.clear_output(wait=True)\n",
        "\n",
        "  print(f\"\\n\\n ({i+1:2} / {total_dirs:3}) {accept_cnt:3}\\n{sd.name}\")\n",
        "\n",
        "  files = [\n",
        "      f\n",
        "      for f in map(Path, sorted(glob.glob(f\"{sd}/*\")))\n",
        "      if f.is_file() and f.suffix in [\".jpg\", \".png\"]\n",
        "    ]\n",
        "\n",
        "  titles_to_use = []\n",
        "  for f in files:\n",
        "    line_num = int(f.stem.split(\"_\")[1])\n",
        "    titles_to_use.append(lines[line_num])\n",
        "\n",
        "  grid = image_grid_list(files, titles=titles_to_use, title_replacements=title_replacements)\n",
        "  display.display(grid)\n",
        "\n",
        "  accepted = False\n",
        "  # have to have this sleep for the input box to show up\n",
        "  time.sleep(6) # 8 needs at\n",
        "\n",
        "  negative_terms = None\n",
        "  while not accepted:\n",
        "\n",
        "    try:\n",
        "      someInput = input(\"new neg?: \")\n",
        "      new_path = None\n",
        "      if someInput == \"s\":\n",
        "        print(\"Skipping\")\n",
        "        accepted = True\n",
        "      elif someInput == \"q\":\n",
        "        print(\"Skipping\")\n",
        "        accepted = True\n",
        "        quit = True\n",
        "      elif len(someInput) > 3:\n",
        "        # get the negative terms to add\n",
        "        negative_terms = someInput\n",
        "        accepted = True\n",
        "\n",
        "    except Exception as e:\n",
        "      print(e)\n",
        "\n",
        "  if negative_terms:\n",
        "    settings_file = list(sorted(sd.glob(\"*_settings.json\")))[-1]\n",
        "    print(f\"using settings: {settings_file}\")\n",
        "    image_settings = get_settings_from_file(settings_file)\n",
        "\n",
        "    # get a new seed to keep track of the change\n",
        "    image_settings.seed = random.randint(1,2147483647)\n",
        "    cur_timestamp = settings_file.name.split(\"_\")[0]\n",
        "\n",
        "    # get an actual dict from the sn\n",
        "    temp_dict = json.loads(json.dumps(image_settings, default=lambda s: vars(s)))\n",
        "\n",
        "    # sets the global used by get_prompt_for_line below\n",
        "    cur_core_prompt_dict = temp_dict['core_prompt_dict']\n",
        "\n",
        "    # adjust the core promp_dict\n",
        "    print(f\"current negative_qualities: {cur_core_prompt_dict['negative_qualities']}\")\n",
        "\n",
        "    new_initial_terms = [t.strip() for t in negative_terms.split(\",\")]\n",
        "    print(f\"new_initial_terms: {new_initial_terms}\")\n",
        "\n",
        "    cur_core_prompt_dict['negative_qualities'] = new_initial_terms + cur_core_prompt_dict['negative_qualities']\n",
        "    print(f\"new negative_qualities: {cur_core_prompt_dict['negative_qualities']}\")\n",
        "\n",
        "    # and turn it back\n",
        "    image_settings.core_prompt_dict = cur_core_prompt_dict\n",
        "\n",
        "    image_settings.line_numbers = [] # empty the line numbers\n",
        "\n",
        "    outdir = sequence_accepted_path / f\"{cur_timestamp}_{image_settings.seed}\"\n",
        "\n",
        "    # save one setting for the whole directory\n",
        "    save_settings(image_settings, settings_file.name, outdir)\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "yNW6drpCx-f-",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 3. Random seed testing (GPU required)"
      ],
      "metadata": {
        "id": "Sth4WyWm5kWk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Given an accepted settings file, randomize the seeds for each line\n",
        "\n",
        "# Go through the accepted images\n",
        "# create 10 versions of each line with a different seed\n",
        "# try to find a path through the images that is consistent\n",
        "# if not, create the folder and read the parameters\n",
        "# go through each line\n",
        "\n",
        "from pathlib import Path\n",
        "import os, re, gc\n",
        "import glob\n",
        "import time\n",
        "import json\n",
        "from types import SimpleNamespace\n",
        "from IPython import display\n",
        "\n",
        "rounds = 2 #@param{type:\"integer\"}\n",
        "\n",
        "kill_when_done = False #@param{type:\"boolean\"}\n",
        "\n",
        "#@markdown much faster if you don't show images\n",
        "show_images = False #@param{type:\"boolean\"}\n",
        "\n",
        "#@markdown within SeqAccepted if you want to do just one\n",
        "settings_dir =  \"\" #@param{type:\"string\"}\n",
        "\n",
        "#@markdown put a seed here to get this one as part of the set\n",
        "original_seed = 0 #@param{type:\"integer\"}\n",
        "\n",
        "if settings_dir:\n",
        "  # do just 1\n",
        "  settings_paths = [sequence_accepted_path / settings_dir]\n",
        "else:\n",
        "  # go through all the directories in sequence_accepted_path\n",
        "  settings_paths = [\n",
        "    f\n",
        "    for f in map(Path, sorted(glob.glob(f\"{sequence_accepted_path}/*\")))\n",
        "    if f.is_dir()\n",
        "  ]\n",
        "\n",
        "print(f\"Settings paths ({len(settings_paths)}): {settings_paths}\")\n",
        "\n",
        "# if you don't want to run all of the lines adjust this\n",
        "lines_to_use = list(range(0, len(lines)))\n",
        "# or\n",
        "# lines_to_use = [0, 1, 7, 9, 10, 11, 12, 13,14, 16, 18, 19, 25, 28, 31, 36, 39, 42, 49, 54, 57, ]\n",
        "\n",
        "for settings_path in settings_paths:\n",
        "  print(f\"settings_path: {settings_path}\")\n",
        "  # get the last one\n",
        "  settings_file = list(sorted(settings_path.glob(\"*_settings.json\")))[-1]\n",
        "  print(f\"adding settings: {settings_file}\")\n",
        "\n",
        "  # get the parent's timestamp\n",
        "  # make this directory to hold the seed directories\n",
        "  settings_timestamp = settings_path.name.split(\"_\")[0]\n",
        "  print(f\"settings_timestamp: {settings_timestamp}\")\n",
        "\n",
        "  # settings timestamp tells us where the original settings came from\n",
        "  timestamp_dir = sequence_review_path / settings_timestamp\n",
        "  os.makedirs(timestamp_dir, exist_ok=True)\n",
        "  print(f\"saving seed directories under: {timestamp_dir}\")\n",
        "\n",
        "  for round in range(rounds):\n",
        "    # to prevent errors\n",
        "    if round+1 % 3==0:\n",
        "      display.clear_output(wait=True)\n",
        "\n",
        "    if original_seed and round == 0:\n",
        "      cur_seed = original_seed\n",
        "    else:\n",
        "      # new random seed\n",
        "      cur_seed = random.randint(1,4294967295) # numpy limit\n",
        "\n",
        "    # new settings format\n",
        "    image_settings = get_settings_from_file(settings_file)\n",
        "    # all the individual lines will get this same new random seed\n",
        "    image_settings.seed = cur_seed\n",
        "\n",
        "    # get an actual dict from the simplenamespace\n",
        "    temp_dict = json.loads(json.dumps(image_settings, default=lambda s: vars(s)))\n",
        "    cur_core_prompt_dict = temp_dict['core_prompt_dict']\n",
        "    # and turn it back\n",
        "    image_settings.core_prompt_dict = cur_core_prompt_dict\n",
        "\n",
        "    # so each round will have a different cur_seed and directory under timestamp_dir\n",
        "    outdir_name = f\"{settings_timestamp}_{cur_seed}\"\n",
        "    outdir = timestamp_dir / outdir_name\n",
        "    os.makedirs(outdir, exist_ok=True)\n",
        "\n",
        "    print(f\"\\nRound {round}: outdir: {outdir}; seed: {cur_seed}\")\n",
        "\n",
        "    # need to make just one new settings file for all the images\n",
        "    # so add prompts, image_files to the settings\n",
        "\n",
        "    line_numbers = list(range(0, len(lines))) # get all the lines\n",
        "    image_settings.line_numbers = line_numbers\n",
        "    image_settings.prompts = [get_prompt_for_line(n) for n in line_numbers]\n",
        "\n",
        "    print(f\"guidance_scale: {image_settings.guidance_scale}; steps:{image_settings.steps};\")\n",
        "    print(f\"final image_settings: {image_settings}\")\n",
        "\n",
        "    # save one setting for the whole directory;\n",
        "    # have to define the filename completely\n",
        "    setting_filename = f\"{outdir_name}_settings.json\"\n",
        "    save_settings(image_settings, setting_filename, outdir)\n",
        "\n",
        "    # go through the prompts and files\n",
        "    cur_image_settings = image_settings\n",
        "    for line_num in line_numbers:\n",
        "      if line_num not in lines_to_use:\n",
        "        continue\n",
        "\n",
        "      print(f\"creating image for line {line_num}: '{image_settings.prompts[line_num]}'\")\n",
        "      # commment out for testing:\n",
        "      cur_image_settings.prompt = image_settings.prompts[line_num]\n",
        "      print(f\"cur_image_setting: {cur_image_settings}\")\n",
        "\n",
        "      # if run_mode == \"Interpolation\":\n",
        "      #   # to match what happens with interpolation base_fulle\n",
        "      #   output_image = get_one_image_from_base_full(cur_image_settings)\n",
        "      # else:\n",
        "      output_image = get_one_image(cur_image_settings)\n",
        "\n",
        "      if show_images:\n",
        "        display.display(output_image)\n",
        "      save_image_with_name(output_image, f\"line_{line_num:02}\", outdir)\n",
        "\n",
        "    # clean up unused memory\n",
        "    gc.collect()\n",
        "    torch.cuda.empty_cache()\n",
        "\n",
        "# kill switch when done\n",
        "if kill_when_done:\n",
        "  print(\"DONE!  Killing runtime\")\n",
        "  from google.colab import runtime\n",
        "  runtime.unassign()\n",
        "\n"
      ],
      "metadata": {
        "id": "ZWEAA7Cj5nOU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title (if necc, do once) copy all accepted dirs into the SeqForReview subfolders (no GPU)\n",
        "from pathlib import Path\n",
        "import glob\n",
        "from distutils.dir_util import copy_tree\n",
        "\n",
        "settings_paths = [\n",
        "    f\n",
        "    for f in map(Path, sorted(glob.glob(f\"{sequence_accepted_path}/*\")))\n",
        "    if f.is_dir()\n",
        "  ]\n",
        "\n",
        "print(f\"Found {len(settings_paths)} setting_paths: {settings_paths}\")\n",
        "\n",
        "for settings_path in settings_paths:\n",
        "  settings_timestamp = settings_path.name.split(\"_\")[0]\n",
        "  print(f\"settings_timestamp: {settings_timestamp}\")\n",
        "\n",
        "  # settings timestamp tells us where the original settings came from\n",
        "  timestamp_dir = sequence_review_path / settings_timestamp\n",
        "  # save a copy in the review path\n",
        "  new_path = timestamp_dir / settings_path.name\n",
        "\n",
        "  copy_tree(str(settings_path), str(new_path))\n",
        "\n",
        "  # settings_path.copy(new_path)\n",
        "  print(f\"copied {settings_path} TO {new_path}\")\n",
        "  # print(f\"will copy {settings_path} TO {new_path}\")\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "6jSxHrtkLQ0k",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Choose the best images for each set (no GPU)\n",
        "\n",
        "# Go through the accepted images\n",
        "# create 10 versions of each line with a different seed\n",
        "# try to find a path through the images that is consistent\n",
        "# if not, create the folder and read the parameters\n",
        "# go through each line\n",
        "\n",
        "from pathlib import Path\n",
        "import os, re, gc\n",
        "import glob\n",
        "import time\n",
        "import json\n",
        "from types import SimpleNamespace\n",
        "from IPython import display\n",
        "from collections import defaultdict\n",
        "\n",
        "import cv2\n",
        "from matplotlib import pyplot as plt\n",
        "import imageio\n",
        "\n",
        "from IPython.core.interactiveshell import InteractiveShell\n",
        "InteractiveShell.ast_node_interactivity = \"all\"\n",
        "from PIL import Image, ImageFont, ImageDraw\n",
        "\n",
        "import shutil\n",
        "\n",
        "#@markdown otherwise copy chosen images to SeqFinal/{timestamp}\n",
        "just_output_seeds = False #@param{type:\"boolean\"}\n",
        "\n",
        "#@markdown otherwise do all dirs in SeqForReview\n",
        "one_timestamp = \"\" #@param{type:\"string\"}\n",
        "\n",
        "\n",
        "\n",
        "###################\n",
        "\n",
        "def image_grid_dict(file_dict, titles, max_height=720):\n",
        "  # images = []\n",
        "\n",
        "  row2imgs = defaultdict(list)\n",
        "\n",
        "  max_rows = len(file_dict.keys())\n",
        "  max_cols = 1\n",
        "  max_w = 0\n",
        "  max_h = 0\n",
        "  # Create a font object\n",
        "  # font = ImageFont.truetype('arial.ttf', 32)\n",
        "\n",
        "  for r in sorted(file_dict.keys()):\n",
        "    for fnum, f in enumerate(file_dict[r]):\n",
        "      # print(f\"plot_images: file: {f}\")\n",
        "      # display.display(Image.open(f))\n",
        "      # img = cv2.cvtColor(cv2.imread(str(f)), cv2.COLOR_BGR2RGB)\n",
        "      img = Image.open(f)\n",
        "      # print(f\"plot_images {r}: file: {f}\")\n",
        "\n",
        "      max_w = max(max_w, img.size[0])\n",
        "      max_h = max(max_h, img.size[1])\n",
        "\n",
        "      # add the line and position. inupper left\n",
        "      # draw = ImageDraw.Draw(img)\n",
        "      # draw.text((15,15), f\"{r}:{fnum}\", fill='white', size=400)\n",
        "\n",
        "      row2imgs[r].append(img)\n",
        "      if len(row2imgs[r]) > max_cols:\n",
        "        max_cols = len(row2imgs[r])\n",
        "\n",
        "  # print(f\"row2imgs ({max_rows}, {max_cols}): {row2imgs}\")\n",
        "  print(f\"row2imgs ({max_rows}, {max_cols})\")\n",
        "  print(f\"max_w: {max_w}, max_h: {max_h}\")\n",
        "\n",
        "\n",
        "  grid = Image.new('RGB', size=(max_cols*max_w, max_rows*max_h))\n",
        "  grid_w, grid_h = grid.size\n",
        "  print(f\"grid_size: {grid.size}\")\n",
        "\n",
        "  # font = ImageFont.load_default()\n",
        "  font = ImageFont.truetype(r'/usr/share/fonts/truetype/liberation/LiberationMono-Regular.ttf', 70)\n",
        "\n",
        "  # add the line and position. inupper left\n",
        "  draw = ImageDraw.Draw(grid)\n",
        "  # draw.text((15,15), f\"{r}:{fnum}\", fill='white', size=400)\n",
        "\n",
        "  for row, r in enumerate(row2imgs.keys()):\n",
        "    for col, img in enumerate(row2imgs[r]):\n",
        "      # print(f\"{row}, {col} pasting at: {row*max_w} , {col*max_h}\")\n",
        "      grid.paste(img, box=(col*max_w, row*max_h))\n",
        "      draw.text((col*max_w +5, row*max_h+5), f\"{r}:{col}\", fill='white', font=font)\n",
        "\n",
        "  # for i, img in enumerate(images):\n",
        "  #     grid.paste(img, box=(i%max_cols*max_w, i//cols*max_h))\n",
        "\n",
        "  if grid_h > max_height:\n",
        "    # shrink down so all the images can be seen at once\n",
        "    ratio = max_height / grid_h\n",
        "    new_h = max_height\n",
        "    new_w = int(ratio * grid_w)\n",
        "    print(f\"resizing to ({new_w}, {new_h})\")\n",
        "    grid = grid.resize((new_w, new_h), Image.Resampling.LANCZOS)\n",
        "\n",
        "  return grid\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "#############################################\n",
        "# 1. get all the unique set names\n",
        "print(f\"sequence_review_path: {sequence_review_path}\")\n",
        "\n",
        "if one_timestamp:\n",
        "  initial_timestamps = [str(sequence_review_path/ one_timestamp)]\n",
        "else:\n",
        "  # go through all the directories in the review path\n",
        "  initial_timestamps = list(sorted(set(\n",
        "    f.name.split(\"_\")[0]\n",
        "    for f in map(Path, sorted(glob.glob(f\"{sequence_review_path}/*\")))\n",
        "    if f.is_dir()))\n",
        "  )\n",
        "\n",
        "print(f\"initial_timestamps {len(initial_timestamps)}: {initial_timestamps}\")\n",
        "\n",
        "# how much to show at a time.\n",
        "rows_at_a_time = 3\n",
        "\n",
        "for ts_num, init_ts in enumerate(initial_timestamps):\n",
        "\n",
        "  # 2. get the whole set of images for each line number\n",
        "  print()\n",
        "  print(f\"{ts_num:02} cur_set: {init_ts}\")\n",
        "\n",
        "  # go through the directories that start with that\n",
        "  cur_versions = [f\n",
        "    for f in map(Path, sorted(glob.glob(f\"{sequence_review_path}/{init_ts}/*\")))\n",
        "    if f.is_dir()]\n",
        "\n",
        "  print(f\"\\tcur_versions ({len(cur_versions)}): {cur_versions}\")\n",
        "\n",
        "  line2files  = defaultdict(list)\n",
        "\n",
        "  # keep track of the version/seed that was shown and chosen\n",
        "  line2seed = defaultdict(list)\n",
        "\n",
        "  image_settings = None\n",
        "\n",
        "  for version_num, version in enumerate(cur_versions):\n",
        "    # get the images for each line number,\n",
        "    # check if they are all there\n",
        "    print(f\"\\t{ts_num:02} {version_num:02} Analyzing: {version}\")\n",
        "\n",
        "    # get the settings file\n",
        "    if not image_settings:\n",
        "      try:\n",
        "        # some older versions don't have a proper settings file\n",
        "        cur_settings_file = list(sorted(version.glob(f\"*_settings.json\")))[-1]\n",
        "        # cur_settings_file = sorted(version.glob(f\"*_settings.json\"))\n",
        "        print(f\"{cur_settings_file}\")\n",
        "        image_settings = get_settings_from_file(cur_settings_file)\n",
        "      except:\n",
        "        print(f\"missing settings for: {version}\")\n",
        "\n",
        "    cur_seed = version.stem.split(\"_\")[-1]\n",
        "\n",
        "\n",
        "    # need to keep track of the seed that made each chosen image\n",
        "    seeds_all = []\n",
        "\n",
        "    # get the lines in this directory\n",
        "    line_files = [\n",
        "      f\n",
        "      for f in map(Path, sorted(glob.glob(f\"{version}/*\")))\n",
        "      if f.is_file()  and f.suffix in [\".jpg\", \".png\"]\n",
        "      ]\n",
        "\n",
        "    print(f\"line_files: {len(line_files)}: {line_files}\")\n",
        "\n",
        "    # set the line to file dict\n",
        "    for line_num, lf in enumerate(line_files):\n",
        "      lf_str = lf.stem\n",
        "      lf_parts = lf_str.split(\"_\")\n",
        "      # print(f\"line_parts: {lf_parts}\")\n",
        "      line = int(lf_parts[1])\n",
        "      # print(f\"{ts_num:02} {version_num:02} {line_num}: {line:02}: {lf}\")\n",
        "      line2files[line].append(lf)\n",
        "      line2seed[line].append(cur_seed)\n",
        "\n",
        "  print()\n",
        "  total_keys_cnt = len(line2files.keys())\n",
        "  print(f\"total keys: {total_keys_cnt}: line2files: {line2files}\")\n",
        "\n",
        "  print(f\"line2seed:  {line2seed}\")\n",
        "\n",
        "  if total_keys_cnt != len(lines):\n",
        "    # if just looking at a few and outputting seeds, this is okay\n",
        "    print(f\"WARNING: Not examples for each line: for {init_ts}\")\n",
        "    each_line = set(range(len(lines)))\n",
        "    cur_lines = set(line2files.keys())\n",
        "    print(f\"missing lines: {each_line - cur_lines}\")\n",
        "\n",
        "\n",
        "  # need to just approve 3 at a time\n",
        "  cur_start = 0\n",
        "\n",
        "  choices_all = []\n",
        "  accepted_all = False\n",
        "\n",
        "  line2chosen_seed = {}\n",
        "\n",
        "  while cur_start < len(lines):\n",
        "    print(f\"line2files: {line2files}\")\n",
        "    cur_file_dict = defaultdict(list)\n",
        "    for line_num in range(cur_start, min(len(lines), cur_start+rows_at_a_time)):\n",
        "      if not line2files[line_num]:\n",
        "        # if we skipped some\n",
        "        continue\n",
        "      cur_file_dict[line_num].extend(line2files[line_num])\n",
        "\n",
        "    if len(cur_file_dict) == 0:\n",
        "      cur_start += rows_at_a_time\n",
        "      continue\n",
        "\n",
        "    print(f\"cur_file_dict: {cur_file_dict}\")\n",
        "\n",
        "    image_grid_dict(cur_file_dict, titles=None)\n",
        "\n",
        "    for ln in cur_file_dict.keys():\n",
        "      print(f\"line ({int(ln)}): {lines[int(ln)]}\")\n",
        "\n",
        "    accepted = False\n",
        "\n",
        "    time.sleep(8) # 8 needs at\n",
        "\n",
        "\n",
        "    # have to have this sleep for the input box to show up\n",
        "    while not accepted:\n",
        "      try:\n",
        "        someInput = input(\"Accept: \")\n",
        "        new_path = None\n",
        "        if re.match(\"^[0-9 ]+$\", someInput):\n",
        "          # just numbers and spaces\n",
        "          someInput = someInput.strip()\n",
        "          choices = someInput.split()\n",
        "          if len(choices) == len(cur_file_dict):\n",
        "            # we have a choice for each line\n",
        "            accepted = True\n",
        "            print(f\"choices: {choices}\")\n",
        "            choices_all.extend([int(c) for c in choices])\n",
        "            # get the seed for each choice\n",
        "            seeds_all.extend([int(line2seed[line_num][int(c)]) for line_num, c in zip(cur_file_dict.keys(), choices)])\n",
        "            for line_num, c in zip(cur_file_dict.keys(), choices):\n",
        "              line2chosen_seed[line_num] = int(line2seed[line_num][int(c)])\n",
        "            image_settings.seeds_all = seeds_all\n",
        "            print(f\"seeds_all: {seeds_all}\")\n",
        "            accepted_all = True\n",
        "          else:\n",
        "            print(f\"Not enough choices: {len(choices)}\")\n",
        "            accepted = False\n",
        "        elif someInput == \"q\":\n",
        "          print(\"Quitting\")\n",
        "          accepted = True\n",
        "          quit = True\n",
        "        elif someInput == \"s\":\n",
        "          print(\"Skipping\")\n",
        "          accepted = True\n",
        "\n",
        "      except Exception as e:\n",
        "        print(e)\n",
        "        pass\n",
        "\n",
        "    cur_start += rows_at_a_time\n",
        "\n",
        "\n",
        "  for line_num in line2chosen_seed.keys():\n",
        "    print(f\"line2seed: {line_num}: {line2chosen_seed[line_num]}\")\n",
        "\n",
        "  if choices_all and accepted_all:\n",
        "    if just_output_seeds:\n",
        "      print(f\"seeds_all: {image_settings.seeds_all}\")\n",
        "    else:\n",
        "\n",
        "      print(f\"init_ts: {type(init_ts)}: {init_ts}\")\n",
        "      init_ts_dir = init_ts.split(\"/\")[-1] # this is a string\n",
        "      move_to_dir = sequence_final_path / init_ts_dir\n",
        "      print(f\"Saving final files to: {move_to_dir}\")\n",
        "\n",
        "      os.makedirs(move_to_dir, exist_ok=True)\n",
        "      setting_filename = f\"{init_ts_dir}_settings.json\"\n",
        "      save_settings(image_settings, setting_filename, move_to_dir)\n",
        "\n",
        "      # get the actual file from choices\n",
        "      for line_num, line_key in enumerate(sorted(line2files.keys())):\n",
        "        print(f\"choices: {line_num}: {line_key}: {line2files[line_key]}\")\n",
        "        chosen_file = line2files[line_key][int(choices_all[line_num])]\n",
        "        new_name = \"_\".join(chosen_file.name.split(\"_\")[1:]) # get rid of the timestamp\n",
        "        new_path = move_to_dir / new_name\n",
        "        print(f\"choices: copying\\n\\t{chosen_file}\\n\\t{new_path}\")\n",
        "        shutil.copy(chosen_file, new_path)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "bPeV1vryziKX",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "PGMTHFl-mlBg",
        "GWkU2lGo69n6",
        "Azxj1uWqfoEJ"
      ],
      "provenance": [],
      "machine_shape": "hm",
      "gpuType": "T4",
      "private_outputs": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.11"
    },
    "vscode": {
      "interpreter": {
        "hash": "25b221746895226ff7c6b9d8aea8c62a9e808c88b786315a5ba5e4e82d158d3f"
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}