In the DesignPrompt research project we developed DesignPrompt, a moodboard web application that lets designers combine multiple modalities — images, color, text — into a GenAI prompt and tweak the results.
It lets designers search images online (a) or generate AI images to create a moodboard (b) using common tools (c) as well as additional semantic meta-data of the moodboard images (d). Designers can compose multimodal GenAI prompts with images (e), colors, semantics and text (f) and finely tune their intentions (g).
Dependencies must be installed separately for the main application and the server.
- Navigate to the root folder and run:
npm install
- Navigate to the server folder and run:
npm install
To compile and hot-reload for development, run:
npm run devDesignPrompt should be running at the default local address
Add your Unsplash API key to the .env file located in the project’s root directory.
- Unsplash API specifications may change over time. Please refer to the official Unsplash documentation for the latest updates.
- Make sure to check your rate limits before using the API to avoid interruptions.
To use the OpenAI API, follow the Quickstart guide to create and export your API key as a local environment variable.
- For the latest API specifications, refer to the OpenAI API reference.
To enable and authenticate the Google Cloud Vision API, follow the setup instructions using the Google Cloud CLI.
Feel free to take a look at DesignPrompt's full publication at: https://doi.org/10.1145/3643834.3661588
@inproceedings{peng2024:designprompt,
author = {Peng, Xiaohan and Koch, Janin and Mackay, Wendy E.},
title = {DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AI},
year = {2024},
isbn = {9798400705830},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3643834.3661588},
doi = {10.1145/3643834.3661588},
booktitle = {Proceedings of the 2024 ACM Designing Interactive Systems Conference},
pages = {804–818},
numpages = {15},
keywords = {Creativity Support Tool, Design Practice, Generative AI, Human-AI Ideation, Human-AI Interaction, Moodboard},
location = {Copenhagen, Denmark},
series = {DIS '24}
}This project is licensed under the MIT License, see the LICENSE.txt file for details.