As a creative individual working with AI models and the Stable Diffusion WebUI, you’ll be pleased to know that there’s an exciting new extension available that enhances your creative process.
The CLIP Interrogator Extension, created by ‘pharmapsychotic’, allows you to seamlessly integrate the CLIP model into the Web UI, for generating innovative prompts and ideas.
In this step-by-step guide, I’ll walk you through how to install and use this extension effectively.
How to Install CLIP Interrogator Extension in StableDiffusion?
Before diving into the exciting features of the CLIP Interrogator Extension, you need to install it.
Here’s a simple guide on how to do that:
1. Go to the ‘extensions‘ tab within the Stable Diffusion WebUI.
2. Inside the ‘extensions‘ tab, click on the ‘Install from URL‘ sub-tab.
3. In the provided field, paste the following URL: https://github.com/pharmapsychotic/clip-interrogator-ext
4. Once you’ve pasted the URL, click the ‘Install‘ button.
5. After clicking ‘Install,’ it’s a good idea to check your terminal window for any potential errors. If you encounter any issues during installation, don’t hesitate to seek assistance from the developer.
6. Once the installation process is complete, restart the Web UI.
7. After restarting, you should now see a new tab labeled ‘Interrogator‘ in your Web UI interface. This is where you’ll access the CLIP Interrogator Extension.
CLIP Interrogator Extension Usage:
Now that you have successfully installed the extension, let’s explore how to use it effectively.
Accessing the CLIP Interrogator:
Click on the ‘Interrogator‘ tab in the Web UI. This will open up the CLIP Interrogator Extension interface.
API Functionality:
The CLIP Interrogator Extension comes with a simple and intuitive API to interact with. You can access the API documentation on the /docs page under /interrogator/* by using the –api flag when starting the Web UI.
- /interrogator/models: This endpoint lists all the available models for interrogation. You can explore different models to achieve diverse results.
- /interrogator/prompt: Use this endpoint to retrieve a prompt for a given image, model, and mode. It’s the core function that generates prompts based on your input image.
- /interrogator/analyze: This endpoint provides a list of words and their scores for the given image, model, and mode. It can help you understand the model’s perception of your input image.
Creating Prompts:
To get started, upload an image that you want to use as a reference for generating prompts. The extension will automatically create prompts that yield similar results using Stable Diffusion v1 and v2.
VRAM Optimization:
The extension is designed to work smoothly across a wide range of GPUs, including those with low to medium VRAM. It performs its own detection to switch into a low VRAM mode, ensuring compatibility with various hardware configurations. It has been tested with an Nvidia 1070 8GB GPU.
Precomputed Text Embeddings:
To expedite processing, the extension downloads precomputed text embeddings in safetensors format from Hugging Face and stores them in the ‘models/clip-interrogator’ directory.
Monitoring Progress:
During the first run, the progress may not be displayed immediately on the Web UI interface. If you wish to monitor the progress, check the console to see how it’s doing.
Conclusion:
The CLIP Interrogator Extension for Stable Diffusion WebUI offers a powerful toolset for artists, researchers, and AI enthusiasts. It seamlessly integrates the CLIP model, allowing you to generate prompts, analyze images, and explore various models effortlessly.
