oh, ok.

Category: Windows

  • Shell-AI: Integrate GPT into your command line

    Shell-AI: Integrate GPT into your command line

    I am currently evaluating a number of ways of integrating large language models into my Linux command line. Shell-AI (shai) is one of the easier ones to set up. With Shell-AI, you can simply input your intent in plain English (or other supported languages), and it will suggest single-line commands that achieve your desired outcome. It is designed to work on Linux, macOS, and Windows, though I only tested it on Linux. It’s backed by OpenAI’s GPT LLM – which is problematic for a number of reasons but also means the overall quality of the responses is cutting edge.

    Features

    • Natural Language Input: Describe what you want to do in plain English (or other supported languages).
    • Command Suggestions: Get single-line command suggestions that accomplish what you asked for. Select a suggestion, dismiss or regenerate in-place.
    • Cross-Platform: Works on Linux, macOS, and Windows.

    Shell-AI result quality

    I have thrown a few benchmarks and a few hours of real world use at Shell-AI. As expected, the LLM component, being based by default on gpt-3.5-turbo (although any OpenAI model can be configured) is top notch. Indeed shai was able to answer most of the questions I would usually have had to Google with reasonable solutions. It also saves time by avoiding the need for copy-pasting and context switching. The surrounding implementation that wraps the GPT-API is decent as well, providing multiple options and making it easy to select one. It asks for confirmation before executing each command. However, it doesn’t feature a built-in option to ask for clarification. For instance, quite often the output will feature a command chain that may be hard to understand. An option to ask GPT for an explanation would be nice, since Shell-AI’s output strips out any of the standard GPT fluff around the actual one-liner code. This means that I found Shell-AI to be a terrible tool for learning and a quite risky one to use at that.

    OpenAI Backend issues

    Shell-AI uses OpenAI’s GPT AI as a backend. That means:

    • You have to have an API key and pay for each call.
    • You need to be online at all times.
    • There are very serious privacy concerns despite shai itself being FOSS.
    • Response times are kinda slow, reducing the overall time-saving effect. With gpt-3.5-turbo which is supposed to be the fastest current option, response time is around 8 seconds. You can choose other models, but they will be even slower and the quality gains aren’t really relevant.

    Conclusion

    While Shell-AI is mildly interesting and it can save time significantly in some situations, I won’t be keeping it around. The main issue for me is privacy, but the poor performance limits overall usefulness as well.

  • Getting started with local Stable Diffusion XL AI

    Getting started with local Stable Diffusion XL AI

    Current image generation AI is amazing, and Stable Diffusion is one of the best models available. It is capable of generating excellent quality images and because it is open source, you can run it locally which means there are no privacy concerns or additional costs involved with using it. Just a few days ago the newest and most powerful version yet, Stable Diffusion XL 1.0 was released which works better at higher resolutions of 768×768 to 1024×1024. With some extra steps you can set it up and use it today with stable-diffusion-webui, an easy to use tool you can run locally and use in your browser to play around with various models. This is how to set it up in just 10 minutes:

    stable-diffusion-webui setup

    First, if you have an Nvidia GPU, make sure you have the latest proprietary driver. You need it in order to make use of CUDA for acceleration of Stable Diffusion.

    Installing python3

    First you will need to install python3 if you don’t already have it. I won’t get into too much detail on this because there are thousands of guides for this, but the easiest way is to use a package manager:

    For Windows 11: Run winget install -e --id Python.Python.3.11 in the Windows terminal

    For Arch Linux: Run sudo pacman -S python

    For Ubuntu: It should already be installed on modern versions

    Running python -V should now yield a 3.x version number.

    Install stable-diffusion-webui

    Next we are going to download and install stable-diffusion-webui which we are later going to use to interact with Stable Diffusion. As of now support for Stable Diffusion XL has not yet been merged into the master branch so we are going to use the dev branch.

    If you don’t have git, you can download the current dev state here: stable-diffusion-webui
    If you do have git however, I recommend that you properly clone the repository. This way you can later update more easily:
    git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
    git checkout dev

    Download Stable Diffusion XL models

    stable-diffusion-webui comes with Stable Diffusion 1.5. If you want to use the improved Stable Diffusion XL model, you will need to download it separately and place it in the directory stable-diffusion-webui/models/Stable-diffusion
    You can get the base model from here: Base Model
    And the optional refiner model from here: Refiner Model

    Running the webui and usage

    Now you can run the webui. Simple start webui.sh if you are on Linux or webui.bat if you are on Windows. It may download and install some dependencies on your first launch, but when it’s done, point your web browser to http://localhost:7860/ and you should see the webui.

    At the top left, make sure you select the XL base model. If you are using Stable Diffusion XL, make sure your resolution is between 768×768 and 1024×1024 or quality will be poor. Higher resolutions will take longer to generate but look sharper. You can also play with the number of sampling steps and sampling method which can influence the final result significantly. Generally, 20-60 steps are good values and the sampler “DPM++ SDE Karras” should yield very good result. You can learn more in this excellent comparison: https://stable-diffusion-art.com/samplers/#Evaluating_samplers
    You can move the CFG slider to influence how creative the model should interpret the prompt. A lower value may lead to less literal, more creative results.
    Next, just enter a prompt and hit generate. If you encounter any issues, make sure to read the next section.

    Finally, if you have installed the refiner model, you can send your generated image to the img2img section. There you can switch to the refiner model to apply modifications and tweaks to the original image. For example, you can change the subject or art style after you are happy with the basic composition.

    Optimizing performance and troubleshooting

    Here are some tips for improving performance:
    If you are on Linux, installing TCMalloc may improve generation speed, for example: sudo apt install --no-install-recommends google-perftools
    If you are using CUDA, running with xformers should speed things up further: webui.sh –xformers
    If you are running low on VRAM and experiencing crashes, try this option to save memory at the cost of speed: webui.sh --medvram
    And finally, if you are generating black images, try this option: webui.sh --no-half-vae

    Examples

    Here are some cool images I was able to generate using Stable Diffusion XL:

    Ancient Rome
    Cyberpunk outfit
    A weird situation happening in public