oh, ok.

Category: Programming

  • Shell-AI: Integrate GPT into your command line

    Shell-AI: Integrate GPT into your command line

    I am currently evaluating a number of ways of integrating large language models into my Linux command line. Shell-AI (shai) is one of the easier ones to set up. With Shell-AI, you can simply input your intent in plain English (or other supported languages), and it will suggest single-line commands that achieve your desired outcome. It is designed to work on Linux, macOS, and Windows, though I only tested it on Linux. It’s backed by OpenAI’s GPT LLM – which is problematic for a number of reasons but also means the overall quality of the responses is cutting edge.

    Features

    • Natural Language Input: Describe what you want to do in plain English (or other supported languages).
    • Command Suggestions: Get single-line command suggestions that accomplish what you asked for. Select a suggestion, dismiss or regenerate in-place.
    • Cross-Platform: Works on Linux, macOS, and Windows.

    Shell-AI result quality

    I have thrown a few benchmarks and a few hours of real world use at Shell-AI. As expected, the LLM component, being based by default on gpt-3.5-turbo (although any OpenAI model can be configured) is top notch. Indeed shai was able to answer most of the questions I would usually have had to Google with reasonable solutions. It also saves time by avoiding the need for copy-pasting and context switching. The surrounding implementation that wraps the GPT-API is decent as well, providing multiple options and making it easy to select one. It asks for confirmation before executing each command. However, it doesn’t feature a built-in option to ask for clarification. For instance, quite often the output will feature a command chain that may be hard to understand. An option to ask GPT for an explanation would be nice, since Shell-AI’s output strips out any of the standard GPT fluff around the actual one-liner code. This means that I found Shell-AI to be a terrible tool for learning and a quite risky one to use at that.

    OpenAI Backend issues

    Shell-AI uses OpenAI’s GPT AI as a backend. That means:

    • You have to have an API key and pay for each call.
    • You need to be online at all times.
    • There are very serious privacy concerns despite shai itself being FOSS.
    • Response times are kinda slow, reducing the overall time-saving effect. With gpt-3.5-turbo which is supposed to be the fastest current option, response time is around 8 seconds. You can choose other models, but they will be even slower and the quality gains aren’t really relevant.

    Conclusion

    While Shell-AI is mildly interesting and it can save time significantly in some situations, I won’t be keeping it around. The main issue for me is privacy, but the poor performance limits overall usefulness as well.

  • Signal backup: How to fix crashes when restoring

    Signal backup: How to fix crashes when restoring

    Signal is the go free and open source messenger app for privacy conscious people, but it there are some issues and limitations. One criticism is that it doesn’t integrate its backup system well with cloud storage. For instance, WhatsApp will more easily allow you to store backups on Google Drive. Apart from that, usage of the Signal backup restore function is not particularly well explained and crashes are very common due to a fairly simple issue. I went down the rabbit hole to analyze the root cause.

    When moving to a different device or freshly setting up an existing one, you probably want to keep your chat history. I had that use case recently, when I installed GrapheneOS on my Pixel 6. A while beforehand I had already heard from a friend that restoring a Signal backup is buggy. They were experiencing crashes and had to give up on restoring their old messages, since backups can only be restored when newly setting up the app. Later on, restoring isn’t possible. There is also a feature to directly transfer chat history from one device to another, but that also didn’t work for them and wasn’t even an option for me, since I was using only a single device.

    What’s the issue?

    Backups can be enabled and scheduled in the Signal settings. When you first do that you’ll get a 30-digit passphrase that you will need to restore the backup. Once that is done two Signal backups will always be available in the specified directory. When you want to restore them, you have to newly install Signal on the respective device. Next choose the restore option and the file. Enter your passphrase and confirm. If it works, it should be pretty quick. When I finally got it to work, it only took about 3 minutes to restore a 5GB backup including all media. The problem is that for me and a lot of users, the app just crashes after a few moments.

    So why does this happen?

    Signal is open source, so it’s possible to just debug the code to find the cause. Also researching online will yield multiple user reports about this happening. And finally you can look at the logcat of the app. On the first startup screen of a newly installed Signal app, you can also just tap the icon about ten times. That will show you the app logs. If you do that after experiencing such a crash, there will be one of two exceptions with relevant stack traces. One indicates a negative array length being used when initializing an array. The other one being an out of memory error. There is a third possibility to that will probably result in an ArrayIndexOutOfBoundsException, however I haven’t been able to observe this since it’s rare.

    Looking at the relevant code, you can see that Signal stores the backed up data in multiple chunks, but in a single file. However when decrypting the backup, the app doesn’t even know how large each chunk is. That information is also encrypted inside the backup file. Later that information is used to allocate a buffer. If that buffer is thus too large or the size is negative the app will crash. The third option can happen when the decrypted buffer size is reasonable, but the restoration process will still fail.

    What causes the buffer size to have such random values?

    Well apparently the app (currently) has no way of checking whether the passphrase you enter for decryption is correct. Because of this, it will simply decrypt it even with an incorrect passphrase and take the value as a number from where the chunk size (=buffer size) WOULD be if the file was decrypted correctly. That’s why entering different wrong passphrases can yield different stack traces. So yeah, that’s it. If Signal crashes for you when restoring a Signal backup, you’re probably entering a wrong passphrase.

    And that’s indeed what I and some other users were doing. I was entering my Signal PIN, another user was entering an old passphrase. The UI IMO does not give sufficient information or a reminder to a user that a 30 digit passphrase needs to be entered. Thus, there are two issues: First are the crashes when entering a wrong phrase, due to insufficient error handling and because Signal doesn’t know the chunk size at that time. Second, the UX is very imperfect, possibly causing users to enter the wrong value.

    How can the Signal backup functionality be fixed?

    I proposed these improvements be made to solve this issue:

    1. The text should be adjusted to remind the user that the 30-digit backup passphrase is required. I forgot this, and many users will forget it, thus entering the Signal PIN or something else, causing confusion.
    2. The error should be caught properly. If the chunk size is unreasonable, immediately display a message stating that the passphrase isn’t correct, or just catch any exception with a reasonable message. This would be a quick fix to prevent crashes and confusion.
    3. When activating backups the passphrase is shown. IIRC you just have to check a box that you wrote it down. I think it would be better to have the user write it back. Maybe using words would be more memorable, too. Given the number of people who are reporting here, a lot of users may have forgotten the passphrase, so UX improvements may be necessary.
    4. Review if it makes sense or is necessary to even encrypt the chunk size. Can it not be in plain in the file header or be a fixed value?
    5. It should be more clear how to change the passphrase. When I switch my phone I may remember to check if I have a recent backup and look at the Chat Backup menu. If I then realize I don’t have the passphrase anymore, it’s not immediately clear how to proceed.

    At least I had fun debugging…