I took Proton’s privacy-first chatbot for a spin and it failed to impress

AI is becoming more and more ingrained in our society, and that trend will only continue. I’ve been using ChatGPT and Gemini ever since they launched, and honestly, I can’t imagine my workday without them. I also use them for personal projects, which means I’ve shared a fair bit of highly private information with these chatbots.

This brings up the crucial question of privacy. Are you truly comfortable sharing your most intimate thoughts and ideas with a chatbot that will use your data for training? While some people don’t care, a growing number certainly do. Proton is aware of this, which is why they recently announced Lumo — a privacy-first AI chatbot.

But is Lumo any good, and should you consider making the switch? Let’s dive in and find out.

Before we get to the hands-on part of the post, it’s important to understand what truly sets Lumo apart from rivals like ChatGPT and Gemini. The company behind it, Proton, makes a bold claim: Lumo is totally private right out of the box. This means your data will never be used to train the AI. Furthermore, all your chats are encrypted and deleted, ensuring no one but you can read them. This also guarantees your data won’t be shared with anyone for advertising or other purposes.

Proton also emphasizes that it’s based in Europe, which boasts stricter privacy laws than many other regions. Additionally, Lumo is built upon open-source language models. This means it’s not in a partnership with OpenAI (which runs ChatGPT) or any other multi-billion-dollar American or Chinese corporation.

This is a stark contrast when compared to the likes of ChatGPT or Gemini. Both of these services will store your chats by default, use them to train their AI models, and may even assign actual humans to read them in hopes of improving the service overall. A bit scary when you think about it, but there are ways of mitigating this, which I’ll discuss a bit later.

First impressions matter, and I have to say, Lumo didn’t start strong. I asked how long a flight from Paris to Berlin takes, and the response I got was that I needed to turn on the “Web search” feature to get an answer. I did, asked again, and received the same response. This happened multiple times. The chatbot did eventually give me a proper and correct answer, but then kept repeating the same issue when asked for a different route. Not a good start at all.

I then proceeded to ask which are the best phones I could buy for under €1,500. It gave me a list straight away, but it contained phones up to two years old. These included the Pixel 7a, OnePlus 11, and the iPhone 15 Pro. In contrast, the free version of ChatGPT provided a more up-to-date list, as did the premium version of Gemini. Lumo’s cut-off date is April 2024, so it doesn’t have access to the latest information, surprisingly not even with the web search feature turned on.

Things got a bit better from there. I fed the chatbot a rough draft of one of my posts and asked it to list and fix typos, as well as help me brainstorm a title. No issues here; it did a decent job. I also asked it to summarize a document and help me figure out a problem I recently had when decorating my apartment, which was how to attach an acoustic panel to a wall so that I can remove it without damaging the wall. The chatbot did a reasonable job, although the answers provided weren’t any better than what I get from ChatGPT or Gemini.

When asked to list alternatives to Obsidian, Lumo provided a list, but I didn’t like how it structured the answer. It listed 10 options with a one-sentence description for each, along with a bulleted list of their key features. I prefer ChatGPT’s formatting, as it categorized alternatives by type (local-first, cloud-based, etc.), which gave me more relevant information from the get-go. Unlike Lumo, ChatGPT also asked me a few questions, including if I’d like to see a comparison table.

I said yes, and then prompted Lumo to do the same. Both chatbots generated tables, but while I liked ChatGPT’s, the table from Lumo was a mess. It was very hard to read, as you can see in the screenshot above.

So, what’s the verdict on the overall performance? Lumo’s performance is far from special. It provided me with a few answers I was happy with, but the constant nudging to turn on the web search feature (which often did nothing), providing out-of-date information, and poorly structuring data and tables means I don’t really see a benefit when compared to ChatGPT or Gemini.

Not to mention that Lumo can’t create images, doesn’t have a “Deep Research” function, or custom GPTs. So from a pure performance perspective, I don’t see myself switching to Lumo any time soon.

Based on my short time with Lumo, Proton isn’t overly focused on raw performance. Instead, it heavily highlights its privacy features, hoping that will be enough to attract users. I mean, I was intrigued enough to try it out, so perhaps others will be too.

However, it’s worth noting that you can achieve a similar privacy experience with ChatGPT and Gemini. While they aren’t privacy-first chatbots, you can disable AI training and the option to save your chats on both services. This means that, just like with Lumo, your chats won’t be saved or used to train the AI.

There are a few differences, though. Gemini’s terms state that even when you opt out of this, the company still has access to your messages for 72 hours, for security and other reasons. This shouldn’t happen with Lumo since the messages are encrypted, so the company couldn’t read them even if it wanted to.

So, there is a bit of a difference in privacy between the two, and I’ll let you decide for yourself whether that difference is worth making the switch to Lumo.

Personally, I have no compelling reason to switch. If I want a privacy-focused chatbot, I can get a very similar experience with Gemini by adjusting a few settings. Perhaps it’s not on quite the same level as Lumo’s complete encryption, but it’s good enough for me.

And when it comes to performance and all the extra features available, both Gemini and ChatGPT are a significant step ahead of Proton’s chatbot. Their answers are more up-to-date, more detailed, and better formatted.

I love seeing more chatbots pop up since more competition is always better for us consumers. I expected a lot from Lumo since it’s a Proton product, but it just feels like a half-baked product at the moment. But hey, so was Gemini (and Bard before it) when it first launched, and now it’s my favorite chatbot.

So, hopefully, Lumo will improve drastically over the next few months and become a real contender in the space. Only time will tell.

What about you? Would you consider switching to Lumo because of its privacy-first promise? Let me know in the comments.