AI-Jungle Guide #4: On-Device AI

Author

Stefan Grothkopp
July 24, 2024

Welcome back to the AI-Jungle Guide!
Because it worked so well in the last issue, I’m continuing with the AI Co-host. Today’s host is Gemini Nano running in Google Chrome [1].

[Me]: Write a 1-2 funny sentence introduction to the newest issue of the AI-Jungle Guide newsletter. It's about on-device AI.
[Gemini Nano]: Welcome to the AI-Jungle Guide Newsletter, the definitive authority on the wild and wacky world of on-device AI!

What an on-device AI looks like according to leonardo.ai. Cheer up little buddy!

News

  1. Meet Gemini Nano: Google Chrome's 3 Billion-Parameter AI Assistant.
    [Me]: Write a title and a funny 1-2 sentence paragraph about Gemini Nano, the AI built into Google Chrome:

    • model has 3B parameters, but it quite capable

    • so far only in Chrome Canary for Developers to test

    • In the future the Chrome Browser will ship with it's very own AI model

    [Gemini Nano]: Imagine having an AI chatbot at your disposal, analyzing your browsing habits and providing personalized recommendations. Meet Gemini Nano [1], the incredibly intelligent AI built into Google Chrome! While still under development and accessible only in Google Chrome Canary for Developers [2], this 3 billion-parameter wonder promises to revolutionize your digital experiences in the future!
    [Me]: Whoa! I didn’t say anything about analyzing browsing habits!

  2. Apple Intelligence: When Your iPhone Knows Too Much.
    [Me]: Write a title and a funny 1-2 sentence paragraph about Apple Intelligence:

    • Apples push for on-device AI, announced on June 10, 2024 at WWDC 2024

    • will be integrated into iOS 18, iPadOS 18, and macOS Sequoia

    • It combines on-device processing with server-based models

    • strong focus on user privacy, using on-device processing

    [Gemini Nano]: Apple Intelligence[4] is the latest brainchild of the tech giant that combines on-device AI with server-based models to enhance user privacy. This innovative new feature, set to be integrated into iOS 18, iPadOS 18, and macOS Sequoia[5], promises to revolutionize the way we interact with our digital devices, all while keeping our personal information secure.
    [Me]: 0 out of 10 on the funny scale :(

  3. ChatHub: The Ultimate AI Chatting Platform.
    [Me]: Write a title and a funny 1-2 sentence paragraph about ChatHub:

    • A chrome extension that lets you use multiple AI-Chats side-by-side

    • compare the output of ChatGPT, Claude, Llama 3 and many others

    • It can use your existing accounts with these services

    • 2 chats at a time are free, more cost a one-time-fee or a subscription

    [Gemini Nano]: ChatHub[6] is the browser extension you never knew you needed until now! With this tool, you can open up multiple AI chat spaces side by side and compare the output of different models like ChatGPT, Claude, and Llama 3, and many more! Try it for free with 2 simultaneous chats, or pay a one-time fee to access unlimited chatting capabilities!
    [Me]: This is not an Ad, I swear! I mean, I tried to get a free license for this, but they didn’t reply.

Chrome Canary’s build-in AI, when used with an extension. [3]

Background

  1. Why is Google putting an LLM inside Chrome?
    I think the big tech companies like Apple, Microsoft, and Google are preparing to fight for users' attention regarding AI. Microsoft and Apple have an advantage regarding desktop and mobile operating systems. Google, of course, has Android, but is paying Apple $20 billion a year to be the default search on iOS [10]. So sneaking an AI onto your Mac, PC, or iPhone via the Chrome browser and getting a piece of that future AI cake is a smart move.

  2. Why is Siri so dumb and will it get smarter?
    I hope so! So far, Siri (and all other voice assistants) are not really AI, at least not by today’s standards. So the introduction of LLMs, on-device or in the cloud, will hopefully bring those assistants up to speed and finally make them really useful.

  3. Why would I need multiple AI chats side-by-side?
    Just try it! It’s really useful to compare which AI gives you the best result for a query. You can pick the answer you like best or combine multiple answers. You can also determine if a smaller model might give you good enough answers for recurring tasks.

4 different AIs competing for the funniest introduction to this newsletter. [6]

Try it yourself:

  1. Use Gemini Nano inside Chrome!

    This involves a few steps: First, you have to download and install the Chrome Canary version [2], then set a few developer flags [7], and finally, you’ll probably want to install a Chrome extension to actually talk to the LLM [3]. The alternative is to write your own code. All of this is very well described by Hugging Face here [7].

  2. Try Apple Intelligence on-device intelligence!
    Apple intelligence is not yet available. Apple just says "this fall" [8], leaving some wiggle room. So at the moment, when you want to get a feel for how it would work, you’re stuck with the available AIs like ChatGPT. To get a little closer to a native experience, you can try to connect ChatGPT with shortcuts into Siri [9]!

  3. Try chathub to compare your favourite AIs!
    Install the Chrome extension [6], and you can start with two AIs side-by-side for free. Make sure you're logged in to all the AI assistants you’re already using, and you can start right away!

Spread the Word!

Learned something new? Please do me a favour and forward this newsletter to someone else who could benefit from it! Or invite them to subscribe at: https://aijungle.guide!