# Open-source AI tools: one pick for each job

Eight everyday AI jobs, from chatting with a model on your computer to testing prompts, each with one open-source pick from our Top 20 lists, one alternative, what it needs and how to start.

Guides · v1.0 · draft · https://hayzlabs.com/guides/open-source-ai-tools/

In short:
- For eight everyday AI jobs there is one free, open-source pick and one alternative, all from our Top 20 lists, and most run on your own computer.
- Start with Jan or Ollama, test a model on a paragraph you know, then ask again with Wi-Fi off.
- Check the licence of the model as well as the app. Many popular models are open weights under the maker's own terms.

## Short answer

For most AI jobs there is a free tool whose code you can read, change and keep using if the company behind it changes course. The picks below are our suggestion for where to start, one per job, with one alternative, and every one is on one of our [Top 20 lists](/reviews/top/). Each licence was read in the project's own repository between 10 and 20 October 2026.

Begin with a chat app and a model. Add another tool only when you have a job for it.

Whether your computer can run a given model, and when a cloud model is the better choice, is covered in [Local or cloud](https://hayzlabs.com/guides/local-or-cloud/). This page is about which tool to install.

## What counts as open source here

Open source here means code under a licence the Open Source Initiative approves, such as MIT, Apache 2.0, GPL-3.0 or AGPL-3.0; GPL and AGPL add that if you share a changed version, you share its source too. Many popular models are open weights (the trained model files are free to download) under the maker's own terms instead, so open the model's licence file as well as the app's. The models we name below are Apache 2.0.

## The picks

| Job | Our pick | Alternative | Licence | What it needs |
|---|---|---|---|---|
| Chat with a model on your computer | Jan | Ollama | Apache 2.0 / MIT | A computer with enough memory for the model; see the next row |
| A model to run | Gemma 4 26B A4B | Qwen3.8-27B | Apache 2.0 / Apache 2.0 | About 17 to 18 GB of free memory at 4-bit (our estimate, not tested), so in practice a machine with 24 GB or more |
| Ask questions of your own documents | AnythingLLM | Open Notebook | MIT / MIT | A model from the row above, or a cloud key (an account key for an online AI service); Open Notebook also needs Docker Desktop |
| Turn PDFs and Office files into clean text | Docling | MarkItDown | MIT / MIT | Python 3.10 or later (a free programming language) |
| Dictate instead of typing | Handy | - | MIT | Nothing extra; it downloads a speech model on first run |
| Make images | ComfyUI | InvokeAI | GPL-3.0 / Apache 2.0 | A graphics card or Apple Silicon, and an image model |
| A coding agent in your terminal (the text window for typing commands) | OpenCode | Codex CLI | MIT / Apache 2.0 | OpenCode: a cloud key or a local model. Codex CLI: a ChatGPT plan or an API key |
| Test prompts and compare models | promptfoo | - | MIT | Node.js, or Homebrew or pip |

## How to start with each

**Chat on your computer.** Jan is a desktop app for Windows, macOS and Linux that downloads and runs models from Hugging Face on your own machine. Its own guidance for macOS is 8 GB of memory for 3B models, 16 GB for 7B and 32 GB for 13B ([Jan](https://github.com/janhq/jan)). Download it from jan.ai, pick a small model in the app, and ask it something you already know the answer to. Choose Ollama instead if you want one engine that other tools connect to. Install it from ollama.com, then run `ollama run gemma4:26b` in a terminal ([Ollama](https://github.com/ollama/ollama), [Gemma 4 on Ollama](https://ollama.com/library/gemma4)).

**A model.** Google's Gemma 4 26B A4B is Apache 2.0. It is a mixture-of-experts model (only part of it runs for each word), with 25.2 billion parameters in total and 3.8 billion active, which Google aims at "consumer GPUs and workstations" ([Gemma 4 26B A4B](https://huggingface.co/google/gemma-4-26B-A4B-it), vendor). Ollama's download for it is 16 to 19 GB ([Ollama](https://ollama.com/library/gemma4)). Qwen3.8-27B is also Apache 2.0 and reads images and video as well as text ([Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), vendor). On a 16 GB laptop the models on our [open-model list](/reviews/top/) are too big to run well, so a cloud model may be the better choice; [Local or cloud](https://hayzlabs.com/guides/local-or-cloud/) helps you decide. Before you build on any model, open its licence file.

**Your own documents.** AnythingLLM is a desktop app that takes your files, answers from them and shows which source it used. It works with local models or a cloud key ([AnythingLLM](https://github.com/Mintplex-Labs/anything-llm)). Open Notebook is a self-hosted take on Google's NotebookLM: you add PDFs, web pages, audio and video, then chat with them. It runs in Docker Desktop, opens in your browser and works with Ollama or a cloud key. Its own comparison calls its source references "Basic", so check every quote against the file ([Open Notebook](https://github.com/lfnovo/open-notebook)). Whether retrieval (looking up passages in your files before answering) is the right fix for your problem is covered in [Prompt, retrieve or fine-tune?](https://hayzlabs.com/guides/prompt-retrieve-or-fine-tune/).

**PDFs into clean text.** Docling reads PDF, Word, PowerPoint, Excel, HTML and images, keeps tables and reading order, can OCR scans (read the text in scanned pages), and runs on your computer. Install with `pip install docling`, then run `docling` followed by the file path to get Markdown you can paste into any model ([Docling](https://github.com/docling-project/docling)). Check the tables against the original before you rely on them.

**Dictation.** Handy is MIT-licensed, works offline on Windows, macOS and Linux, and pastes text where your cursor is when you release a keyboard shortcut ([Handy](https://github.com/cjpais/Handy)). To transcribe a recording and check it before you quote it, see [Transcribe a recording, then check it](https://hayzlabs.com/workflows/transcribe-then-check/). To translate, see [Translating with AI](https://hayzlabs.com/guides/translating-with-ai/).

**Images.** ComfyUI has a desktop app for Windows and macOS and builds images from a visual node graph. It gives you control over every step, which takes time to learn ([ComfyUI](https://github.com/Comfy-Org/ComfyUI)). InvokeAI offers a canvas with brush tools and painting in or out of the frame, and installs with a launcher ([InvokeAI](https://github.com/invoke-ai/InvokeAI)). The model has its own licence. Black Forest Labs released FLUX.2 [klein] 4B under Apache 2.0, while FLUX.2 [klein] 9B and FLUX.2 [dev] are under its FLUX Non-Commercial License ([VentureBeat](https://venturebeat.com/technology/black-forest-labs-launches-open-source-flux-2-klein-to-generate-ai-images-in), January 2026). How to prompt and check an image is in [Make an image with AI](https://hayzlabs.com/guides/make-an-image-with-ai/).

**A coding agent.** OpenCode says it supports "75+ LLM providers" and local models ([OpenCode](https://opencode.ai/docs/providers/), vendor). It has a read-only plan agent, so you can see what it would do before it changes anything ([OpenCode](https://github.com/anomalyco/opencode)). OpenAI's Codex CLI is Apache 2.0 and runs in your terminal. It installs with `npm install -g @openai/codex` and signs in with a ChatGPT plan or an API key ([Codex CLI](https://github.com/openai/codex)). If you don't write code, read [Coding agents for people who don't write code](https://hayzlabs.com/guides/coding-agents-for-non-coders/) first.

**Testing prompts.** promptfoo runs the same prompts across models and shows the answers side by side. Install it with `npm install -g promptfoo`, run `promptfoo init --example getting-started`, then `promptfoo eval`. The company joined OpenAI, and the project says it "remains open source and MIT licensed" ([promptfoo](https://github.com/promptfoo/promptfoo)). What to put in the test set is in [Test any new model on your own tasks](https://hayzlabs.com/workflows/personal-eval-set/).

## Your first run (illustrative)

Sam has a laptop with 32 GB of memory and wants a model that works without sending anything online. About 15 minutes.

1. Sam installs Ollama from ollama.com, typing the address himself, and opens a terminal.
2. He runs `ollama run gemma4:26b`. The model downloads once, then a prompt appears.
3. He pastes a short paragraph from a report he wrote last month and types: `List the three main points of this paragraph in one line each.`
4. He compares the answer with what he knows the paragraph says. All three points are his; nothing is added that is not in the text.
5. He turns off Wi-Fi and asks again. It still answers, so it is running on his laptop.

A good result names the points you wrote and adds nothing. If it invents a point or the laptop slows to a crawl, close other apps and try again, or see [Local or cloud](https://hayzlabs.com/guides/local-or-cloud/). In Jan the same run is: download from jan.ai, pick a small model in the app, paste the same paragraph.

## How to tell it worked

- The model answers a question you can check, and the answer matches what you know.
- It still answers with Wi-Fi off, so your text stayed on your computer.
- You opened the licence file of both the app and the model and saw Apache 2.0, MIT or another licence from the table.

## Popular, but not open source

Open WebUI (its licence forbids removing the branding once a deployment has more than 50 users in 30 days, unless you have permission ([licence](https://github.com/open-webui/open-webui/blob/main/LICENSE))), n8n (source available under its "Sustainable Use License" ([licence](https://github.com/n8n-io/n8n/blob/master/LICENSE.md))) and Meta's Llama models (open weights; services with more than 700 million monthly users need a separate licence from Meta ([Llama 3.1 licence](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE))) are left off the picks for that reason.

## Before you install

> [!CHECK]
> Download from the project's own site or repository, and type the address yourself. Software names are easy to imitate. A local tool with web access or other tools can still be fed hostile text, so give it only the folders and keys it needs. Keep these tools updated and never open their local server to the internet. Ollama before 0.17.1 lets anyone who can reach it read server memory, which can include keys (CVE-2026-7482). In April 2026 Censys found attackers turning internet-facing ComfyUI servers into a cryptomining botnet through unsafe custom nodes. Install ComfyUI custom nodes only from sources you trust. For what should never go into any AI tool, local or not, see [What not to paste](https://hayzlabs.com/start/what-not-to-paste/).

## Limits

> [!LIMIT]
> These are our picks from reading each project's own documentation, not from a head-to-head test. Memory figures and features are the projects' own claims. Licences can change between versions, so read the licence of the version you install. The memory figures for models are our estimate from their size, not a test. Prices and plans for the companies' hosted versions are not covered here.

## Try this next

1. Do the first run above with a paragraph of your own, then repeat it with Wi-Fi off. (15 minutes)
2. Run five of your real tasks on it and on the cloud model you use, as in [Local or cloud](https://hayzlabs.com/guides/local-or-cloud/). (30 minutes)
3. Add one more tool from the table only for a job you did this week.

## Sources

1. Meta. Llama 3.1 Community License. [Link](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)
2. Jan, Ollama, AnythingLLM, Open Notebook. Repositories, licences and README files, read 10 and 18 October 2026. [Jan](https://github.com/janhq/jan) · [Ollama](https://github.com/ollama/ollama) · [AnythingLLM](https://github.com/Mintplex-Labs/anything-llm) · [Open Notebook](https://github.com/lfnovo/open-notebook)
3. Google Gemma 4 26B A4B and Qwen3.8-27B model cards, and Ollama's Gemma 4 page, read 20 October 2026. Vendor. [Gemma 4 26B A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) · [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) · [Ollama gemma4](https://ollama.com/library/gemma4)
4. Docling, MarkItDown, Handy. Repositories and licences. [Docling](https://github.com/docling-project/docling) · [MarkItDown](https://github.com/microsoft/markitdown) · [Handy](https://github.com/cjpais/Handy)
5. ComfyUI and InvokeAI repositories; VentureBeat on FLUX.2 [klein] licences, January 2026. News. [ComfyUI](https://github.com/Comfy-Org/ComfyUI) · [InvokeAI](https://github.com/invoke-ai/InvokeAI) · [VentureBeat](https://venturebeat.com/technology/black-forest-labs-launches-open-source-flux-2-klein-to-generate-ai-images-in)
6. OpenCode, Codex CLI, promptfoo. Repositories, licences and OpenCode's provider docs. [OpenCode](https://github.com/anomalyco/opencode) · [OpenCode providers](https://opencode.ai/docs/providers/) · [Codex CLI](https://github.com/openai/codex) · [promptfoo](https://github.com/promptfoo/promptfoo)
7. Open WebUI and n8n licence files. [Open WebUI](https://github.com/open-webui/open-webui/blob/main/LICENSE) · [n8n](https://github.com/n8n-io/n8n/blob/master/LICENSE.md)
8. Security records: CVE-2026-7482 (Ollama), CVE record; Censys on ComfyUI servers, April 2026. [CVE-2026-7482](https://cveawg.mitre.org/api/cve/CVE-2026-7482) · [Censys](https://censys.com/blog/comfyui-servers-cryptomining-proxy-botnet/)