Local AI Meets the Unix Pipeline With ShellGPT Source
Markdown source
1---2title: "Local AI Meets the Unix Pipeline With ShellGPT"3date: "2026-06-18"4published: true5tags: ["ai", "shell-gpt", "ollama", "gemma", "local-ai", "unix", "bash", "terminal", "linux"]6author: "Gavin Jackson"7excerpt: "I started using ShellGPT today with a locally hosted Gemma 4 26B quantized model through Ollama on my RTX PRO 4000 Blackwell workstation. Asking questions in the terminal was fine, but the real value showed up when local AI met the Unix pipeline."8---910# Local AI Meets the Unix Pipeline With ShellGPT11121314I started using [ShellGPT](https://github.com/ther1d/shell_gpt) today, pointed at my locally hosted Gemma 4 26B quantized model through Ollama.1516This is running on the same NVIDIA RTX PRO 4000 Blackwell workstation I wrote about in [Running Modern Edge LLMs on an RTX PRO 4000 Blackwell](/post/running-edge-llms-rtx-pro-4000-blackwell). That setup already made local inference feel practical. ShellGPT adds a different layer on top: it puts the model directly into the terminal, where a lot of real system work already happens.1718The result is interesting. Not because typing a question into a terminal is magically better than typing it into a chat window. That part was fine, but not revolutionary. It felt very similar to asking Ollama directly.1920The useful part was composability.2122ShellGPT becomes much more compelling when it is chained with commands, fed real output from the machine, and used to turn human language into shell commands or bash scripts.2324That is where it started to click.2526Note - to keep this article down to a reasonable length, none of my examples actually show output from the commands, but this should get a better idea of what is possible.2728## What ShellGPT Does2930ShellGPT is a command-line productivity tool for talking to large language models from the shell. The project supports regular prompts, shell command generation, code generation, chat sessions, REPL mode, and shell integration.3132The command is `sgpt`, so the basic shape is simple:3334```bash35sgpt "Explain what the Linux load average means in plain English"36```3738That is useful enough. It means I can ask a question without leaving the terminal, switching to a browser, opening another app, or copying context around.3940But with a local model, the more important detail is that the question stays on the workstation. The prompt goes to Ollama on `localhost`, not to a cloud API.4142That matters for the sort of environments I care about. A lot of shell work involves logs, hostnames, service names, paths, configuration snippets, and operational clues that should not be casually pasted into a public SaaS tool.4344## Asking Questions Was Fine4546The first thing I tried was the obvious thing:4748```bash49sgpt "What is the difference between a systemd service and a systemd target?"50```5152That worked exactly as expected. It gave a reasonable explanation. I tried a few more:5354```bash55sgpt "What does set -euo pipefail do in a bash script?"56```5758```bash59sgpt "Explain the difference between a bind mount and a named Docker volume"60```6162This is handy, but it is not very different from:6364```bash65ollama run gemma4-26b "What does set -euo pipefail do in a bash script?"66```6768For standalone questions, ShellGPT is mostly a nicer terminal wrapper around a model. That is not a criticism. It is still convenient. But if that was all it did, I probably would not write a whole post about it.6970The real power showed up when I stopped treating it like chat and started treating it like another Unix tool.7172## Piping Real Output Into The Model7374The shell is already good at collecting evidence. It can list files, stream logs, inspect services, query APIs, and format text. ShellGPT becomes useful when it can sit at the end of that pipeline and reason over the output.7576The pattern is simple:7778> Use normal shell tools to collect the evidence, then pipe that evidence into `sgpt` and ask the model to interpret it.7980That feels much more useful than asking generic troubleshooting questions in isolation.8182For example, instead of asking a generic question about disk usage, I can give it the actual disk usage:8384```bash85df -h | sgpt "Which filesystem should I investigate first? Keep the answer short."86```8788Or I can give it a size breakdown:8990```bash91du -xh --max-depth=1 /var 2>/dev/null \92 | sort -h \93 | tail -20 \94 | sgpt "Summarise which directories are taking space and suggest the safest next checks."95```9697That is a better workflow than asking "how do I troubleshoot disk space?" because the model has the current state in front of it.9899Failed services are another nice fit:100101```bash102systemctl --failed --no-pager \103 | sgpt "Explain these failed services and rank them by likely importance."104```105106Logs are an obvious example:107108```bash109journalctl -u ollama -n 120 --no-pager \110 | sgpt "Summarise the last failure and suggest the next three commands to run."111```112113For Docker:114115```bash116docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}' \117 | sgpt "Explain which services are exposed and whether anything looks surprising."118```119120If a specific container is misbehaving, I can hand ShellGPT the recent logs directly:121122```bash123docker logs --tail 120 my-container 2>&1 \124 | sgpt "Find the likely root cause of the error in these logs."125```126127For Git:128129```bash130git diff --stat \131 | sgpt "Summarise the scope of this change in one paragraph."132```133134And for quick commit-message drafting:135136```bash137git diff --cached \138 | sgpt "Write a concise git commit message for these staged changes."139```140141For local AI work, GPU state is useful context:142143```bash144nvidia-smi \145 | sgpt "Summarise GPU utilisation and whether this looks healthy for local LLM inference."146```147148And because Ollama exposes a local API, I can query it and ask ShellGPT to make the output more readable:149150```bash151curl -s http://localhost:11434/api/tags \152 | sgpt "List the available Ollama models and explain which one looks like the default candidate."153```154155This is the part I liked most. The terminal already has the context. ShellGPT lets the local model read that context without me manually copying it somewhere else.156157## Generating Shell Commands From Human Language158159ShellGPT has a shell command mode:160161```bash162sgpt --shell "find all markdown files changed in the last 7 days and print their titles"163```164165or with the short flag:166167```bash168sgpt -s "show me the ten largest files under /var/log modified in the last week"169```170171It returns a command and then gives an interactive choice to execute, describe, or abort.172173That interaction is important. I do not want a local model blindly running system commands on my behalf. I want it to propose a command, let me inspect it, ask it to describe the command if needed, then decide whether to run it.174175One detail I appreciated is that ShellGPT is smart enough to account for the shell you are using. Bash, Zsh, and Fish all have slightly different syntax and conventions, and the generated commands or scripts can reflect that instead of treating every terminal as generic POSIX `sh`. That matters once you get into arrays, globbing, completions, aliases, functions, and shell-specific quality-of-life features.176177Some examples that fit my normal workflow:178179```bash180sgpt -s "find markdown posts that contain the word ollama and sort them newest first"181```182183```bash184sgpt -s "show listening TCP ports with process names on Ubuntu"185```186187```bash188sgpt -s "compress all png files in this directory into a tar.gz archive named images-backup.tar.gz"189```190191```bash192sgpt -s "use curl to check whether http://localhost:11434/api/tags is responding"193```194195```bash196sgpt -s "create a one-line command to list the current GPU name, driver version, and used VRAM"197```198199This is where ShellGPT feels different from a normal chatbot. I am not asking for a tutorial. I am asking for the exact command I need right now. I'm going to be using this a lot for tcpdump options - I always forget these!200201That is a small shift, but it changes the feel of the workflow.202203## Building Bash Scripts From Plain English204205The best use case today was using ShellGPT to draft more complex bash scripts.206207For example, I can ask:208209```bash210sgpt --code "Write a bash script called ollama-health.sh that:211- checks whether the Ollama API is listening on localhost:11434212- prints the installed Ollama models213- sends a short test prompt to gemma4-26b214- prints GPU memory usage with nvidia-smi215- exits non-zero if any check fails"216```217218That is the sort of script I would normally write myself, but it takes a few minutes of looking up curl flags, remembering the Ollama API shape, and deciding how neat I want the error handling to be.219220ShellGPT can produce a first draft immediately:221222```bash223#!/usr/bin/env bash224set -euo pipefail225226MODEL="${1:-gemma4-26b}"227OLLAMA_URL="${OLLAMA_URL:-http://127.0.0.1:11434}"228229echo "Checking Ollama API..."230curl -fsS "$OLLAMA_URL/api/tags" >/dev/null231232echo "Installed models:"233ollama list234235echo "Testing model: $MODEL"236ollama run "$MODEL" "Reply with one short sentence confirming you are ready."237238echo "GPU memory:"239nvidia-smi --query-gpu=name,memory.used,memory.total --format=csv240```241242Would I run that blindly on a production host? No.243244Would I use it as a starting point, review it, tighten it, and save myself some time? Absolutely.245246Another useful prompt:247248```bash249sgpt --code "Write a bash script that scans a directory of markdown blog posts, extracts YAML title and date fields, and prints a sorted table of date, title, and filename."250```251252Or:253254```bash255sgpt --code "Write a bash script that checks all Docker containers, prints unhealthy ones, shows their last 50 log lines, and exits 1 if any container is unhealthy."256```257258Or:259260```bash261sgpt --code "Write a bash script that backs up /etc/nginx and /etc/systemd/system to a timestamped tar.gz file, writes a SHA256 checksum, and keeps only the last 10 backups."262```263264This is the sweet spot for me. I know enough bash to review the result. I know enough Linux to catch dangerous assumptions. What I want is acceleration: a draft that gets the boring structure in place so I can focus on the intent.265266## Installation With Ollama267268The upstream project is here:269270[ther1d/shell_gpt on GitHub](https://github.com/ther1d/shell_gpt)271272The basic install from the README is:273274```bash275python3 -m pip install --upgrade shell-gpt276```277278By default, ShellGPT uses OpenAI's API. To use Ollama, install ShellGPT with LiteLLM support:279280```bash281python3 -m pip install --upgrade "shell-gpt[litellm]"282```283284If Ollama is not already installed, the Linux installer is:285286```bash287curl -fsSL https://ollama.com/install.sh | sh288```289290If you are using the official ShellGPT Ollama guide as a quick test, it uses:291292```bash293ollama pull gemma4:e4b294```295296In my case, I already had a larger Gemma 4 26B quantized model imported into Ollama from earlier testing. The important thing is the model name that appears in `ollama list` (**IMPORTANT**: don't forget to prefix this with ollama/ - see below). Use that exact name in the ShellGPT configuration.297298Open the config file:299300```bash301nano ~/.config/shell_gpt/.sgptrc302```303304Set the Ollama-related values:305306```ini307DEFAULT_MODEL=ollama/gemma4-26b:latest308OPENAI_USE_FUNCTIONS=false309USE_LITELLM=true310API_BASE_URL=http://localhost:11434311OPENAI_API_KEY=0312```313314Replace `gemma4-26b:latest` with the exact model name from `ollama list`. If your model is called `gemma4:e4b`, use:315316```ini317DEFAULT_MODEL=ollama/gemma4:e4b318```319320Then test it:321322```bash323sgpt "Hello Ollama. Reply in one short sentence."324```325326If that works, try shell mode:327328```bash329sgpt -s "print the current directory, git branch, and latest commit"330```331332## A Few Practical Notes333334ShellGPT's own documentation notes that local models may not behave exactly like the default hosted OpenAI path. That matches my experience. Local models are good enough to be useful, but they still need supervision.335336A few habits make the workflow safer:337338- Use `--shell` when you want a command, because the interactive execute, describe, and abort flow is useful.339- Ask for a description before running anything destructive.340- Avoid piping `--no-interaction` directly into `bash`.341- Treat generated scripts as drafts.342- Keep Ollama bound to localhost unless you have a deliberate reason to expose it.343- Use the exact model name from `ollama list`.344345The last point sounds mundane, but it matters. With Ollama, the name is the contract. If the model is registered as `gemma4-26b:latest`, that is what ShellGPT and LiteLLM need to see.346347## The Takeaway348349My first impression is that ShellGPT is not most interesting as "chat in a terminal."350351That is fine, but familiar.352353It is more interesting as a shell-native bridge to a local model. I can pipe logs into it, ask it to interpret real command output, generate one-off commands, and draft bash scripts in the place where those scripts are going to run.354355That makes local AI feel less like a separate destination and more like part of the operating environment.356357For a local Gemma 4 model running through Ollama on the RTX PRO 4000 Blackwell, that is exactly the kind of workflow I wanted to test. The model does not need to be perfect. It needs to be nearby, private, fast enough, and useful in the flow of work.358359Today, ShellGPT cleared that bar and will be part of my toolkit.360361## Links362363- [ther1d/shell_gpt on GitHub](https://github.com/ther1d/shell_gpt)364- [ShellGPT Ollama wiki page](https://github.com/TheR1D/shell_gpt/wiki/Ollama)365- [Ollama](https://ollama.com/)366- [My earlier post: Running Modern Edge LLMs on an RTX PRO 4000 Blackwell](/post/running-edge-llms-rtx-pro-4000-blackwell)367