LOST

Local Offline Search Tool — a web search engine that runs entirely on your computer.

The database is stored locally, nothing is ever sent anywhere, and there's no search history. It keeps finding results with no internet connection at all.

Usage

Open desktop application (GUI)

lost

Running local web server

lost -s                             # web UI and Web API on http://127.0.0.1:1057
lost -s -a 0.0.0.0 -p 8080          # serve the local network on port 8080
lost -b                             # same as -s, but in the background
lost -w                             # run desktop app and web server together

Searching straight in the terminal

lost hacker news                    # Search for hacker news website
lost cache:https://slashdot.org     # View stored copy in the terminal
lost -f json rust -site:crates.io   # Output search results formatted as JSON
lost --suggest "rust comp"          # autocomplete
lost --suggest "how to _ catfish"   # autocomplete, fill in the blank

Working with datasets

lost -d hacker-news.csv           # import a BigQuery export of bigquery-public-data.hacker_news.full
lost -d CC-MAIN-…-00000.warc.wet.gz CC-MAIN-…-00000.warc.wat.gz   # import Common Crawl data
lost --imports                    # list past imports to see what's in the database
lost --remove-import 3            # remove import along with all pages that came from it
lost -e rust.lost rust compiler   # export what a search finds (or everything: lost -e all.lost)
lost -e rust.lost --sign rust     # the same, signed with your ID (exports are unsigned by default)

Supported search operators:

• "exact phrase"
• "how to * catfish"
• -exclude
• OR
• ( )
• site:
• intitle:
• allintitle:
• inurl: (inurl:gemini://, inurl:.onion/)
• allinurl:
• intext:
• allintext:
• filetype:
• link:
• contains: (contains:pdf: pages linking to PDFs)
• related: (related:example.com)
• lang: / language: (lang:de, language:german)
• near: (catfish near:3 cornmeal)
• word* (catf*)

Data

LOST searches through whatever datasets you import. It comes out of the box with two pages (the front pages of Hacker News and Slashdot), so you could start seeing it in action right away.

Supported dataset formats

• Common Crawl (WET and WAT files)
• Hacker News (BigQuery CSV export)
• .lost files — LOST's own format
• Gemini, Gopher, Spartan, Nex, Finger, .onion, I2P, IPFS, Arweave, Nostr, TON and Swarm — crawled with lost-spider (see below)

Datasets

Common Crawl

Common Crawl is a 501(c)(3) American non-profit organization that crawls the internet and provides a free, open-source archive of web data to the public. Their datasets are available for free to anyone here. The combined full size of their latest dataset is over 20TiB.

Here's one of the ways to obtain a small piece of their data:

curl -s https://data.commoncrawl.org/crawl-data/CC-MAIN-2026-39/wet.paths.gz | zcat | head -c 500000000 > sample.wet
curl -s https://data.commoncrawl.org/crawl-data/CC-MAIN-2026-39/wat.paths.gz | zcat | head -c 500000000 > sample.wat

Hacker News

The Hacker News public dataset is available on Google BigQuery. The dataset is part of the Google Cloud Marketplace Public Datasets and contains over 50 million rows (stories, comments, job posts) derived directly from the official Hacker News Firebase API. Merging this dataset into your LOST database will allow you to search through pages of Hacker News locally. The full size exceeds 50GiB.

Sample SQL command to select a chunk of data:

SELECT * FROM `bigquery-public-data.hacker_news.full` WHERE timestamp >= '2026-01-01' AND timestamp < '2026-10-10';

LOST files

LOST is capable of importing and exporting .lost dataset files, which makes it possible to freely and safely exchange crawl data with other LOST users. They aren't signed unless you ask (lost -e FILE --sign, or the "Sign it" checkbox): a signature lets others tell which files are yours and keeps your database from getting corrupted or spammed, but it also says who made the file. Both exports and imports can be made either via CLI (see Usage) or GUI (Settings → Data).

Crawling: the small internet, darknets and web3

LOST itself never goes online. For Geminispace, Gopherspace, Spartan, Nex and Finger, .onion sites, I2P sites, IPFS, Arweave, Nostr, TON Sites and Swarm there's lost-spider, a separate crawler that isn't built or installed by default:

cargo install lost --features spider               # or: make spider
lost-spider gemini                                 # crawl all of Geminispace
lost-spider gemini --link example.org -o my.lost   # start at a capsule nothing links to
lost-spider gemini --link example.org --dont-crawl # only that page, nothing else
lost-spider gemini -c -o my.lost                   # carry on where it stopped
lost-spider gopher                                 # crawl all of Gopherspace
lost-spider spartan                                # Spartan capsules (Gemini without TLS)
lost-spider nex                                    # Nex servers
lost-spider finger --link finger://plan.cat/someone # people's .plans
lost-spider onion                                  # .onion sites, through Tor (127.0.0.1:9050)
lost-spider i2p                                    # I2P sites, through an I2P router (127.0.0.1:4444)
lost-spider ipfs                                   # IPFS sites, through an IPFS node's gateway (127.0.0.1:8080)
lost-spider ipfs --link vitalik.eth                # an ENS name, or a CID, or an IPNS name
lost-spider arweave                                # the permaweb, through a public gateway
lost-spider nostr                                  # Nostr: notes, articles and profiles, from relays
lost-spider nostr --link npub1…                    # someone's profile and posts first
lost-spider nostr --search "bitcoin mining"        # and what relays find for that
lost-spider ton                                    # TON Sites, through a TON proxy (127.0.0.1:8080)
lost-spider swarm                                  # Swarm sites, through a public gateway
lost-spider swarm --gateway http://127.0.0.1:1633  # … or through your own Bee node
lost-spider gemini --proxy socks5://127.0.0.1:9050 # Gemini through Tor
lost -d my.lost                                    # import what it found

It obeys robots.txt, asks each server for one thing a second (--delay), and saves everything into one .lost file, unsigned unless you ask for --sign. The file is complete even if you stop it with Ctrl-C. See lost-spider --help for search boxes, blacklists, proxies and limits.

onion needs Tor running, ton a TON proxy (rldp-http-proxy -p 8080 -c 3333 -C global.config.json, or Tonutils Proxy), i2p an I2P router (i2pd or Java I2P), and ipfs an IPFS node (Kubo, ipfs daemon), or --gateway https://ipfs.io to ask a public gateway instead (one request at a time). arweave needs nothing: it asks a public gateway (one request at a time), and also goes through everything Arweave stores, newest first, besides following links. Neither does swarm, but public Swarm gateways only serve the sites they allow, so a Bee node of your own (--gateway http://127.0.0.1:1633) reaches more; Swarm has no index, so it follows links only. nostr needs nothing either: it lists what well-known relays have, newest first, and goes from authors to their profiles and the relays they use; every post's signature is checked, and posts meant to expire, or that their authors asked to delete, aren't kept. With --search, relays that can search are asked for each phrase, and an npub1… line in a --blacklist skips that author's posts. I2P is slow: reaching a site for the first time can take a minute or more, so its timeout is 120 seconds by default (-t changes it, for any network), and on a router (or Tor) that has just started, lost-spider waits a few minutes until it can reach sites. Images and other files are never downloaded, only their links. Before crawling .onion sites, lost-spider gets Ahmia's list of onion sites banned for child abuse material and skips them; it won't start without that list. Darknets host all sorts of things, so it's worth keeping a --blacklist with title:/…/ and body:/…/ rules for what you don't want in your dataset.

Database

The local database is stored by default either in ~/.lost/ (Linux, BSD), ~/Library/Application Support/LOST/ (macOS), or %LOCALAPPDATA%\LOST\ (Windows, i.e. C:\Users\NAME\AppData\Local\LOST\).

Building

Everything

cargo build --release

Only CLI

cargo build --release --no-default-features --features cli

CLI & web server

cargo build --release --no-default-features --features cli,server

CLI & GUI

cargo build --release --no-default-features --features cli,gui

Bare library

cargo build --release --no-default-features --lib

Spider (crawler)

cargo build --release --no-default-features --features spider --bin lost-spider

Packages

make tarball     # (see the Makefile)
make deb         # needs cargo-deb
make appimage    # needs appimagetool

Installation

Download a package from the releases:

• Debian, Ubuntu, Mint: sudo apt install ./lost_…_amd64.deb
• Any Linux: the .AppImage — make it executable and run it. Or unpack the tarball and run sudo sh dist/install.sh.
• macOS: unzip, drag LOST to Applications. The first time, right-click it and choose Open (it isn't signed yet).
• Windows: unzip and run lost.exe.

To run the web server as a service (Linux, after installing the .deb or with install.sh): sudo systemctl enable --now lost, then open http://127.0.0.1:1057.

From source: make && sudo make install (sudo make uninstall to remove).

License

0BSD.

res/web/mikado.light.min.js is Mikado, © Nextapps GmbH, Apache-2.0.