File types

The tool reads the first bytes of a file, and the classifiers decide its type from those bytes. Then:

• The first converter in converters that accepts the type reads the file.
• With no converter, the tool reads a text file as lines.
• With no converter, any other type returns status unsupported, with a note that names the type.

The default converters is [imageConverter()], so by default the tool reads UTF-8 text and images. A Jupyter notebook is JSON text, so the tool reads it as lines. Other types return unsupported:

[read:unsupported-pdf] This content is not supported as UTF-8 text. Use a PDF extraction tool.

A folder is refused with status error and code NOT_A_FILE, unless a converter lists it.

Every integration has the same default. A list you set replaces it, so keep imageConverter() in the list to read images. Set converters: [] to turn images off.

Add converters

Set converters to a list. The first converter that accepts a file reads it:

import { nodeFileSystem } from "@better-fs-tools/node";
import {
  createReadTool,
  defaultClassifiers,
  directoryListing,
  imageConverter,
  notebookClassifier,
  notebookConverter,
} from "@better-fs-tools/read";

const cwd = process.cwd();
const fs = nodeFileSystem({ cwd, allowedRoots: [cwd] });
const read = createReadTool({
  fs,
  classifiers: [notebookClassifier(), ...defaultClassifiers()],
  converters: [imageConverter(), notebookConverter(), directoryListing()],
});

Converted text works like a text file. offset, limit, the limits, and the continuation apply to the converted lines. result.conversion names the converter that read the file.

Read images

With imageConverter(), the default, an image returns status media. result.content holds the image:

const result = await read({ path: "docs/diagram.png" });

result.status; // "media"
result.content; // [{ type: "media", mediaType: "image/png", data: Uint8Array, name: "diagram.png" }]

The AI SDK integration sends the image to the model as a file part. The Pi integration sends it as an image part. In your own integration, send the media parts in the form your model API takes.

Set transform to make large images smaller. It gets the bytes and the media type, and returns new bytes of the same media type:

declare function resize(bytes: Uint8Array, mediaType: string, width: number): Promise<Uint8Array>;

imageConverter({ transform: (bytes, mediaType) => resize(bytes, mediaType, 1024) });

Read notebooks

By default a notebook is JSON text. A read with limit: 4 returns the first lines of the JSON:

1|{
2| "cells": [
3|  {
4|   "cell_type": "markdown",

[read:continue] Output stopped at the line limit. Continue with {"path":"notebooks/explore.ipynb","offset":5,"limit":4}.

To read a notebook cell by cell, put notebookClassifier() before the default classifiers, and add notebookConverter(), as in the example above. notebookConverter() accepts only the NOTEBOOK type of notebookClassifier(), so without the classifier it does nothing. The converter gives each cell a header line, then its source, then its text outputs:

1|[cell 1: markdown]
2|# Explore
3|
4|[cell 2: code]
5|1 + 1
6|[output]
7|2

Read PDFs and other formats

textConverter() wraps your own text extractor, for example pdftotext, pandoc, or markitdown. accepts decides which files it reads. run gets the file bytes and returns the text:

import { textConverter } from "@better-fs-tools/read";

declare function pdfToText(source: AsyncIterable<Uint8Array>): AsyncIterable<string>;

const pdf = textConverter({
  id: "pdftotext",
  accepts: (match) =>
    match.classification.kind === "unsupported" && match.classification.code === "PDF",
  mimeType: "text/plain",
  run: (source) => pdfToText(source),
});

The converter runs again for each continuation, so cache the output if it is slow. A converter that starts a process needs Node or Bun, and does not work in a Cloudflare Worker.

With the default classifiers, match.classification.code is one of BINARY, IMAGE, PDF, OFFICE_DOCUMENT, or UNKNOWN_ENCODING. notebookClassifier() adds NOTEBOOK. Your own classifier can add other codes.

List folders

directoryListing() lists a folder, one entry for each line. offset and limit then work on entries:

const read = createReadTool({
  fs,
  converters: [directoryListing({ trailingSlash: true, sort: "type-then-name" })],
});

1|docs/
2|logs/
3|notebooks/
4|src/
5|.env

A listing stops at 200 entries. Set limits.maxDirectoryEntries to change this.

Decide the file type

The default classifiers is defaultClassifiers(). It finds images, PDFs, Office documents, and binary files, and it treats the rest as UTF-8 text. A notebook is text to it. notebookClassifier() finds notebooks. Put it before the defaults. The first classifier that decides a type wins.

To decide by extension, put extensionClassifier() before the defaults:

import { defaultClassifiers, extensionClassifier } from "@better-fs-tools/read";

const read = createReadTool({
  fs,
  classifiers: [
    extensionClassifier({ unsupported: { ".log": { code: "BINARY" } }, text: [".lock"] }),
    ...defaultClassifiers(),
  ],
});

[read:unsupported-binary] Files with the .log extension are not supported as UTF-8 text.

Keep defaultClassifiers() in the list. Without it, the tool does not find images, PDFs, or binary files.

Set notes on defaultClassifiers() to change the note for a type:

defaultClassifiers({
  notes: { PDF: { message: "PDFs are not supported here. Ask the user for a text copy." } },
});

Size limits for converters

A converter reads at most 64 MiB of a file, and the media parts of one result can be at most 5 MiB. Past these limits, the read returns status unsupported with code TOO_LARGE. Set limits.maxConvertBytes and limits.maxMediaBytes to change them.

Write your own converter

A file converter is an object with an id, target: "file", an accepts(match) function, and an async convert(input, ctx) function. convert returns one of:

• { kind: "text", text, mimeType }, where text is a string or an async iterable of strings
• { kind: "media", parts } with media parts
• { kind: "refuse", code, note } to refuse the file

A folder converter has target: "directory" and no accepts. Use textConverter() when your converter returns text. The read reference lists every field.