Compatibility: Only available on Node.js.
DirectoryLoader document loaders. For detailed documentation of all DirectoryLoader features and configurations head to the API reference.
This example goes over how to load data from folders with multiple files. The second argument is a map of file extensions to loader factories. Each file will be passed to the matching loader, and the resulting documents will be concatenated together.
Example folder:
src/document_loaders/example_data/example/
├── example.json
├── example.jsonl
├── example.txt
└── example.csv
Overview
Integration details
Setup
To access DirectoryLoader document loader you’ll need to install the langchain package.
Installation
The LangChain DirectoryLoader integration lives in the langchain package:
npm install langchain @langchain/core
Instantiation
Now we can instantiate our model object and load documents:
import { DirectoryLoader } from "@langchain/classic/document_loaders/fs/directory";
import {
JSONLoader,
JSONLinesLoader,
} from "@langchain/classic/document_loaders/fs/json";
import { TextLoader } from "@langchain/classic/document_loaders/fs/text";
import { CSVLoader } from "@langchain/community/document_loaders/fs/csv";
const loader = new DirectoryLoader(
"../../../../../../examples/src/document_loaders/example_data",
{
".json": (path) => new JSONLoader(path, "/texts"),
".jsonl": (path) => new JSONLinesLoader(path, "/html"),
".txt": (path) => new TextLoader(path),
".csv": (path) => new CSVLoader(path, "text"),
}
);
Load
const docs = await loader.load()
// disable console.warn calls
console.warn = () => {}
docs[0]
Document {
pageContent: 'Foo\nBar\nBaz\n\n',
metadata: {
source: '/Users/bracesproul/code/lang-chain-ai/langchainjs/examples/src/document_loaders/example_data/example.txt'
},
id: undefined
}
console.log(docs[0].metadata)
{
source: '/Users/bracesproul/code/lang-chain-ai/langchainjs/examples/src/document_loaders/example_data/example.txt'
}
API reference
For detailed documentation of all DirectoryLoader features and configurations head to the API reference.
