Skip to main content
문자 기반 분할은 텍스트 분할의 가장 간단한 방식입니다. 지정된 문자 시퀀스(기본값: "\n\n")를 사용하여 텍스트를 나누며, 청크 길이는 문자 개수로 측정됩니다. 주요 사항:
  1. 텍스트 분할 방식: 지정된 문자 구분자를 기준으로 분할합니다.
  2. 청크 크기 측정 방식: 문자 개수로 측정합니다.
다음 중에서 선택할 수 있습니다:
  • .splitText — 일반 문자열 청크를 반환합니다.
  • .createDocuments — LangChain Document 객체를 반환하며, 하위 작업에서 메타데이터를 보존해야 할 때 유용합니다.
npm install @langchain/textsplitters

import { CharacterTextSplitter } from "@langchain/textsplitters";
import { readFileSync } from "fs";

// 예제: 긴 문서 읽기
const stateOfTheUnion = readFileSync("state_of_the_union.txt", "utf8");

const splitter = new CharacterTextSplitter({
    separator: "\n\n",
    chunkSize: 1000,
    chunkOverlap: 200,
});
const texts = splitter.createDocuments([{ pageContent: stateOfTheUnion }]);
console.log(texts[0]);

Document {
    pageContent: 'Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \n\nLast year COVID-19 kept us apart. This year we are finally together again. \n\nTonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \n\nWith a duty to one another to the American people to the Constitution. \n\nAnd with an unwavering resolve that freedom will always triumph over tyranny. \n\nSix days ago, Russia’s Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \n\nHe thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \n\nHe met the Ukrainian people. \n\nFrom President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world.'
}
각 문서와 연결된 메타데이터를 출력 청크에 전파하려면 .createDocuments를 사용하세요: 
const metadatas = [{"document": 1}, {"document": 2}]
const documents = splitter.createDocuments(
    [{ pageContent: stateOfTheUnion }, { pageContent: stateOfTheUnion }],
    { metadatas: metadatas }
);
console.log(documents[0]);

Document {
    pageContent: 'Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \n\nLast year COVID-19 kept us apart. This year we are finally together again. \n\nTonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \n\nWith a duty to one another to the American people to the Constitution. \n\nAnd with an unwavering resolve that freedom will always triumph over tyranny. \n\nSix days ago, Russia’s Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \n\nHe thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \n\nHe met the Ukrainian people. \n\nFrom President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world.',
    metadata: {'document': 1}
}
문자열 콘텐츠를 직접 얻으려면 .splitText를 사용하세요: 
splitter.splitText(stateOfTheUnion)[0]

'Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \n\nLast year COVID-19 kept us apart. This year we are finally together again. \n\nTonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \n\nWith a duty to one another to the American people to the Constitution. \n\nAnd with an unwavering resolve that freedom will always triumph over tyranny. \n\nSix days ago, Russia’s Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \n\nHe thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \n\nHe met the Ukrainian people. \n\nFrom President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world.'
Edit the source of this page on GitHub.
Connect these docs programmatically to Claude, VSCode, and more via MCP for real-time answers.
⌘I