Skip to main content

Class: SentenceSplitter()

Parse text with a preference for complete sentences.

Extends​

SentenceSplitter<Options>(nodes, options?): Promise<BaseNode<Metadata>[]>

Parse text with a preference for complete sentences.

Type Parameters​

• Options extends Record<string, unknown>

Parameters​

• nodes: BaseNode<Metadata>[]

• options?: Options

Returns​

Promise<BaseNode<Metadata>[]>

Defined in​

packages/core/node-parser/dist/index.d.ts:32

Constructors​

new SentenceSplitter()​

new SentenceSplitter(params?): SentenceSplitter

Parameters​

• params?: object & SplitterParams

Returns​

SentenceSplitter

Overrides​

MetadataAwareTextSplitter.constructor

Defined in​

packages/core/node-parser/dist/index.d.ts:54

Properties​

chunkOverlap​

chunkOverlap: number

The token overlap of each chunk when splitting.

Defined in​

packages/core/node-parser/dist/index.d.ts:41

chunkSize​

chunkSize: number

The token chunk size for each chunk.

Defined in​

packages/core/node-parser/dist/index.d.ts:37

id​

id: string

Inherited from​

MetadataAwareTextSplitter.id

Defined in​

packages/core/schema/dist/index.d.ts:171

includeMetadata​

includeMetadata: boolean

Inherited from​

MetadataAwareTextSplitter.includeMetadata

Defined in​

packages/core/node-parser/dist/index.d.ts:6

includePrevNextRel​

includePrevNextRel: boolean

Inherited from​

MetadataAwareTextSplitter.includePrevNextRel

Defined in​

packages/core/node-parser/dist/index.d.ts:7

paragraphSeparator​

paragraphSeparator: string

Separator between paragraphs.

Defined in​

packages/core/node-parser/dist/index.d.ts:49

secondaryChunkingRegex​

secondaryChunkingRegex: string

Backup regex for splitting into sentences.

Defined in​

packages/core/node-parser/dist/index.d.ts:53

separator​

separator: string

Default separator for splitting into words

Defined in​

packages/core/node-parser/dist/index.d.ts:45

tokenSize()​

tokenSize: (text) => number

Parameters​

• text: string

Returns​

number

Defined in​

packages/core/node-parser/dist/index.d.ts:58

Methods​

_splitText()​

_splitText(text, chunkSize): string[]

Parameters​

• text: string

• chunkSize: number

Returns​

string[]

Defined in​

packages/core/node-parser/dist/index.d.ts:57

getMetadataString()​

protected getMetadataString(node): string

Parameters​

• node: TextNode<Metadata>

Returns​

string

Inherited from​

MetadataAwareTextSplitter.getMetadataString

Defined in​

packages/core/node-parser/dist/index.d.ts:21

getNodesFromDocuments()​

getNodesFromDocuments(documents): TextNode<Metadata>[]

Parameters​

• documents: TextNode<Metadata>[]

Returns​

TextNode<Metadata>[]

Inherited from​

MetadataAwareTextSplitter.getNodesFromDocuments

Defined in​

packages/core/node-parser/dist/index.d.ts:11

parseNodes()​

protected parseNodes(nodes): TextNode<Metadata>[]

Parameters​

• nodes: TextNode<Metadata>[]

Returns​

TextNode<Metadata>[]

Inherited from​

MetadataAwareTextSplitter.parseNodes

Defined in​

packages/core/node-parser/dist/index.d.ts:22

postProcessParsedNodes()​

protected postProcessParsedNodes(nodes, parentDocMap): TextNode<Metadata>[]

Parameters​

• nodes: TextNode<Metadata>[]

• parentDocMap: Map<string, TextNode<Metadata>>

Returns​

TextNode<Metadata>[]

Inherited from​

MetadataAwareTextSplitter.postProcessParsedNodes

Defined in​

packages/core/node-parser/dist/index.d.ts:9

splitText()​

splitText(text): string[]

Parameters​

• text: string

Returns​

string[]

Overrides​

MetadataAwareTextSplitter.splitText

Defined in​

packages/core/node-parser/dist/index.d.ts:56

splitTextMetadataAware()​

splitTextMetadataAware(text, metadata): string[]

Parameters​

• text: string

• metadata: string

Returns​

string[]

Overrides​

MetadataAwareTextSplitter.splitTextMetadataAware

Defined in​

packages/core/node-parser/dist/index.d.ts:55

splitTexts()​

splitTexts(texts): string[]

Parameters​

• texts: string[]

Returns​

string[]

Inherited from​

MetadataAwareTextSplitter.splitTexts

Defined in​

packages/core/node-parser/dist/index.d.ts:15

splitTextsMetadataAware()​

splitTextsMetadataAware(texts, metadata): string[]

Parameters​

• texts: string[]

• metadata: string[]

Returns​

string[]

Inherited from​

MetadataAwareTextSplitter.splitTextsMetadataAware

Defined in​

packages/core/node-parser/dist/index.d.ts:20