RecursiveCharacterTextSplitter class

Implementation of splitting text that looks at characters. Recursively tries to split by different characters to find one that works.

Inheritance
Implementers

Constructors

RecursiveCharacterTextSplitter.new({List<String> separators = const ['\n\n', '\n', ' ', ''], int chunkSize = 4000, int chunkOverlap = 200, int lengthFunction(String) = TextSplitter.defaultLengthFunction, bool keepSeparator = true, bool addStartIndex = false})
Implementation of splitting text that looks at characters. Recursively tries to split by different characters to find one that works.
const

Properties

addStartIndex → bool
If true, includes chunk's start_index in metadata.
finalinherited
chunkOverlap → int
Overlap in characters between chunks.
finalinherited
chunkSize → int
Maximum size of chunks to return.
finalinherited
defaultOptions → BaseLangChainOptions
The default options to use when invoking the Runnable.
finalinherited
hashCode → int
The hash code for this object.
no setterinherited
keepSeparator → bool
Whether to keep the separator in the chunks.
finalinherited
lengthFunction → int Function(String)
Function that measures the length of given chunks.
finalinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
separators → List<String>
List of separators to use for splitting.
final

Methods

batch(List<List<Document>> inputs, {List<BaseLangChainOptions>? options}) → Future<List<List<Document>>>
Batches the invocation of the Runnable on the given inputs.
inherited
bind(BaseLangChainOptions options) → RunnableBinding<List<Document>, BaseLangChainOptions, List<Document>>
Binds the Runnable to the given options.
inherited
close() → void
Cleans up any resources associated with it the Runnable.
inherited
createDocuments(List<String> texts, {List<String>? ids, List<Map<String, dynamic>>? metadatas}) → List<Document>
Creates documents from a list of texts.
inherited
getCompatibleOptions(RunnableOptions? options) → BaseLangChainOptions?
Returns the given options if they are compatible with the Runnable, otherwise returns null.
inherited
invoke(List<Document> input, {BaseLangChainOptions? options}) → Future<List<Document>>
Transform a list of documents.
inherited
mergeSplits(List<String> splits, String separator) → List<String>
Merges smaller pieces into medium size chunks to send to the LLM.
inherited
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
pipe<NewRunOutput extends Object?, NewCallOptions extends RunnableOptions>(Runnable<List<Document>, NewCallOptions, NewRunOutput> next) → RunnableSequence<List<Document>, NewRunOutput>
Pipes the output of this Runnable into another Runnable using a RunnableSequence.
inherited
splitDocuments(List<Document> documents) → List<Document>
Splits documents.
inherited
splitText(String text) → List<String>
Split text into multiple components.
override
stream(List<Document> input, {BaseLangChainOptions? options}) → Stream<List<Document>>
Streams the output of invoking the Runnable on the given input.
inherited
streamFromInputStream(Stream<List<Document>> inputStream, {BaseLangChainOptions? options}) → Stream<List<Document>>
Streams the output of invoking the Runnable on the given inputStream.
inherited
toString() → String
A string representation of this object.
inherited
transformDocuments(List<Document> documents) → Future<List<Document>>
Transform a list of documents.
inherited
withFallbacks(List<Runnable<List<Document>, RunnableOptions, List<Document>>> fallbacks) → RunnableWithFallback<List<Document>, List<Document>>
Adds fallback runnables to be invoked if the primary runnable fails.
inherited
withRetry({int maxRetries = 3, FutureOr<bool> retryIf(Object e)?, List<Duration?>? delayDurations, bool addJitter = false}) → RunnableRetry<List<Document>, List<Document>>
Adds retry logic to an existing runnable.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited