Skip to main content

​
deduplicate_chunks

Function
This function is defined in the rag module.
Deduplicate chunks by content similarity.

​
Signature

def deduplicate_chunks(chunks: Vec<ContextChunk>, _threshold: f32) -> Vec<ContextChunk>

​
Parameters

​
chunks
Vec<ContextChunk>
required
No description available.
​
_threshold
f32
required
No description available.

​
Returns

​
Returns
Vec<ContextChunk>
The result of the operation.

​
Uses

  • any
  • comparison

​
Source

View on GitHub

praisonai/src/rag/mod.rs at line 623

Rust Chunking

Rust RAG