refactor(data_processing): enhance chunking and embedding generation
69b7911
YanBoChen
commited on
Spaces:
Sleeping
Sleeping
Commit History
WIP: Enhance dual keyword chunking to include pre-calculated metadata for treatment chunks
c0317b2
YanBoChen
commited on
feat(data_processing): Implement token length control with semantic preservation
922ed80
YanBoChen
commited on
refactor(data_processing): optimize chunking strategy with token-based approach
87dcd9d
YanBoChen
commited on
feat(data-processing): implement data processing pipeline with embeddings
68cfce0
YanBoChen
commited on