how to handle multi-line string
#9
by StephennFernandes - opened
. i handle concurrent user request by dynamically batching them together and inferencing the as a batch.
However i wanted to know i i could somehow handle new line \n in my input string and the translated response retained the new line token somehow (eg: \n or any special token).
Ive tried other alternative solutions like slicing the input string on the \n and inferencing these separate pieces as a batch and merging them together, but concurrent throughput in my application suffers.
please let me know incase of a possible solution or a hacky way of how i could achieve this