Can split passages into sentences. Improved embedding, LLM representation models, improved zero shot capabilities

Updated packages. Improve hierarchy vis. Better models - mixedbread and phi3. Now option to split texts into sentences before modelling.

Hopefully now LLM download from hub should work

Some text changes. Fixed a couple of TF-IDF embeddings issues

Fixed file load with files including capital letters

Lots of general fixes. New visualisations, fixed hierarchical vis for zero shot. Added calc all probabilities.

Switched embeddings model to BGE Small 1.5 as Jina seemed unable to do zero shot topic modelling properly

Split off LLM representation, visualisation, and reduce outliers from main function. Added hierarchical visualisation and logs

App should now check if embeddings are loaded before topic modelling. And will save only once.

Returned TruncatedSVD components to 100 - higher values don't seem to help

Greatly increased low resource process dimensions for higher quality. Visualisations disabled by default to increase speed.

Greatly improved low resource mode speed (at cost of potential quality)

Fixed llm_config, environmental variable, zero shot topic model errors with quick embeddings

Model export changed to safetensors. Improved representational model function. Got zero shot topic modelling working

first commit