- Convert remaining xlsx files to parquet: db-polymer-properties,
db-D-interp-37-clean, db-D-interp-50-clean
- Update quantity_module/functions.py to read new parquet files
- Untrack all xlsx files from git and add *.xlsx to .gitignore
- Update convert_to_parquet.py to include newly converted files
Co-Authored-By: Claude Code <noreply@anthropic.com>
Update qrf_train.py to use Parquet files instead of Excel
- Replace pd.read_excel with pd.read_parquet for all three data files
- Derive colnames_mordred from df_desc.columns instead of instantiating
a full mordred Calculator (removes unnecessary mordred import)
- Add qrf_parameters_allT.xlsx -> .parquet to convert_to_parquet.py
- Commit converted qrf_parameters_allT.parquet
Co-Authored-By: Claude Code <noreply@anthropic.com>
Convert large Excel data files to Parquet for faster startup
- Add convert_to_parquet.py one-time conversion script
- Convert 4 large xlsx files (~42 MB total) to parquet (~11 MB total):
qrf/db-D-interp-allT-semiclean: 12MB -> 2.4MB
qrf/mordred-descriptors: 9.8MB -> 6.7MB
quantity_module/data/db-D-interp-allT-clean: 10MB -> 2.0MB
quantity_module/data/data-descriptors-mordred-numconfs51: 9.8MB -> 23KB
quantity_module/data/solvent-viscosity, vrentas-duda-params (both sheets)
- Update qrf/functions.py and quantity_module/functions.py to use parquet
- In qrf/functions.py: replace mordred Calculator init for column names with
direct column lookup from df_desc; share a single _calc_all/_desc_by_name
at module level instead of re-creating Calculator per request in QRF_Apply
- Add pyarrow<16 to requirements.txt
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Cache QRF model bundle loading with lru_cache: models are now loaded
from disk once per temperature and reused across requests instead of
re-reading the pickle file on every POST
- Reduce Monte Carlo sample size from 1e6 to 1e5: 10x faster per
request with statistically indistinguishable results
Co-Authored-By: Claude Code <noreply@anthropic.com>
