Buckets:
| import pandas as pd, numpy as np | |
| idx = pd.read_parquet('/tmp/two/index_timeseries.parquet') | |
| print('cols', list(idx.columns)) | |
| print() | |
| over = idx[idx.t0_bin20 + 40 > idx.n_bins_20ms] | |
| print('windows overrunning segment:', len(over)) | |
| print() | |
| print('per source stats:') | |
| print(idx.groupby('source').agg(segs=('segment_idx','nunique'), win=('window_idx','size'), | |
| units_min=('n_units','min'), units_max=('n_units','max'), | |
| bins_min=('n_bins_20ms','min'), bins_max=('n_bins_20ms','max'), | |
| ev_mean=('explained_var','mean')).round(3).to_string()) | |
| print() | |
| print('windows per segment (describe):') | |
| print(idx.groupby(['source','segment_idx']).size().groupby('source').describe().round(1).to_string()) | |
| print() | |
| print('dup (segment_idx, win_in_segment):', | |
| idx.duplicated(['segment_idx','win_in_segment']).sum()) | |
| print('domain_l1 x region:') | |
| print(idx.groupby(['region','domain_l1']).size().to_string()) | |
Xet Storage Details
- Size:
- 908 Bytes
- Xet hash:
- d822c276fd642b644a344e7b575ddf0f1bbaddfd1a013022eacb7410ceacadbe
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.