Add files using upload-large-folder tool

.gitignore

@@ -0,0 +1,9 @@
+/src/*.egg-info/
+*__pycache__*
+/.vscode/
+
+/data/
+/.old/
+/outputs*/
+/logs/
+hf_token.txt

Readme.md

@@ -0,0 +1,42 @@
+# Large Language Models adaptation using post-hoc calibration (PHC) and supervised fine-tuning (SFT)
+
+This repo contains the code to run adaptation of generative language models. The `scripts` directory contains the scripts to be run and everything (datasets, number of adaptation samples, base model) can be configured using the `scripts/env.sh` file. 
+
+## Adaptation methods
+
+Supported adaptation methods are distributed across different scripts files:
+- `scripts/cal_vs_samples.sh`: Compute the logits for the current model and then run Post-hoc calibration (PHC) with affine calibration for four different variants:
+  - Vector Scaling (**PHC-VS**): The matrix of the affine transformation is assumed to be diagonal.
+  - Direction Preserving (**PHC-DP**): The matrix of the affine transformation is replaced by a scalar, $\alpha$.
+  - Bias Only (**PHC-BO**): Same as PHC-DP, but with the weight $\alpha$ fixed at 1.
+  - Temperature Scaling (**PHC-TS**): Same as PHC-DP but with the bias term fixed at 0.
+- `scripts/lora_vs_samples.sh`: Run SFT using LoRA on all the datasets. There are three variants implemented depending on how we use the adaptation samples:
+  - **SFT-w-val**: we leave a portion of the adaptation set for validation, and stop training based on the CE loss on this set.
+  - **SFT-retrain**: we perform SFT with validation, as above, and keep track of the optimal number of steps. Then, we rerun SFT on the full training set, stopping after that same number of steps. 
+  - **SFT-wo-val**: we use all the available samples for training until convergence based on the training loss.
+SFT and computing the model posteriors is implemented using the [litgpt](https://github.com/Lightning-AI/litgpt) library.
+
+## Datasets and Models
+
+Currently, there are 5 supported datasets:
+- **SST-2**: The Stanford Sentiment Treebank, which includes movie reviews annotated as either positive or negative.
+- **AGNews**: a collection of news articles grouped into four categories. 
+- **DBPedia**: a dataset derived from Wikipedia, where each article is categorized into one of 14 topics.
+- **20NewsGroups**: posts from online newsgroups categorized into 20 different topics. 
+- **Banking77**: customer service queries related to online banking, classified into 77 intent categories. 
+
+All the models of the `litgpt>=0.5.3` library are supported, but for our experiments we used Llama3.2-1B and Qwen2.5-7B. Be sure to configure your token in order to use model that require permission.
+
+## Installation
+
+To use this you can clone it and pip-install it direclty but we recommend creating a separate conda enviroment:
+```bash
+conda create -n llmcal python=3.11
+conda activate llmcal
+pip install -r requirements.txt
+pip install -e .
+```
+
+## Cite
+
+Coming soon in OpenReview!

TODO

@@ -0,0 +1,130 @@
+- Terminar de correr todo hasta ahora
+- Para DBPedia, comparar DPCal con LoRA en función de la cantidad de muestras entrenando con un conjunto limitado de datos. Repetir para cuando el modelo fue previamente fine-tuneado con datos out-of-domain. Graficar Zero-shot. Ver plot dbpedia_nce. Debería verse que en algún momento LoRA supera a DPCal pero para pocas muestras, debería funcionar bien DPCal.
+
+DPCal helps on non-finetuned and fine-tuned-on-matched-data models:
+Zero-shot/Few-shot/LoraMatchedAns/LoraMatchedFS/BERT + DPCal/HistBinning/VectorScaling/etc
+
+You can choose DPCal if you have limited budget of data:
+Zero-shot/LoramatchedAns/LoraMatchedFS/DPCal vs number of samples
+
+If you have other data, calibrate after train:
+LoraMismatchedAns/LoraMismatchedFS/Instruct/InstructFewShot + DPCal (use zero-shot/zero-shot+dpcal as baseline)
+
+
+sin es con 100 deberia dar mejor que con 70
+el mejor deberia ser sin es calibrado en el 30
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+instruct base
+instruc + dpcal en todos
+instruct + 70-30 lora
+instruct + 70-30 lora + dpcal
+instruc + 100 lora (no es)
+instruct + 100 lora (con steps de lora70-30)
+
+
+
+Data split:
+- Set 1: first p% of the data
+- Set 2: last 100-p% of the data
+- Set 3: 100% of the data
+
+Methods:
+1. No Adaptation (baseline): Do not use training data
+2. Calibration: Use Set 3 for calibration
+
+3. LoRA p%: Use Set 1 for training LoRA and Set 2 for validation
+4. LoRA 100%: Use Set 3 for training LoRA with number of steps used in method 3.
+
+5. LoRA p% without ES: Use Set 1 for training LoRA without Early Stopping
+6. LoRA 100% without ES: Use Set 3 for training LoRA without Early Stopping
+
+7. LoRA p% + Cal: Method 3 plus calibration on Set 2.
+8. LoRA 100% + Cal: Method 4 plus calibration with parameters of method 7.
+
+9. LoRA p% without ES + Cal: Method 5 plus calibration on Set 2.
+10. LoRA 100% without ES + Cal: Method 6 plus calibration with parameters of method 9.
+
+* = No ES
+x = ES
+
+Red = LoRA p
+Green = LoRA 100
+
+Solid (-) = No calibration
+Dashed (--) = Temp Scaling
+Dotted (:) = DP Calibration
+
+
+
+icen laboratorio
+cerca del RR, tambien hay bus a la universidad
+estacion de orsay
+qualcum
+
+
+
+
+
+Cosas para el paper:
+
+- Hay que argumentar que un modelo robusto es un modelo que puede clasificar bien para cualquier set de respuestas que se le presente
+- Capaz podemos venderlo como un framework alternativo a prompt engineering en donde elgís las posibles respuestas y se las das como answers en lugar de en el prompt. 
+
+- Métodos PEFTs son efectivos: https://aclanthology.org/2021.acl-long.172.pdf
+- Ejemplo de catastrofic forgeting en medical domain: https://arxiv.org/abs/2203.13381
+- Otro ejemplo de catastrofic forgeting: https://arxiv.org/abs/2308.08747
+- Mixture of Prompts: https://arxiv.org/pdf/2310.02842
+- Filtrado de muestras para hacer in domain finetuning: https://arxiv.org/abs/2410.14745
+- Blog que habla sobre catastrofic forgetting: https://blog.arcee.ai/the-hidden-obstacles-of-domain-adaptation-in-llms/
+- Synthetic generation: https://arxiv.org/abs/2303.00807
+- Training budget guide: https://aclanthology.org/2024.eamt-1.51/
+- Prompt engineering: https://ieeexplore.ieee.org/abstract/document/9908590
+- Paper de continual pretraining: https://arxiv.org/abs/2004.10964
+
+
+
+
+No tiene que ver con el paper pero sí con el proyecto de pablo: https://arxiv.org/abs/2310.01444
+
+Mostrar NER y NCE con todos los métodos de adaptación
+otro plot con LoRA p=1.0 no ES con calibracion
+en dbpedia, comparar distribuciones en pocas mustras del sin calibrar y del calibrado.
+
+
+
+TODO:
+- Hacer y agregar plots de la primera sección de resultados
+- Hacer y agregar plots de la segunda sección de resultados
+- Análisis de las posteriors de las clases
+- Análisis de las posteriors por fuera de la distribución de tokens de interés
+- Escribir la primera parte de la sección de entrenamiento (descripción de LoRA y fine-tuning)
+- Agregar related works
+- Escribir el experimental setup
+- agregar las referencias al .bib
+- Terminar de correr Qwen y agregar al apéndice
+<!-- - Ver intervalos y ver si agrego muestras al test
+- Ver si tengo que correr más seeds -->
+<!-- - Correr / buscar en resultados viejos para muchas muestras en Llama3.2 y todos los datasets, y agregar calibración:
+    - Base model
+    - Instruct model
+    - Base trained on mismatched data
+    - Instruct trained on mismatched data
+    - LoRA p=1.0 con early stopping en Base
+    - LoRA p=1.0 con early stopping en Instruct -->
+
+
+

configs/methods2.yaml

@@ -0,0 +1,75 @@
+markersize: &markersize 5
+linewidth: &linewidth 4
+
+lora_1.0_no_es:
+  label: "SFT w/o Early Stopping"
+  color: "tab:blue"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_0.7_no_es:
+  label: "SFT w/o Early-stopping (70%)"
+  color: "tab:purple"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_0.7_no_es_plus_dpcal:
+  label: "SFT SFT w/o Early Stopping (70%) + DP Cal"
+  color: "tab:brown"
+  linestyle: "--"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_0.7_no_es_plus_tempscaling:
+  label: "SFT w/o Early Stopping (70%) + Temp Scaling"
+  color: "tab:cyan"
+  linestyle: "--"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+
+lora_1.0:
+  label: "SFT w/ Early Stopping"
+  color: "tab:orange"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_dpcal:
+  label: "SFT w/o Early Stopping\n+ Direction Preserving"
+  color: "tab:green"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_dpcal_naive:
+  label: "SFT w/o Early Stopping\n+ Direction Preserving (Naive)"
+  color: "tab:green"
+  linestyle: "--"
+  marker: "x"
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_tempscaling:
+  label: "SFT w/o Early Stopping\n+ Temperature Scaling"
+  color: "tab:green"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_tempscaling_naive:
+  label: "SFT w/o Early Stopping\n+ Temperature Scaling (Naive)"
+  color: "tab:orange"
+  linestyle: "--"
+  marker: "o"
+  linewidth: *linewidth
+  markersize: *markersize

configs/methods_final.yaml

@@ -0,0 +1,136 @@
+markersize: &markersize 6
+linewidth: &linewidth 4
+
+
+no_adaptation:
+  label: "NoA" # No Adaptation
+  color: "black"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+temp_scaling:
+  label: "PHC-TS" # Temperature Scaling
+  # color: "tab:green"
+  # color: "#c6dbef"
+  color: "#17becf"
+  # linestyle: "-."
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+dp_calibration:
+  label: "PHC-DP" # Direction Preserving
+  # color: "tab:red"
+  # color: "#9ecae1"
+  color: "#2e86c1"
+  # linestyle: "-."
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+bias_shift:
+  label: "PHC-BO" # Bias Shift
+  # color: "tab:cyan"
+  # color: "#6baed6"
+  color: "#5dade2"
+  # linestyle: "-."
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+vector_scaling:
+  label: "PHC-VS" # Vector Scaling
+  # color: "tab:purple"
+  # color: "#3182bd"
+  color: "#1f77b4"
+  # linestyle: "-."
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_0.7:
+  label: "SFT-w-val" # "SFT (70% Train + 30% Val)\nw/ Early Stopping"
+  # label: "SFT (70% Train + 30% Val)\nw/ Early Stopping"
+  # color: "tab:pink"
+  # color: "#c7e9c0"
+  color: "#2ca02c"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0:
+  label: "SFT-retrain" # "SFT (100% Train)\nw/ Early Stopping" 
+  # label: "SFT (100% Train)\nw/ Early Stopping" 
+  # color: "tab:blue"
+  # color: "#74c476"
+  color: "#27ae60"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es:
+  label: "SFT-wo-val" # "SFT (100% Train)\nw/o Early Stopping" 
+  # label: "SFT (100% Train)\nw/o Early Stopping" 
+  # color: "tab:orange"
+  # color: "#238b45"
+  color: "#82e0aa"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_tempscaling:
+  # label: "SFT retrain + PHC TS"
+  label: "SFT-wo-val + PHC-TS"
+  # color: "tab:green"
+  # color: "#fcbba1"
+  color: "#e74c3c"
+  # linestyle: "--"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_dpcal:
+  # label: "SFT retrain + PHC DP"
+  label: "SFT-wo-val + PHC-DP"
+  # color: "tab:red"
+  # color: "#fc9272"
+  color: "#c0392b"
+  # linestyle: "--"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_biasshift:
+  # label: "SFT retrain + PHC BO"
+  label: "SFT-wo-val + PHC-BO"
+  # color: "tab:cyan"
+  # color: "#fb6a4a"
+  color: "#cd6155"
+  # linestyle: "--"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize
+
+lora_1.0_no_es_plus_vectorscaling:
+  # label: "SFT retrain + PHC VS"
+  label: "SFT-wo-val + PHC-VS"
+  # color: "tab:purple"
+  # color: "#de2d26"
+  color: "#d62728"
+  # linestyle: "--"
+  linestyle: "-"
+  marker: null
+  linewidth: *linewidth
+  markersize: *markersize

lists/agnews/size=128/seed=2/0.0-0.3.txt

@@ -0,0 +1,77 @@
+126292
+57811
+93714
+116305
+100814
+75979
+23433
+92178
+78581
+104328
+36096
+119546
+74241
+95253
+97186
+6611
+82012
+69708
+124774
+66031
+93429
+29123
+12413
+66183
+56550
+49448
+65056
+38579
+105572
+60527
+66389
+36393
+31445
+97209
+125715
+62561
+124256
+35056
+104698
+21432
+534
+98043
+5229
+95450
+25662
+78598
+89432
+16274
+77362
+19218
+86550
+69230
+111104
+11620
+45324
+13396
+70264
+20065
+84158
+112605
+113394
+78188
+18768
+57344
+61758
+73494
+76050
+105346
+98939
+88258
+60554
+47534
+99759
+88007
+123627
+50875
+58858

lists/agnews/size=128/seed=2/0.0-0.7.txt

@@ -0,0 +1,180 @@
+126292
+57811
+93714
+116305
+100814
+75979
+23433
+92178
+78581
+104328
+36096
+119546
+74241
+95253
+97186
+6611
+82012
+69708
+124774
+66031
+93429
+29123
+12413
+66183
+56550
+49448
+65056
+38579
+105572
+60527
+66389
+36393
+31445
+97209
+125715
+62561
+124256
+35056
+104698
+21432
+534
+98043
+5229
+95450
+25662
+78598
+89432
+16274
+77362
+19218
+86550
+69230
+111104
+11620
+45324
+13396
+70264
+20065
+84158
+112605
+113394
+78188
+18768
+57344
+61758
+73494
+76050
+105346
+98939
+88258
+60554
+47534
+99759
+88007
+123627
+50875
+58858
+74144
+73217
+65271
+82690
+69870
+39707
+111794
+124929
+15336
+89833
+36223
+45221
+87099
+53697
+84358
+98039
+105659
+89422
+53195
+20743
+62796
+30605
+19416
+31587
+21814
+80637
+34353
+63118
+24185
+103001
+49280
+18280
+48568
+87904
+67757
+27635
+107232
+34434
+14066
+51384
+36018
+99321
+104479
+104449
+120525
+62981
+8399
+42818
+46652
+954
+54819
+44913
+123397
+101045
+47919
+33277
+537
+25894
+33777
+51002
+106747
+108844
+17413
+94089
+67394
+66802
+33625
+54516
+72709
+68450
+5411
+79521
+52134
+57282
+11372
+97818
+27204
+45550
+112086
+89554
+67244
+81174
+67790
+2896
+109946
+99103
+69092
+15352
+113987
+71497
+116107
+32517
+2729
+67085
+114361
+52191
+86559
+86248
+86914
+68478
+109420
+115754
+34632

lists/agnews/size=128/seed=2/0.0-1.0.txt

@@ -0,0 +1,256 @@
+126292
+57811
+93714
+116305
+100814
+75979
+23433
+92178
+78581
+104328
+36096
+119546
+74241
+95253
+97186
+6611
+82012
+69708
+124774
+66031
+93429
+29123
+12413
+66183
+56550
+49448
+65056
+38579
+105572
+60527
+66389
+36393
+31445
+97209
+125715
+62561
+124256
+35056
+104698
+21432
+534
+98043
+5229
+95450
+25662
+78598
+89432
+16274
+77362
+19218
+86550
+69230
+111104
+11620
+45324
+13396
+70264
+20065
+84158
+112605
+113394
+78188
+18768
+57344
+61758
+73494
+76050
+105346
+98939
+88258
+60554
+47534
+99759
+88007
+123627
+50875
+58858
+74144
+73217
+65271
+82690
+69870
+39707
+111794
+124929
+15336
+89833
+36223
+45221
+87099
+53697
+84358
+98039
+105659
+89422
+53195
+20743
+62796
+30605
+19416
+31587
+21814
+80637
+34353
+63118
+24185
+103001
+49280
+18280
+48568
+87904
+67757
+27635
+107232
+34434
+14066
+51384
+36018
+99321
+104479
+104449
+120525
+62981
+8399
+42818
+46652
+954
+54819
+44913
+123397
+101045
+47919
+33277
+537
+25894
+33777
+51002
+106747
+108844
+17413
+94089
+67394
+66802
+33625
+54516
+72709
+68450
+5411
+79521
+52134
+57282
+11372
+97818
+27204
+45550
+112086
+89554
+67244
+81174
+67790
+2896
+109946
+99103
+69092
+15352
+113987
+71497
+116107
+32517
+2729
+67085
+114361
+52191
+86559
+86248
+86914
+68478
+109420
+115754
+34632
+114895
+110404
+35744
+80841
+13081
+100415
+115590
+78007
+90658
+102502
+37510
+18892
+88412
+39145
+3733
+40938
+870
+50646
+79822
+47153
+65233
+9274
+29894
+34700
+7533
+95722
+56024
+53860
+124542
+53889
+104181
+123521
+81041
+77646
+55565
+95174
+107204
+113671
+116631
+76759
+55841
+31969
+106856
+74644
+51511
+32574
+101889
+124775
+78189
+79178
+31472
+98577
+110611
+1997
+64976
+42524
+78707
+33313
+106468
+25676
+52987
+13647
+102071
+19842
+9367
+15994
+37268
+11051
+37579
+112447
+32596
+76957
+123314
+56255
+119248
+91188

lists/agnews/size=128/seed=2/0.7-1.0.txt

@@ -0,0 +1,76 @@
+114895
+110404
+35744
+80841
+13081
+100415
+115590
+78007
+90658
+102502
+37510
+18892
+88412
+39145
+3733
+40938
+870
+50646
+79822
+47153
+65233
+9274
+29894
+34700
+7533
+95722
+56024
+53860
+124542
+53889
+104181
+123521
+81041
+77646
+55565
+95174
+107204
+113671
+116631
+76759
+55841
+31969
+106856
+74644
+51511
+32574
+101889
+124775
+78189
+79178
+31472
+98577
+110611
+1997
+64976
+42524
+78707
+33313
+106468
+25676
+52987
+13647
+102071
+19842
+9367
+15994
+37268
+11051
+37579
+112447
+32596
+76957
+123314
+56255
+119248
+91188

lists/agnews/size=128/seed=3/0.0-0.3.txt

@@ -0,0 +1,77 @@
+17325
+70416
+101644
+103757
+10755
+48974
+106457
+101137
+99225
+102798
+77287
+17980
+102281
+66364
+26010
+39586
+25362
+55992
+46414
+81171
+70330
+53790
+10261
+99336
+18289
+46519
+106137
+46902
+23530
+76678
+94008
+15144
+106806
+119932
+105677
+25505
+43084
+21581
+99728
+69329
+50530
+52862
+940
+74536
+979
+126670
+105705
+32067
+1388
+47767
+33501
+121156
+38679
+114189
+104626
+4627
+126243
+116596
+17918
+91133
+118037
+61575
+44101
+59664
+98476
+102857
+99982
+119478
+33362
+67745
+103623
+46863
+80988
+19085
+40650
+11212
+50583

lists/agnews/size=128/seed=3/0.0-0.7.txt

@@ -0,0 +1,180 @@
+17325
+70416
+101644
+103757
+10755
+48974
+106457
+101137
+99225
+102798
+77287
+17980
+102281
+66364
+26010
+39586
+25362
+55992
+46414
+81171
+70330
+53790
+10261
+99336
+18289
+46519
+106137
+46902
+23530
+76678
+94008
+15144
+106806
+119932
+105677
+25505
+43084
+21581
+99728
+69329
+50530
+52862
+940
+74536
+979
+126670
+105705
+32067
+1388
+47767
+33501
+121156
+38679
+114189
+104626
+4627
+126243
+116596
+17918
+91133
+118037
+61575
+44101
+59664
+98476
+102857
+99982
+119478
+33362
+67745
+103623
+46863
+80988
+19085
+40650
+11212
+50583
+22270
+117543
+23094
+73296
+9885
+72655
+117992
+72302
+9182
+96920
+56313
+68174
+111656
+50968
+118118
+21992
+7138
+10211
+96948
+122982
+23521
+120394
+101612
+73215
+63351
+45966
+118232
+56232
+76911
+63037
+66304
+105573
+107497
+74066
+115094
+51607
+96907
+1517
+52749
+71981
+120683
+61071
+120487
+66272
+115756
+107034
+63696
+102783
+781
+78063
+97748
+25435
+117563
+4972
+13817
+52938
+89809
+64394
+40961
+107969
+40040
+65317
+102002
+66901
+61337
+47298
+82512
+105283
+69735
+63167
+111525
+8368
+115705
+120897
+33626
+38474
+103532
+97364
+85438
+25396
+41903
+49737
+64397
+124476
+32784
+92090
+13993
+48135
+35223
+26096
+891
+72264
+60825
+121851
+58728
+123037
+10368
+97366
+20966
+51610
+124694
+48007
+19622

lists/agnews/size=128/seed=3/0.0-1.0.txt

@@ -0,0 +1,256 @@
+17325
+70416
+101644
+103757
+10755
+48974
+106457
+101137
+99225
+102798
+77287
+17980
+102281
+66364
+26010
+39586
+25362
+55992
+46414
+81171
+70330
+53790
+10261
+99336
+18289
+46519
+106137
+46902
+23530
+76678
+94008
+15144
+106806
+119932
+105677
+25505
+43084
+21581
+99728
+69329
+50530
+52862
+940
+74536
+979
+126670
+105705
+32067
+1388
+47767
+33501
+121156
+38679
+114189
+104626
+4627
+126243
+116596
+17918
+91133
+118037
+61575
+44101
+59664
+98476
+102857
+99982
+119478
+33362
+67745
+103623
+46863
+80988
+19085
+40650
+11212
+50583
+22270
+117543
+23094
+73296
+9885
+72655
+117992
+72302
+9182
+96920
+56313
+68174
+111656
+50968
+118118
+21992
+7138
+10211
+96948
+122982
+23521
+120394
+101612
+73215
+63351
+45966
+118232
+56232
+76911
+63037
+66304
+105573
+107497
+74066
+115094
+51607
+96907
+1517
+52749
+71981
+120683
+61071
+120487
+66272
+115756
+107034
+63696
+102783
+781
+78063
+97748
+25435
+117563
+4972
+13817
+52938
+89809
+64394
+40961
+107969
+40040
+65317
+102002
+66901
+61337
+47298
+82512
+105283
+69735
+63167
+111525
+8368
+115705
+120897
+33626
+38474
+103532
+97364
+85438
+25396
+41903
+49737
+64397
+124476
+32784
+92090
+13993
+48135
+35223
+26096
+891
+72264
+60825
+121851
+58728
+123037
+10368
+97366
+20966
+51610
+124694
+48007
+19622
+42419
+88591
+121286
+104441
+74341
+30726
+46493
+78991
+48403
+108653
+124178
+117181
+80117
+71523
+13998
+100693
+116280
+95601
+108705
+27834
+84878
+107896
+104826
+7554
+61471
+15884
+11551
+85245
+54098
+4227
+105982
+24162
+39011
+4555
+73175
+87633
+33747
+88516
+3335
+27814
+16936
+28689
+60061
+84431
+1043
+38653
+45326
+114838
+205
+61034
+57555
+35457
+6672
+100714
+9946
+87344
+101579
+76129
+32275
+194
+25910
+84206
+18980
+107407
+95979
+52597
+10556
+55281
+14549
+56215
+26038
+96873
+79841
+20378
+74958
+93767

lists/agnews/size=128/seed=3/0.7-1.0.txt

@@ -0,0 +1,76 @@
+42419
+88591
+121286
+104441
+74341
+30726
+46493
+78991
+48403
+108653
+124178
+117181
+80117
+71523
+13998
+100693
+116280
+95601
+108705
+27834
+84878
+107896
+104826
+7554
+61471
+15884
+11551
+85245
+54098
+4227
+105982
+24162
+39011
+4555
+73175
+87633
+33747
+88516
+3335
+27814
+16936
+28689
+60061
+84431
+1043
+38653
+45326
+114838
+205
+61034
+57555
+35457
+6672
+100714
+9946
+87344
+101579
+76129
+32275
+194
+25910
+84206
+18980
+107407
+95979
+52597
+10556
+55281
+14549
+56215
+26038
+96873
+79841
+20378
+74958
+93767

lists/agnews/size=128/seed=4/0.0-0.3.txt

@@ -0,0 +1,77 @@
+56010
+661
+103156
+67353
+63411
+31108
+104842
+7456
+85033
+55459
+58290
+70408
+20715
+51405
+53467
+91088
+69362
+40649
+745
+101406
+40089
+105881
+8948
+92790
+87944
+77310
+105839
+54485
+49140
+78559
+111227
+120575
+68388
+5347
+22283
+73383
+70524
+100429
+114666
+105965
+76828
+69408
+65501
+100830
+115789
+101637
+33985
+103033
+3908
+4079
+74014
+64182
+88643
+26765
+125469
+14221
+31685
+5643
+117240
+22405
+91760
+91261
+127119
+105487
+120138
+111685
+104330
+25394
+61226
+67963
+89729
+62040
+51177
+35001
+58856
+20888
+61237

lists/agnews/size=128/seed=4/0.0-0.7.txt

@@ -0,0 +1,180 @@
+56010
+661
+103156
+67353
+63411
+31108
+104842
+7456
+85033
+55459
+58290
+70408
+20715
+51405
+53467
+91088
+69362
+40649
+745
+101406
+40089
+105881
+8948
+92790
+87944
+77310
+105839
+54485
+49140
+78559
+111227
+120575
+68388
+5347
+22283
+73383
+70524
+100429
+114666
+105965
+76828
+69408
+65501
+100830
+115789
+101637
+33985
+103033
+3908
+4079
+74014
+64182
+88643
+26765
+125469
+14221
+31685
+5643
+117240
+22405
+91760
+91261
+127119
+105487
+120138
+111685
+104330
+25394
+61226
+67963
+89729
+62040
+51177
+35001
+58856
+20888
+61237
+67485
+99337
+54136
+85887
+97550
+64579
+35078
+30380
+31796
+36085
+1968
+42655
+61579
+116837
+36025
+95692
+11915
+821
+79161
+69354
+5929
+105747
+9593
+19924
+123914
+645
+23602
+72383
+12522
+35966
+70208
+103839
+101839
+98203
+111136
+6263
+73971
+28953
+63727
+29102
+75537
+78281
+62492
+20273
+124663
+123858
+108351
+28603
+103462
+2741
+32666
+35783
+87017
+19889
+55249
+27810
+118210
+76541
+7021
+51822
+88891
+111148
+42200
+21130
+123503
+3771
+16915
+67085
+15967
+49367
+74294
+41367
+39510
+87055
+113421
+125316
+6692
+40144
+121924
+29771
+92753
+122234
+101809
+118945
+70098
+29617
+90463
+46611
+14424
+20881
+99757
+87607
+81465
+72213
+97797
+79802
+46962
+86215
+97284
+97088
+120725
+55058
+6468

lists/agnews/size=128/seed=4/0.0-1.0.txt

@@ -0,0 +1,256 @@
+56010
+661
+103156
+67353
+63411
+31108
+104842
+7456
+85033
+55459
+58290
+70408
+20715
+51405
+53467
+91088
+69362
+40649
+745
+101406
+40089
+105881
+8948
+92790
+87944
+77310
+105839
+54485
+49140
+78559
+111227
+120575
+68388
+5347
+22283
+73383
+70524
+100429
+114666
+105965
+76828
+69408
+65501
+100830
+115789
+101637
+33985
+103033
+3908
+4079
+74014
+64182
+88643
+26765
+125469
+14221
+31685
+5643
+117240
+22405
+91760
+91261
+127119
+105487
+120138
+111685
+104330
+25394
+61226
+67963
+89729
+62040
+51177
+35001
+58856
+20888
+61237
+67485
+99337
+54136
+85887
+97550
+64579
+35078
+30380
+31796
+36085
+1968
+42655
+61579
+116837
+36025
+95692
+11915
+821
+79161
+69354
+5929
+105747
+9593
+19924
+123914
+645
+23602
+72383
+12522
+35966
+70208
+103839
+101839
+98203
+111136
+6263
+73971
+28953
+63727
+29102
+75537
+78281
+62492
+20273
+124663
+123858
+108351
+28603
+103462
+2741
+32666
+35783
+87017
+19889
+55249
+27810
+118210
+76541
+7021
+51822
+88891
+111148
+42200
+21130
+123503
+3771
+16915
+67085
+15967
+49367
+74294
+41367
+39510
+87055
+113421
+125316
+6692
+40144
+121924
+29771
+92753
+122234
+101809
+118945
+70098
+29617
+90463
+46611
+14424
+20881
+99757
+87607
+81465
+72213
+97797
+79802
+46962
+86215
+97284
+97088
+120725
+55058
+6468
+83499
+43074
+121850
+19593
+86934
+37918
+48514
+4073
+16491
+53242
+22366
+125071
+26034
+29935
+74711
+10215
+106041
+12983
+20487
+127550
+59780
+88941
+35900
+59193
+95727
+58233
+88062
+98263
+115333
+72896
+60558
+30209
+122890
+70724
+66927
+88804
+4557
+45258
+98542
+123534
+53113
+47123
+93184
+37367
+112132
+118271
+89217
+50934
+20305
+51638
+18010
+93085
+35994
+92395
+101051
+119049
+127242
+121311
+48410
+100233
+680
+52333
+94444
+62839
+53320
+63161
+39348
+90430
+71115
+66663
+10630
+1375
+52315
+23723
+30329
+34764

lists/agnews/size=128/seed=4/0.7-1.0.txt

@@ -0,0 +1,76 @@
+83499
+43074
+121850
+19593
+86934
+37918
+48514
+4073
+16491
+53242
+22366
+125071
+26034
+29935
+74711
+10215
+106041
+12983
+20487
+127550
+59780
+88941
+35900
+59193
+95727
+58233
+88062
+98263
+115333
+72896
+60558
+30209
+122890
+70724
+66927
+88804
+4557
+45258
+98542
+123534
+53113
+47123
+93184
+37367
+112132
+118271
+89217
+50934
+20305
+51638
+18010
+93085
+35994
+92395
+101051
+119049
+127242
+121311
+48410
+100233
+680
+52333
+94444
+62839
+53320
+63161
+39348
+90430
+71115
+66663
+10630
+1375
+52315
+23723
+30329
+34764

lists/agnews/size=128/seed=6/0.0-0.3.txt

@@ -0,0 +1,77 @@
+46284
+62317
+106055
+24713
+34127
+89606
+3759
+6210
+88672
+83034
+35304
+63831
+36285
+126915
+28118
+96827
+80738
+97233
+35552
+110930
+88923
+124866
+21484
+81226
+102427
+49371
+42175
+94666
+100199
+20434
+6751
+96915
+23117
+71763
+85356
+35880
+39303
+106443
+83444
+114400
+114496
+116753
+41393
+52008
+40795
+10259
+71027
+117004
+58598
+53591
+12662
+108255
+105672
+96954
+41942
+117460
+79677
+3213
+92063
+89798
+23374
+10482
+54255
+78534
+89612
+125969
+86985
+120477
+82249
+87949
+66957
+88701
+70302
+73552
+116621
+97293
+6443

lists/agnews/size=128/seed=6/0.0-0.7.txt

@@ -0,0 +1,180 @@
+46284
+62317
+106055
+24713
+34127
+89606
+3759
+6210
+88672
+83034
+35304
+63831
+36285
+126915
+28118
+96827
+80738
+97233
+35552
+110930
+88923
+124866
+21484
+81226
+102427
+49371
+42175
+94666
+100199
+20434
+6751
+96915
+23117
+71763
+85356
+35880
+39303
+106443
+83444
+114400
+114496
+116753
+41393
+52008
+40795
+10259
+71027
+117004
+58598
+53591
+12662
+108255
+105672
+96954
+41942
+117460
+79677
+3213
+92063
+89798
+23374
+10482
+54255
+78534
+89612
+125969
+86985
+120477
+82249
+87949
+66957
+88701
+70302
+73552
+116621
+97293
+6443
+56621
+40131
+123790
+51959
+32121
+81078
+103506
+115880
+103743
+72342
+3153
+88900
+103181
+33808
+91533
+93418
+28478
+85641
+59852
+81235
+122069
+35103
+39096
+20262
+27921
+67846
+20447
+82132
+86467
+98576
+90871
+73221
+39920
+31592
+127106
+25055
+2995
+16974
+62119
+9482
+59337
+81126
+12107
+96998
+110792
+65911
+47250
+4436
+49784
+44000
+102053
+40880
+21380
+13543
+17956
+110438
+94927
+24793
+125499
+38229
+76185
+106608
+4190
+53148
+2353
+118364
+27846
+75134
+73730
+20534
+33733
+9117
+75461
+113013
+57812
+70402
+15965
+39899
+49469
+13707
+3443
+39201
+89012
+77329
+60155
+17964
+14866
+51616
+38844
+93997
+6571
+89746
+84195
+59847
+90916
+104709
+122945
+19151
+26696
+95652
+22458
+54751
+37503

lists/agnews/size=128/seed=6/0.0-1.0.txt

@@ -0,0 +1,256 @@
+46284
+62317
+106055
+24713
+34127
+89606
+3759
+6210
+88672
+83034
+35304
+63831
+36285
+126915
+28118
+96827
+80738
+97233
+35552
+110930
+88923
+124866
+21484
+81226
+102427
+49371
+42175
+94666
+100199
+20434
+6751
+96915
+23117
+71763
+85356
+35880
+39303
+106443
+83444
+114400
+114496
+116753
+41393
+52008
+40795
+10259
+71027
+117004
+58598
+53591
+12662
+108255
+105672
+96954
+41942
+117460
+79677
+3213
+92063
+89798
+23374
+10482
+54255
+78534
+89612
+125969
+86985
+120477
+82249
+87949
+66957
+88701
+70302
+73552
+116621
+97293
+6443
+56621
+40131
+123790
+51959
+32121
+81078
+103506
+115880
+103743
+72342
+3153
+88900
+103181
+33808
+91533
+93418
+28478
+85641
+59852
+81235
+122069
+35103
+39096
+20262
+27921
+67846
+20447
+82132
+86467
+98576
+90871
+73221
+39920
+31592
+127106
+25055
+2995
+16974
+62119
+9482
+59337
+81126
+12107
+96998
+110792
+65911
+47250
+4436
+49784
+44000
+102053
+40880
+21380
+13543
+17956
+110438
+94927
+24793
+125499
+38229
+76185
+106608
+4190
+53148
+2353
+118364
+27846
+75134
+73730
+20534
+33733
+9117
+75461
+113013
+57812
+70402
+15965
+39899
+49469
+13707
+3443
+39201
+89012
+77329
+60155
+17964
+14866
+51616
+38844
+93997
+6571
+89746
+84195
+59847
+90916
+104709
+122945
+19151
+26696
+95652
+22458
+54751
+37503
+114182
+61901
+38881
+15209
+16292
+19738
+92407
+62266
+113820
+22995
+70986
+96105
+41518
+62764
+30808
+16922
+111722
+53481
+108535
+98321
+4222
+37512
+56778
+97632
+221
+15962
+38584
+61581
+9325
+56978
+89751
+34277
+122338
+72073
+663
+123651
+107641
+5468
+125220
+72698
+71408
+46471
+108811
+77513
+2341
+70640
+80361
+10014
+91518
+78408
+106071
+114521
+67690
+89591
+21958
+28388
+119863
+98034
+53255
+125797
+100163
+14496
+99614
+50193
+73057
+115834
+109825
+71231
+81368
+33307
+111033
+120862
+92759
+65473
+39450
+100242

lists/agnews/size=128/seed=6/0.7-1.0.txt

@@ -0,0 +1,76 @@
+114182
+61901
+38881
+15209
+16292
+19738
+92407
+62266
+113820
+22995
+70986
+96105
+41518
+62764
+30808
+16922
+111722
+53481
+108535
+98321
+4222
+37512
+56778
+97632
+221
+15962
+38584
+61581
+9325
+56978
+89751
+34277
+122338
+72073
+663
+123651
+107641
+5468
+125220
+72698
+71408
+46471
+108811
+77513
+2341
+70640
+80361
+10014
+91518
+78408
+106071
+114521
+67690
+89591
+21958
+28388
+119863
+98034
+53255
+125797
+100163
+14496
+99614
+50193
+73057
+115834
+109825
+71231
+81368
+33307
+111033
+120862
+92759
+65473
+39450
+100242

lists/agnews/size=128/seed=8/0.0-0.7.txt

@@ -0,0 +1,180 @@
+62744
+113690
+96524
+115982
+97017
+64627
+71957
+92563
+46779
+77187
+41197
+5736
+98241
+28670
+20195
+95666
+31656
+49504
+9879
+70423
+31546
+89919
+22750
+61483
+49085
+118748
+68402
+23585
+10740
+18656
+15867
+104194
+18705
+61431
+48574
+114072
+124159
+2076
+85129
+40014
+1811
+43430
+78088
+75980
+118948
+121411
+65573
+99765
+25638
+77627
+18047
+38672
+46041
+55661
+101475
+96827
+26556
+115706
+114141
+52960
+81868
+105213
+86934
+114952
+60595
+78684
+98957
+105642
+102281
+83205
+110364
+95392
+26657
+59075
+40202
+122503
+127324
+20744
+43818
+26607
+29690
+910
+29601
+56172
+60867
+121903
+61011
+50334
+75054
+60028
+48305
+59198
+38381
+25674
+48922
+89047
+2011
+116091
+57142
+17800
+126890
+50476
+120265
+37298
+4154
+34419
+33457
+27062
+36737
+44661
+6429
+71563
+61732
+119711
+83842
+123865
+47850
+3998
+44352
+97244
+28808
+103806
+80302
+114184
+76641
+51299
+28638
+127486
+85124
+53890
+4508
+46737
+7046
+92404
+35067
+80235
+13304
+33348
+17580
+57362
+106753
+74051
+113831
+125065
+106616
+108469
+65674
+33679
+68191
+78935
+50040
+69126
+64837
+51225
+37635
+7180
+124920
+16849
+124728
+108426
+9962
+81300
+97565
+112226
+55886
+94322
+95818
+47138
+58894
+12974
+102835
+60283
+78680
+47406
+42544
+100041
+72124
+72540
+68245
+19851

lists/agnews/size=128/seed=8/0.0-1.0.txt

@@ -0,0 +1,256 @@
+62744
+113690
+96524
+115982
+97017
+64627
+71957
+92563
+46779
+77187
+41197
+5736
+98241
+28670
+20195
+95666
+31656
+49504
+9879
+70423
+31546
+89919
+22750
+61483
+49085
+118748
+68402
+23585
+10740
+18656
+15867
+104194
+18705
+61431
+48574
+114072
+124159
+2076
+85129
+40014
+1811
+43430
+78088
+75980
+118948
+121411
+65573
+99765
+25638
+77627
+18047
+38672
+46041
+55661
+101475
+96827
+26556
+115706
+114141
+52960
+81868
+105213
+86934
+114952
+60595
+78684
+98957
+105642
+102281
+83205
+110364
+95392
+26657
+59075
+40202
+122503
+127324
+20744
+43818
+26607
+29690
+910
+29601
+56172
+60867
+121903
+61011
+50334
+75054
+60028
+48305
+59198
+38381
+25674
+48922
+89047
+2011
+116091
+57142
+17800
+126890
+50476
+120265
+37298
+4154
+34419
+33457
+27062
+36737
+44661
+6429
+71563
+61732
+119711
+83842
+123865
+47850
+3998
+44352
+97244
+28808
+103806
+80302
+114184
+76641
+51299
+28638
+127486
+85124
+53890
+4508
+46737
+7046
+92404
+35067
+80235
+13304
+33348
+17580
+57362
+106753
+74051
+113831
+125065
+106616
+108469
+65674
+33679
+68191
+78935
+50040
+69126
+64837
+51225
+37635
+7180
+124920
+16849
+124728
+108426
+9962
+81300
+97565
+112226
+55886
+94322
+95818
+47138
+58894
+12974
+102835
+60283
+78680
+47406
+42544
+100041
+72124
+72540
+68245
+19851
+68949
+14846
+81196
+60230
+25424
+50982
+29689
+64334
+49123
+118626
+58023
+93225
+36369
+4362
+90627
+101082
+98759
+101939
+77723
+64795
+4549
+109733
+92562
+35584
+118448
+10126
+69770
+43877
+79794
+46329
+122547
+89780
+26155
+69520
+29750
+12862
+125129
+86756
+57701
+120492
+115233
+25857
+52279
+121424
+104437
+100666
+76845
+64263
+87496
+85220
+42732
+21581
+67462
+16755
+85251
+10046
+88714
+96103
+111916
+107359
+31992
+16638
+115254
+32649
+37340
+68162
+116149
+68837
+107632
+72402
+53631
+2177
+90682
+45106
+78503
+109227

lists/agnews/size=128/seed=8/0.7-1.0.txt

@@ -0,0 +1,76 @@
+68949
+14846
+81196
+60230
+25424
+50982
+29689
+64334
+49123
+118626
+58023
+93225
+36369
+4362
+90627
+101082
+98759
+101939
+77723
+64795
+4549
+109733
+92562
+35584
+118448
+10126
+69770
+43877
+79794
+46329
+122547
+89780
+26155
+69520
+29750
+12862
+125129
+86756
+57701
+120492
+115233
+25857
+52279
+121424
+104437
+100666
+76845
+64263
+87496
+85220
+42732
+21581
+67462
+16755
+85251
+10046
+88714
+96103
+111916
+107359
+31992
+16638
+115254
+32649
+37340
+68162
+116149
+68837
+107632
+72402
+53631
+2177
+90682
+45106
+78503
+109227

lists/agnews/size=128/seed=9/0.0-0.3.txt

@@ -0,0 +1,77 @@
+59930
+116325
+27821
+100717
+78074
+88974
+78483
+70970
+34345
+3875
+79314
+81720
+63950
+85212
+104792
+15008
+93
+72450
+46721
+64654
+37576
+55690
+57051
+8337
+4929
+49686
+80841
+106434
+61847
+121702
+17550
+108405
+99264
+774
+51699
+108073
+14537
+102309
+27610
+84378
+105162
+86250
+112221
+59889
+39843
+72534
+78005
+9855
+122540
+68269
+15139
+67024
+52559
+90406
+104774
+40008
+3990
+13995
+108650
+51285
+47023
+44227
+30830
+2128
+84892
+45232
+50196
+9353
+25051
+113561
+125026
+115010
+124386
+113248
+114363
+64426
+89784

lists/agnews/size=128/seed=9/0.0-0.7.txt

@@ -0,0 +1,180 @@
+59930
+116325
+27821
+100717
+78074
+88974
+78483
+70970
+34345
+3875
+79314
+81720
+63950
+85212
+104792
+15008
+93
+72450
+46721
+64654
+37576
+55690
+57051
+8337
+4929
+49686
+80841
+106434
+61847
+121702
+17550
+108405
+99264
+774
+51699
+108073
+14537
+102309
+27610
+84378
+105162
+86250
+112221
+59889
+39843
+72534
+78005
+9855
+122540
+68269
+15139
+67024
+52559
+90406
+104774
+40008
+3990
+13995
+108650
+51285
+47023
+44227
+30830
+2128
+84892
+45232
+50196
+9353
+25051
+113561
+125026
+115010
+124386
+113248
+114363
+64426
+89784
+63059
+111220
+39343
+3549
+89189
+75615
+57171
+98848
+5792
+64166
+114861
+74468
+16669
+103421
+5997
+110957
+7419
+30127
+64438
+26369
+8027
+31206
+26187
+87576
+65028
+18732
+21244
+25084
+42357
+56511
+14103
+5362
+93131
+102709
+24723
+73070
+104375
+51980
+103601
+537
+55520
+12541
+51644
+53896
+73629
+113311
+21077
+32692
+82482
+73168
+120123
+92405
+91400
+23645
+91948
+23245
+14257
+84672
+5748
+104648
+62822
+114536
+57439
+59050
+398
+4029
+127430
+120778
+101794
+112156
+121307
+20514
+80792
+4624
+96408
+78746
+63622
+106075
+56401
+69632
+25010
+83105
+77507
+76134
+81645
+107890
+48428
+44494
+66014
+93774
+83374
+41968
+54384
+103136
+48170
+106169
+33782
+101358
+86246
+31941
+109756
+4030
+19080

lists/agnews/size=128/seed=9/0.0-1.0.txt

@@ -0,0 +1,256 @@
+59930
+116325
+27821
+100717
+78074
+88974
+78483
+70970
+34345
+3875
+79314
+81720
+63950
+85212
+104792
+15008
+93
+72450
+46721
+64654
+37576
+55690
+57051
+8337
+4929
+49686
+80841
+106434
+61847
+121702
+17550
+108405
+99264
+774
+51699
+108073
+14537
+102309
+27610
+84378
+105162
+86250
+112221
+59889
+39843
+72534
+78005
+9855
+122540
+68269
+15139
+67024
+52559
+90406
+104774
+40008
+3990
+13995
+108650
+51285
+47023
+44227
+30830
+2128
+84892
+45232
+50196
+9353
+25051
+113561
+125026
+115010
+124386
+113248
+114363
+64426
+89784
+63059
+111220
+39343
+3549
+89189
+75615
+57171
+98848
+5792
+64166
+114861
+74468
+16669
+103421
+5997
+110957
+7419
+30127
+64438
+26369
+8027
+31206
+26187
+87576
+65028
+18732
+21244
+25084
+42357
+56511
+14103
+5362
+93131
+102709
+24723
+73070
+104375
+51980
+103601
+537
+55520
+12541
+51644
+53896
+73629
+113311
+21077
+32692
+82482
+73168
+120123
+92405
+91400
+23645
+91948
+23245
+14257
+84672
+5748
+104648
+62822
+114536
+57439
+59050
+398
+4029
+127430
+120778
+101794
+112156
+121307
+20514
+80792
+4624
+96408
+78746
+63622
+106075
+56401
+69632
+25010
+83105
+77507
+76134
+81645
+107890
+48428
+44494
+66014
+93774
+83374
+41968
+54384
+103136
+48170
+106169
+33782
+101358
+86246
+31941
+109756
+4030
+19080
+34050
+46465
+72763
+29058
+113219
+11412
+43955
+10609
+105032
+8466
+100662
+85873
+28144
+103156
+126669
+12602
+119839
+22556
+108284
+6465
+32616
+19528
+59337
+36023
+69830
+92556
+32141
+12031
+46071
+108585
+45824
+111483
+109904
+74713
+34809
+125537
+127439
+7773
+58005
+85329
+71465
+28761
+120247
+31642
+59977
+47107
+82563
+34445
+12020
+2365
+29508
+111396
+92649
+98431
+104647
+48793
+105671
+103594
+8178
+27549
+35342
+46316
+98770
+22881
+38254
+69837
+17020
+63863
+79243
+54750
+41688
+88004
+116552
+126607
+21332
+69320

lists/agnews/size=128/seed=9/0.7-1.0.txt

@@ -0,0 +1,76 @@
+34050
+46465
+72763
+29058
+113219
+11412
+43955
+10609
+105032
+8466
+100662
+85873
+28144
+103156
+126669
+12602
+119839
+22556
+108284
+6465
+32616
+19528
+59337
+36023
+69830
+92556
+32141
+12031
+46071
+108585
+45824
+111483
+109904
+74713
+34809
+125537
+127439
+7773
+58005
+85329
+71465
+28761
+120247
+31642
+59977
+47107
+82563
+34445
+12020
+2365
+29508
+111396
+92649
+98431
+104647
+48793
+105671
+103594
+8178
+27549
+35342
+46316
+98770
+22881
+38254
+69837
+17020
+63863
+79243
+54750
+41688
+88004
+116552
+126607
+21332
+69320

lists/agnews/size=8/seed=0/0.0-0.3.txt

@@ -0,0 +1,5 @@
+39155
+8934
+31196
+45484
+36446

lists/agnews/size=8/seed=0/0.0-0.7.txt

@@ -0,0 +1,12 @@
+39155
+8934
+31196
+45484
+36446
+27069
+98154
+38190
+76908
+114510
+20161
+62713

lists/agnews/size=8/seed=0/0.0-1.0.txt

@@ -0,0 +1,16 @@
+39155
+8934
+31196
+45484
+36446
+27069
+98154
+38190
+76908
+114510
+20161
+62713
+32678
+55068
+46296
+67446

lists/agnews/size=8/seed=0/0.7-1.0.txt

@@ -0,0 +1,4 @@
+32678
+55068
+46296
+67446

lists/agnews/size=8/seed=1/0.0-0.3.txt

@@ -0,0 +1,5 @@
+23737
+115866
+74029
+119768
+104853

lists/agnews/size=8/seed=1/0.0-0.7.txt

@@ -0,0 +1,12 @@
+23737
+115866
+74029
+119768
+104853
+12606
+22683
+59347
+48713
+107261
+56098
+5056

lists/agnews/size=8/seed=1/0.0-1.0.txt

@@ -0,0 +1,16 @@
+23737
+115866
+74029
+119768
+104853
+12606
+22683
+59347
+48713
+107261
+56098
+5056
+47061
+72767
+18365
+119473

lists/agnews/size=8/seed=1/0.7-1.0.txt

@@ -0,0 +1,4 @@
+47061
+72767
+18365
+119473

lists/agnews/size=8/seed=2/0.0-0.3.txt

@@ -0,0 +1,5 @@
+126292
+57811
+93714
+116305
+100814

lists/agnews/size=8/seed=2/0.0-0.7.txt

@@ -0,0 +1,12 @@
+126292
+57811
+93714
+116305
+100814
+75979
+23433
+92178
+78581
+104328
+36096
+119546

lists/agnews/size=8/seed=2/0.0-1.0.txt

@@ -0,0 +1,16 @@
+126292
+57811
+93714
+116305
+100814
+75979
+23433
+92178
+78581
+104328
+36096
+119546
+74241
+95253
+97186
+6611

lists/agnews/size=8/seed=2/0.7-1.0.txt

@@ -0,0 +1,4 @@
+74241
+95253
+97186
+6611

lists/agnews/size=8/seed=3/0.0-0.3.txt

@@ -0,0 +1,5 @@
+17325
+70416
+101644
+103757
+10755

lists/agnews/size=8/seed=3/0.0-0.7.txt

@@ -0,0 +1,12 @@
+17325
+70416
+101644
+103757
+10755
+48974
+106457
+101137
+99225
+102798
+77287
+17980

lists/agnews/size=8/seed=3/0.0-1.0.txt

@@ -0,0 +1,16 @@
+17325
+70416
+101644
+103757
+10755
+48974
+106457
+101137
+99225
+102798
+77287
+17980
+102281
+66364
+26010
+39586

lists/agnews/size=8/seed=3/0.7-1.0.txt

@@ -0,0 +1,4 @@
+102281
+66364
+26010
+39586

lists/agnews/size=8/seed=4/0.0-0.3.txt

@@ -0,0 +1,5 @@
+56010
+661
+103156
+67353
+63411

lists/agnews/size=8/seed=5/0.0-0.3.txt

@@ -0,0 +1,5 @@
+33883
+25778
+85655
+71399
+77521

lists/agnews/size=8/seed=5/0.0-0.7.txt

@@ -0,0 +1,12 @@
+33883
+25778
+85655
+71399
+77521
+24208
+98868
+2144
+111351
+11711
+41649
+118957

lists/agnews/size=8/seed=5/0.0-1.0.txt

@@ -0,0 +1,16 @@
+33883
+25778
+85655
+71399
+77521
+24208
+98868
+2144
+111351
+11711
+41649
+118957
+100590
+112134
+102107
+48457

lists/agnews/size=8/seed=5/0.7-1.0.txt

@@ -0,0 +1,4 @@
+100590
+112134
+102107
+48457

lists/agnews/size=8/seed=6/0.0-0.3.txt

@@ -0,0 +1,5 @@
+46284
+62317
+106055
+24713
+34127