File size: 2,326 Bytes
64ab846
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
# Predictive Irrigation Models

This repository contains data preprocessing and analytics pipelines for the distribution of Predictive Irrigation Models, integrating data from field sensors, weather, soil, crop, and remote sensing sources.

## Project Structure
```

predictive_irrigation_models/

β”œβ”€β”€ config/

β”‚ β”œβ”€β”€ params.yml

β”‚ β”œβ”€β”€ xgcast_params.yml

β”‚ β”œβ”€β”€ aquacrop_params.yml

β”‚ └── fieldsensor_irrigator_mapping_anonym.yaml.yml

β”œβ”€β”€ pipelines/

β”‚ β”œβ”€β”€ __init__.py

β”‚ β”œβ”€β”€ aquacrop_preparation_pipeline.py

β”‚ β”œβ”€β”€ data_collection_pipeline.py

β”‚ β”œβ”€β”€ aquacrop_preparation_pipeline.py

β”‚ β”œβ”€β”€ demo_run.py

β”‚ β”œβ”€β”€ model_preparation_pipeline.py

β”‚ β”œβ”€β”€ preprocessing_pipeline.py

β”‚ β”œβ”€β”€ resample_impute_pipeline.py

β”‚ β”œβ”€β”€ soilcast_pipeline.py

β”‚ β”œβ”€β”€ xgcast_pipeline.py

β”‚ └── xgcast_run.py

β”œβ”€β”€ data/

β”‚ β”œβ”€β”€ 03_primary/

β”‚ β”œβ”€β”€ 04_model_input/

β”‚ β”œβ”€β”€ 05_aquacrop_input/

β”‚ β”œβ”€β”€ 05_xgcast_input/

β”‚ β”œβ”€β”€ 06_aquacrop_output/

β”‚ └── 06_xgcast_output/

β”œβ”€β”€ tools/

β”‚ └── ...

└── README.md

```

## Features

- **Prefect-based Pipelines:** Modular tasks and flows for data collection, transformation, and saving.
- **Sensor & Weather Data Integration:** Reads and merges raw sensor and weather data for multiple consortia.
- **Soil, Crop, and Remote Sensing Data:** Integrates geospatial and tabular data sources.
- **Automated Testing:** Prefect tasks and flows for validating preprocessing results.
- **Artifact Logging:** Data summary artifacts for monitoring pipeline outputs.

## Getting Started

### Prerequisites

- Python 3.8+
- [Prefect](https://www.prefect.io/)
- uv

Install dependencies:
```uv sync```

Configuration
Edit `config/params.yml` to specify consortia names, data folders, and other parameters.

### Running the Pipeline

From the project root, run:

```python -m main```

### Running Tests
From the project root, run:

```python -m test.test_results```

### Contributing
Feel free to open issues or submit pull requests for improvements or bug fixes.

### License
MIT License

### Contact
For questions or collaboration, please contact the repository owner. ``````