DEPENDENCY_NOTES.md

# Dependency Management Notes

This document outlines known dependency constraints and issues in the Schema Descriptor project. It's intended for maintainers and contributors to understand the dependency landscape.

## Critical Dependencies

### Protobuf (3.20.3)

This project requires protobuf version 3.20.3 due to several conflicting constraints:

- Streamlit 1.12.0 requires protobuf < 4.0.0
- Google Cloud BigQuery libraries require protobuf >= 3.19.5
- gRPC-status requires protobuf >= 4.21.6 (which we've worked around)

Using any other version of protobuf will cause dependency conflicts:
- Lower versions won't work with Google Cloud libraries
- Higher versions (4.x) won't work with Streamlit

### Altair (4.2.2)

Altair must be exactly version 4.2.2 because:

- Streamlit 1.12.0 depends on the `altair.vegalite.v4` module
- Newer versions of Altair (>= 5.0.0) removed this module structure
- This causes `ModuleNotFoundError: No module named 'altair.vegalite.v4'` errors

### Streamlit (1.12.0)

The application is built with Streamlit 1.12.0. Upgrading to newer versions would:
- Require extensive refactoring
- Potentially solve some dependency issues
- But introduce backwards compatibility concerns

### OpenAI (0.28.0)

OpenAI version 0.28.0 is used because:
- It has a compatible API with our LLM service integration
- Newer client libraries (>=1.0.0) have completely different APIs

## Dependency Resolution Strategy

The following installation order helps resolve dependency conflicts:

1. Install protobuf first: `pip install protobuf==3.20.3`
2. Install Altair: `pip install altair==4.2.2`  
3. Install Streamlit: `pip install streamlit==1.12.0`
4. Install OpenAI: `pip install openai==0.28.0`
5. Install remaining dependencies: `pip install -r requirements.txt --no-deps`

## CI/CD Pipeline

In the GitHub Actions workflow:

1. We explicitly install test dependencies first
2. Then install key dependencies in the correct order
3. Use `--no-deps` to avoid dependency resolution issues
4. Skip tests that are known to fail due to dependency mocking issues

## Future Improvements

Potential improvements to dependency management:

1. **Upgrade Streamlit**: Moving to a newer version would resolve several issues but require code changes
2. **Use Docker**: Containerization would provide a more consistent environment
3. **Split Requirements**: Create separate requirement files for main, dev, and test dependencies
4. **Fix Test Mocks**: Update tests to work better with the current dependency constraints

## Known Dependency-Related Issues

1. **Test failures**: Some tests fail when mocking dependencies due to strict type checking
2. **Install errors**: Non-ordered installation may lead to dependency resolution failures
3. **Version control**: Pin exact versions vs compatible versions (`==` vs `~=`) trade-off