Upload 1.3 with bioimageio.spec 0.5.7.1

README.md

@@ -10,7 +10,6 @@ HyLFM-Net trained on static images of arrested medaka hatchling hearts. The netw
 - [Bias, Risks, and Limitations](#bias-risks-and-limitations)
 - [How to Get Started with the Model](#how-to-get-started-with-the-model)
 - [Training Details](#training-details)
-- [Evaluation](#evaluation)
 - [Environmental Impact](#environmental-impact)
 - [Technical Specifications](#technical-specifications)
 
@@ -19,7 +18,7 @@ HyLFM-Net trained on static images of arrested medaka hatchling hearts. The netw
 
 ## Model Description
 
-- **model version:** 1.2
+- **model version:** 1.3
 - **Additional model documentation:** [package/README.md](package/README.md)
 - **Developed by:** 
     - Beuttenmueller, Wagner, N., F., Norlin, N. et al. Deep learning-enhanced light-field imaging with continuous validation. Nat Methods 18, 557–563 (2021).: https://www.doi.org/10.1038/s41592-021-01136-0
@@ -42,7 +41,15 @@ HyLFM-Net trained on static images of arrested medaka hatchling hearts. The netw
 
 This model is compatible with the bioimageio.spec Python package (version >= 0.5.7.1) and the bioimageio.core Python package supporting model inference in Python code or via the `bioimageio` CLI.
 
+```python
+from bioimageio.core import predict
 
+output_sample = predict("huggingface/thefynnbe/ambitious-sloth/1.3", inputs={'lf': '<path or tensor>'})
+
+output_tensor = output_sample.members["prediction"]
+xarray_dataarray = output_tensor.data
+numpy_ndarray = output_tensor.data.to_numpy()
+```
 
 ## Downstream Use
 
@@ -88,7 +95,7 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 # How to Get Started with the Model
 
-You can use "huggingface/thefynnbe/ambitious-sloth/v1.2" as the resource identifier to load this model directly from the Hugging Face Hub using bioimageio.spec or bioimageio.core.
+You can use "huggingface/thefynnbe/ambitious-sloth/1.3" as the resource identifier to load this model directly from the Hugging Face Hub using bioimageio.spec or bioimageio.core.
 
 See [bioimageio.core documentation: Get started](https://bioimage-io.github.io/core-bioimage-io-python/latest/get-started) for instructions on how to load and run this model using the `bioimageio.core` Python package or the bioimageio CLI.
 
@@ -109,20 +116,13 @@ This model was trained on `10.5281/zenodo.7612115`.
 - **Model size:** 234.44 MB
 
 
-# Evaluation
-
-missing
-### Validation on External Data
-
-missing
-
 # Environmental Impact
 
 - **Hardware Type:** GTX 2080 Ti
 - **Hours used:** 10.0
 - **Cloud Provider:** EMBL Heidelberg
 - **Compute Region:** Germany
-- **Carbon Emitted:** 0.54
+- **Carbon Emitted:** 0.54 kg CO2e
 
 
 
@@ -138,7 +138,8 @@ missing
   - Axes: `batch, channel, y, x`
   - Shape: `1 × 1 × 1235 × 1425`
   - Data type: `float32`
-    - Values: 1.0 arbitrary unit with offset: None in range (None, None)
+  - Value unit: arbitrary unit
+  - Value scale factor: 1.0
   - example
     ![lf sample](images/input_lf_sample.png)
 
@@ -147,7 +148,8 @@ missing
   - Axes: `batch, channel, z, y, x`
   - Shape: `1 × 1 × 49 × 244 × 284`
   - Data type: `float32`
-    - Values: 1.0 arbitrary unit with offset: None in range (None, None)
+  - Value unit: arbitrary unit
+  - Value scale factor: 1.0
   - example
     prediction sample](images/output_prediction_sample.png)
 
@@ -162,7 +164,7 @@ missing
 ### Software
 
 - **Framework:** ONNX: opset version: 15 or Pytorch State Dict: 1.13 or TorchScript: 1.13
-- **Libraries:** Dependencies for Pytorch State dict weights are listed in [environment.yaml](package/environment.yaml).
+- **Libraries:** None beyond the respective framework library.
 - **BioImage.IO partner compatibility:** [Compatibility Reports](https://bioimage-io.github.io/collection/latest/compatibility/#compatibility-by-resource)
 
 ---

package/README.md

@@ -1,431 +1,9 @@
-![License](https://img.shields.io/github/license/bioimage-io/spec-bioimage-io.svg)
-![PyPI](https://img.shields.io/pypi/v/bioimageio-spec.svg?style=popout)
-![conda-version](https://anaconda.org/conda-forge/bioimageio.spec/badges/version.svg)
+# HyLFM-Net Example
 
-# Specifications for bioimage.io
+Reference example for a HyLFM-Net developed at [kreshuklab/hylfm-net](https://github.com/kreshuklab/hylfm-net).
+This network is not expected to generalize to other microscopy light field datasets.
+See [Deep learning-enhanced light-field imaging withcontinuous validation](https://rdcu.be/cktHs) for details.
 
-This repository contains specifications defined by the bioimage.io community. These specifications are used for defining fields in YAML 1.2 files which should be named `rdf.yaml`. Such a rdf.yaml --- along with files referenced in it --- can be downloaded from or uploaded to the [bioimage.io website](https://bioimage.io) and may be produced or consumed by bioimage.io-compatible consumers (e.g. image analysis software like ilastik).
+## Validation
 
-bioimage.io-compatible resources must fulfill the following rules:
-
-Note that the Python package PyYAML does not support YAML 1.2 .
-We therefore use and recommend [ruyaml](https://ruyaml.readthedocs.io/en/latest/).
-For differences see <https://ruamelyaml.readthedocs.io/en/latest/pyyaml>.
-
-Please also note that the best way to check whether your `rdf.yaml` file is bioimage.io-compliant is to call `bioimageio.core.validate` from the [bioimageio.core](https://github.com/bioimage-io/core-bioimage-io-python) Python package.
-The [bioimageio.core](https://github.com/bioimage-io/core-bioimage-io-python) Python package also provides the bioimageio command line interface (CLI) with the `validate` command:
-
-```terminal
-bioimageio validate path/to/your/rdf.yaml
-```
-
-## Format version overview
-
-All bioimage.io description formats are defined as [Pydantic models](https://docs.pydantic.dev/latest/).
-
-| type | format version | documentation |
-| --- | --- | --- |
-| model | 0.5 </br> 0.4 | [model_descr_v0-5.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/model_descr_v0-5.md) </br> [model_descr_v0-4.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/model_descr_v0-4.md) |
-| dataset | 0.3 </br> 0.2 | [dataset_descr_v0-3.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/dataset_descr_v0-3.md) </br> [dataset_descr_v0-2.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/dataset_descr_v0-2.md) |
-| notebook | 0.3 </br> 0.2 | [notebook_descr_v0-3.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/notebook_descr_v0-3.md) </br> [notebook_descr_v0-2.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/notebook_descr_v0-2.md) |
-| application | 0.3 </br> 0.2 | [application_descr_v0-3.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/application_descr_v0-3.md) </br> [application_descr_v0-2.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/application_descr_v0-2.md) |
-| collection | 0.3 </br> 0.2 | [collection_descr_v0-3.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/collection_descr_v0-3.md) </br> [collection_descr_v0-2.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/collection_descr_v0-2.md) |
-| generic | 0.3 </br> 0.2 | [generic_descr_v0-3.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/generic_descr_v0-3.md) </br> [generic_descr_v0-2.md](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/user_docs/generic_descr_v0-2.md) |
-
-## JSON schema
-
-Simplified descriptions are available as [JSON schema](https://json-schema.org/):
-
-| bioimageio.spec version | JSON schema |
-| --- | --- |
-| latest | [bioimageio_schema_latest.json](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/bioimageio_schema_latest.json) |
-| 0.5 | [bioimageio_schema_v0-5.json](https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages/bioimageio_schema_v0-5.json) |
-
-These are primarily intended for syntax highlighting and form generation.
-
-## Examples
-
-We provide some [examples for using rdf.yaml files to describe models, applications, notebooks and datasets](https://github.com/bioimage-io/spec-bioimage-io/blob/main/example_descriptions/examples.md).
-
-## 💁 Recommendations
-
-* Due to the limitations of storage services such as Zenodo, which does not support subfolders, it is recommended to place other files in the same directory level of the `rdf.yaml` file and try to avoid using subdirectories.
-* Use the [bioimageio.core Python package](https://github.com/bioimage-io/core-bioimage-io-python) to validate your `rdf.yaml` file.
-* bioimageio.spec keeps evolving. Try to use and upgrade to the most current format version!
-
-## ⌨ bioimageio command-line interface (CLI)
-
-The bioimageio CLI has moved entirely to [bioimageio.core](https://github.com/bioimage-io/core-bioimage-io-python).
-
-## 🖥 Installation
-
-bioimageio.spec can be installed with either `conda` or `pip`, we recommend to install `bioimageio.core` instead:
-
-```console
-conda install -c conda-forge bioimageio.core
-```
-
-or
-
-```console
-pip install -U bioimageio.core
-```
-
-## 🏞 Environment variables
-
-TODO: link to settings in dev docs
-
-## 🤝 How to contribute
-
-## ♥ Contributors
-
-<a href="https://github.com/bioimage-io/spec-bioimage-io/graphs/contributors">
-  <img alt="bioimageio.spec contributors" src="https://contrib.rocks/image?repo=bioimage-io/spec-bioimage-io" />
-</a>
-
-Made with [contrib.rocks](https://contrib.rocks).
-
-## Δ Changelog
-
-### bioimageio.spec Python package
-
-#### bioimageio.spec 0.5.2post1
-
-* fix model packaging with weights format priority
-
-#### bioimageio.spec 0.5.2
-
-* new patch version model 0.5.2
-
-#### bioimageio.spec 0.5.1
-
-* new patch version model 0.5.1
-
-#### bioimageio.spec 0.5.0post2
-
-* don't fail if CI env var is a string
-
-#### bioimageio.spec 0.5.0post1
-
-* fix `_internal.io_utils.identify_bioimageio_yaml_file()`
-
-#### bioimageio.spec 0.5.0
-
-* new description formats: [generic 0.3, application 0.3, collection 0.3, dataset 0.3, notebook 0.3](generic-030--application-030--collection-030--dataset-030--notebook-030) and [model 0.5](model-050).
-* various API changes, most important functions:
-  * `bioimageio.spec.load_description` (replaces `load_raw_resource_description`, interface changed)
-  * `bioimageio.spec.validate_format` (new)
-  * `bioimageio.spec.dump_description` (replaces `serialize_raw_resource_description_to_dict`, interface changed)
-  * `bioimageio.spec.update_format` (interface changed)
-* switch from Marshmallow to Pydantic
-  * extended validation
-  * one joint, more precise JSON schema
-
-#### bioimageio.spec 0.4.9
-
-* small bugixes
-* better type hints
-* improved tests
-
-#### bioimageio.spec 0.4.8post1
-
-* add `axes` and `eps` to `scale_mean_var`
-
-#### bioimageio.spec 0.4.7post1
-
-* add simple forward compatibility by treating future format versions as latest known (for the respective resource type)
-
-#### bioimageio.spec 0.4.6post3
-
-* Make CLI output more readable
-
-* find redirected URLs when checking for URL availability
-
-#### bioimageio.spec 0.4.6post2
-
-* Improve error message for non-existing RDF file path given as string
-
-* Improve documentation for model description's `documentation` field
-
-#### bioimageio.spec 0.4.6post1
-
-* fix enrich_partial_rdf_with_imjoy_plugin (see <https://github.com/bioimage-io/spec-bioimage-io/pull/452>)
-
-#### bioimageio.spec 0.4.5post16
-
-* fix rdf_update of entries in `resolve_collection_entries()`
-
-#### bioimageio.spec 0.4.5post15
-
-* pass root to `enrich_partial_rdf` arg of `resolve_collection_entries()`
-
-#### bioimageio.spec 0.4.5post14
-
-* keep `ResourceDescrption.root_path` as URI for remote resources. This fixes the collection description as the collection entries are resolved after the collection description has been loaded.
-
-#### bioimageio.spec 0.4.5post13
-
-* new bioimageio.spec.partner module adding validate-partner-collection command if optional 'lxml' dependency is available
-
-#### bioimageio.spec 0.4.5post12
-
-* new env var `BIOIMAGEIO_CACHE_WARNINGS_LIMIT` (default: 3) to avoid spam from cache hit warnings
-
-* more robust conversion of ImportableSourceFile for absolute paths to relative paths (don't fail on non-path source file)
-
-#### bioimageio.spec 0.4.5post11
-
-* resolve symlinks when transforming absolute to relative paths during serialization; see [#438](https://github.com/bioimage-io/spec-bioimage-io/pull/438)
-
-#### bioimageio.spec 0.4.5post10
-
-* fix loading of collection description with id (id used to be ignored)
-
-#### bioimageio.spec 0.4.5post9
-
-* support loading bioimageio resources by their animal nickname (currently only models have nicknames).
-
-#### bioimageio.spec 0.4.5post8
-
-* any field previously expecting a local relative path is now also accepting an absolute path
-
-* load_raw_resource_description returns a raw resource description which has no relative paths (any relative paths are converted to absolute paths).
-
-#### bioimageio.spec 0.4.4post7
-
-* add command `commands.update_rdf()`/`update-rdf`(cli)
-
-#### bioimageio.spec 0.4.4post2
-
-* fix unresolved ImportableSourceFile
-
-#### bioimageio.spec 0.4.4post1
-
-* fix collection description conversion for type field
-
-#### bioimageio.spec 0.4.3post1
-
-* fix to shape validation for model description 0.4: output shape now needs to be bigger than halo
-
-* moved objects from bioimageio.spec.shared.utils to bioimageio.spec.shared\[.node_transformer\]
-* additional keys to validation summary: bioimageio_spec_version, status
-
-#### bioimageio.spec 0.4.2post4
-
-* fixes to generic description:
-  * ignore value of field `root_path` if present in yaml. This field is used internally and always present in RDF nodes.
-
-#### bioimageio.spec 0.4.1.post5
-
-* fixes to collection description:
-  * RDFs specified directly in collection description are validated correctly even if their source field does not point to an RDF.
-  * nesting of collection description allowed
-
-#### bioimageio.spec 0.4.1.post4
-
-* fixed missing field `icon` in generic description's raw node
-
-* fixes to collection description:
-  * RDFs specified directly in collection description are validated correctly
-  * no nesting of collection description allowed for now
-  * `links` is no longer an explicit collection entry field ("moved" to unknown)
-
-#### bioimageio.spec 0.4.1.post0
-
-* new model spec 0.3.5 and 0.4.1
-
-#### bioimageio.spec 0.4.0.post3
-
-* `load_raw_resource_description` no longer accepts `update_to_current_format` kwarg (use `update_to_format` instead)
-
-#### bioimageio.spec 0.4.0.post2
-
-* `load_raw_resource_description` accepts `update_to_format` kwarg
-
-### Resource Description Format Versions
-
-#### model 0.5.2
-
-* Non-breaking changes
-  * added `concatenable` flag to index, time and space input axes
-
-#### model 0.5.1
-
-* Non-breaking changes
-  * added `DataDependentSize` for `outputs.i.size` to specify an output shape that is not known before inference is run.
-  * added optional `inputs.i.optional` field to indicate that a tensor may be `None`
-  * made data type assumptions in `preprocessing` and `postprocessing` explicit by adding `'ensure_dtype'` operations per default.
-  * allow to specify multiple thresholds (along an `axis`) in a 'binarize' processing step
-
-#### generic 0.3.0 / application 0.3.0 / collection 0.3.0 / dataset 0.3.0 / notebook 0.3.0
-
-* Breaking canges that are fully auto-convertible
-  * dropped `download_url`
-  * dropped non-file attachments
-  * `attachments.files` moved to `attachments.i.source`
-* Non-breaking changes
-  * added optional `parent` field
-
-#### model 0.5.0
-
-all generic 0.3.0 changes (except models already have the `parent` field) plus:
-
-* Breaking changes that are partially auto-convertible
-  * `inputs.i.axes` are now defined in more detail (same for `outputs.i.axes`)
-  * `inputs.i.shape` moved per axes to `inputs.i.axes.size` (same for `outputs.i.shape`)
-  * new pre-/postprocessing 'fixed_zero_mean_unit_variance' separated from 'zero_mean_unit_variance', where `mode=fixed` is no longer valid.
-    (for scalar values this is auto-convertible.)
-* Breaking changes that are fully auto-convertible
-  * changes in `weights.pytorch_state_dict.architecture`
-    * renamed `weights.pytorch_state_dict.architecture.source_file` to `...architecture.source`
-  * changes in `weights.pytorch_state_dict.dependencies`
-    * only conda environment allowed and specified by `weights.pytorch_state_dict.dependencies.source`
-    * new optional field `weights.pytorch_state_dict.dependencies.sha256`
-  * changes in `weights.tensorflow_model_bundle.dependencies`
-    * same as changes in `weights.pytorch_state_dict.dependencies`
-  * moved `test_inputs` to `inputs.i.test_tensor`
-  * moved `test_outputs` to `outputs.i.test_tensor`
-  * moved `sample_inputs` to `inputs.i.sample_tensor`
-  * moved `sample_outputs` to `outputs.i.sample_tensor`
-  * renamed `inputs.i.name` to `inputs.i.id`
-  * renamed `outputs.i.name` to `outputs.i.id`
-  * renamed `inputs.i.preprocessing.name` to `inputs.i.preprocessing.id`
-  * renamed `outputs.i.postprocessing.name` to `outputs.i.postprocessing.id`
-* Non-breaking changes:
-  * new pre-/postprocessing: `id`='ensure_dtype' with kwarg `dtype`
-
-#### generic 0.2.4 and model 0.4.10
-
-* Breaking changes that are fully auto-convertible
-  * `id` overwritten with value from `config.bioimageio.nickname` if available
-* Non-breaking changes
-  * `version_number` is a new, optional field indicating that an RDF is the nth published version with a given `id`
-  * `id_emoji` is a new, optional field (set from `config.bioimageio.nickname_icon` if available)
-  * `uploader` is a new, optional field with `email` and an optional `name` subfields
-
-#### model 0.4.9
-
-* Non-breaking changes
-  * make pre-/postprocessing kwargs `mode` and `axes` always optional for model description 0.3 and 0.4
-
-#### model 0.4.8
-
-* Non-breaking changes
-  * `cite` field is now optional
-
-#### generic 0.2.2 and model 0.4.7
-
-* Breaking changes that are fully auto-convertible
-  * name field may not include '/' or '\' (conversion removes these)
-
-#### model 0.4.6
-
-* Non-breaking changes
-  * Implicit output shape can be expanded by inserting `null` into `shape:scale` and indicating length of new dimension D in the `offset` field. Keep in mind that `D=2*'offset'`.
-
-#### model 0.4.5
-
-* Breaking changes that are fully auto-convertible
-  * `parent` field changed to hold a string that is a bioimage.io ID, a URL or a local relative path (and not subfields `uri` and `sha256`)
-
-#### model 0.4.4
-
-* Non-breaking changes
-  * new optional field `training_data`
-
-#### dataset 0.2.2
-
-* Non-breaking changes
-  * explicitly define and document dataset description (for now, clone of generic description with type="dataset")
-
-#### model 0.4.3
-
-* Non-breaking changes
-  * add optional field `download_url`
-  * add optional field `dependencies` to all weight formats (not only pytorch_state_dict)
-  * add optional `pytorch_version` to the pytorch_state_dict and torchscript weight formats
-
-#### model 0.4.2
-
-* Bug fixes:
-  * in a `pytorch_state_dict` weight entry `architecture` is no longer optional.
-
-#### collection 0.2.2
-
-* Non-breaking changes
-  * make `authors`, `cite`, `documentation` and `tags` optional
-
-* Breaking changes that are fully auto-convertible
-  * Simplifies collection description 0.2.1 by merging resource type fields together to a `collection` field,
-    holindg a list of all resources in the specified collection.
-
-#### generic 0.2.2 / model 0.3.6 / model 0.4.2
-
-* Non-breaking changes
-  * `rdf_source` new optional field
-  * `id` new optional field
-
-#### collection 0.2.1
-
-* First official release, extends generic description with fields `application`, `model`, `dataset`, `notebook` and (nested)
-  `collection`, which hold lists linking to respective resources.
-
-#### generic 0.2.1
-
-* Non-breaking changes
-  * add optional `email` and `github_user` fields to entries in `authors`
-  * add optional `maintainers` field (entries like in `authors` but  `github_user` is required (and `name` is not))
-
-#### model 0.4.1
-
-* Breaking changes that are fully auto-convertible
-  * moved field `dependencies` to `weights:pytorch_state_dict:dependencies`
-
-* Non-breaking changes
-  * `documentation` field accepts URLs as well
-
-#### model 0.3.5
-
-* Non-breaking changes
-  * `documentation` field accepts URLs as well
-
-#### model 0.4.0
-
-* Breaking changes
-  * model inputs and outputs may not use duplicated names.
-  * model field `sha256` is required if `pytorch_state_dict` weights are defined.
-    and is now moved to the `pytroch_state_dict` entry as `architecture_sha256`.
-
-* Breaking changes that are fully auto-convertible
-  * model fields language and framework are removed.
-  * model field `source` is renamed `architecture` and is moved together with `kwargs` to the `pytorch_state_dict`
-    weights entry (if it exists, otherwise they are removed).
-  * the weight format `pytorch_script` was renamed to `torchscript`.
-* Other changes
-  * model inputs (like outputs) may be defined by `scale`ing and `offset`ing a `reference_tensor`
-  * a `maintainers` field was added to the model description.
-  * the entries in the `authors` field may now additionally contain `email` or `github_user`.
-  * the summary returned by the `validate` command now also contains a list of warnings.
-  * an `update_format` command was added to aid with updating older RDFs by applying auto-conversion.
-
-#### model 0.3.4
-
-* Non-breaking changes
-  * Add optional parameter `eps` to `scale_range` postprocessing.
-
-#### model 0.3.3
-
-* Breaking changes that are fully auto-convertible
-  * `reference_input` for implicit output tensor shape was renamed to `reference_tensor`
-
-#### model 0.3.2
-
-* Breaking changes
-  * The RDF file name in a package should be `rdf.yaml` for all the RDF (not `model.yaml`);
-  * Change `authors` and `packaged_by` fields from List[str] to List[Author] with Author consisting of a dictionary `{name: '<Full name>', affiliation: '<Affiliation>', orcid: 'optional orcid id'}`;
-  * Add a mandatory `type` field to comply with the generic description. Only valid value is 'model' for model description;
-  * Only allow `license` identifier from the [SPDX license list](https://spdx.org/licenses/);
-
-* Non-breaking changes
-  * Add optional `version` field (default 0.1.0) to keep track of model changes;
-  * Allow the values in the `attachments` list to be any values besides URI;
+HyLFM-Net reconstructions should be validated using light sheet ground truth acquired with the same HyLFM.

package/bioimageio.yaml

@@ -35,7 +35,7 @@ tags:
   - image-reconstruction
   - nuclei
   - hylfm
-version: 1.2
+version: 1.3
 format_version: 0.5.7
 type: model
 id: ambitious-sloth
@@ -138,7 +138,6 @@ weights:
     sha256: 461f1151d7fea5857ce8f9ceaf9cdf08b5f78ce41785725e39a77d154ccea90a
     architecture:
       source: model.py
-      sha256: 7fbc9010a764a89e1bb6c162fc9df16eadb95d63bf3a1233cbcb61d82e3bab07
       callable: HyLFM_Net
       kwargs:
         c_in_3d: 64
@@ -162,9 +161,6 @@ weights:
         nnum: 19
         z_out: 49
     pytorch_version: 1.13
-    dependencies:
-      source: environment.yaml
-      sha256: e0c059d829fa03193eede76961746f464ac9b07d072b1e6ee62395d5c03c8606
   torchscript:
     source: weights_torchscript.pt
     sha256: ec01e0c212b5eb422dda208af004665799637a2f2729d0ebf2e884e5d9966fc2

package/environment.yaml

@@ -1,10 +0,0 @@
-name: hylfm
-
-channels:
-- conda-forge
-
-dependencies:
-- python=3.8.*
-- pytorch=1.9.*
-- torchvision=0.10.*
-- inferno=v0.4.2

package/model.py

@@ -1,289 +1,951 @@
-import collections
-import inspect
-from enum import Enum
-from functools import partial
-from typing import List, Optional, Sequence, Tuple, Union
-
-import torch.nn as nn
-from inferno.extensions.initializers import (
-    Constant,
-    Initialization,
-    KaimingNormalWeightsZeroBias,
-)
-from inferno.extensions.layers import convolutional as inferno_convolutional
-
-Conv2D = inferno_convolutional.Conv2D
-ValidConv3D = inferno_convolutional.ValidConv3D
-
-
-class Crop(nn.Module):
-    def __init__(self, *slices: slice):
-        super().__init__()
-        self.slices = slices
-
-    def extra_repr(self):
-        return str(self.slices)
-
-    def forward(self, input):
-        return input[self.slices]
-
-
-class ChannelFromLightField(nn.Module):
-    def __init__(self, nnum: int):
-        super().__init__()
-        self.nnum = nnum
-
-    def forward(self, tensor):
-        assert len(tensor.shape) == 4, tensor.shape
-        b, c, x, y = tensor.shape
-        assert c == 1
-        assert x % self.nnum == 0, (x, self.nnum)
-        assert y % self.nnum == 0, (y, self.nnum)
-        return (
-            tensor.reshape(b, x // self.nnum, self.nnum, y // self.nnum, self.nnum)
-            .transpose(1, 2)
-            .transpose(2, 4)
-            .transpose(3, 4)
-            .reshape(b, self.nnum**2, x // self.nnum, y // self.nnum)
-        )
-
-
-class ResnetBlock(nn.Module):
-    def __init__(
-        self,
-        in_n_filters,
-        n_filters,
-        kernel_size=(3, 3),
-        batch_norm=False,
-        conv_per_block=2,
-        valid: bool = False,
-        activation: str = "ReLU",
-    ):
-        super().__init__()
-        if batch_norm and activation != "ReLU":
-            raise NotImplementedError("batch_norm with non ReLU activation")
-
-        assert isinstance(kernel_size, tuple), kernel_size
-        assert conv_per_block >= 2
-        self.debug = False  #  sys.gettrace() is not None
-
-        Conv = getattr(
-            inferno_convolutional,
-            f"{'BNReLU' if batch_norm else ''}{'Valid' if valid else ''}Conv{'' if batch_norm else activation}{len(kernel_size)}D",
-        )
-        FinalConv = getattr(
-            inferno_convolutional, f"{'BNReLU' if batch_norm else ''}{'Valid' if valid else ''}Conv{len(kernel_size)}D"
-        )
-
-        layers = []
-        layers.append(Conv(in_channels=in_n_filters, out_channels=n_filters, kernel_size=kernel_size))
-
-        for _ in range(conv_per_block - 2):
-            layers.append(Conv(n_filters, n_filters, kernel_size))
-
-        layers.append(FinalConv(n_filters, n_filters, kernel_size))
-
-        self.block = nn.Sequential(*layers)
-
-        if n_filters != in_n_filters:
-            ProjConv = getattr(inferno_convolutional, f"Conv{len(kernel_size)}D")
-            self.projection_layer = ProjConv(in_n_filters, n_filters, kernel_size=1)
-        else:
-            self.projection_layer = None
-
-        if valid:
-            crop_each_side = [conv_per_block * (ks // 2) for ks in kernel_size]
-            self.crop = Crop(..., *[slice(c, -c) for c in crop_each_side])
-        else:
-            self.crop = None
-
-        self.relu = nn.ReLU()
-
-        # determine shrinkage
-        # self.shrinkage = (1, 1) + tuple([conv_per_block * (ks - 1) for ks in kernel_size])
-
-    def forward(self, input):
-        x = self.block(input)
-        if self.crop is not None:
-            input = self.crop(input)
-
-        if self.projection_layer is None:
-            x = x + input
-        else:
-            projected = self.projection_layer(input)
-            x = x + projected
-
-        x = self.relu(x)
-        return x
-
-
-class HyLFM_Net(nn.Module):
-    class InitName(str, Enum):
-        uniform_ = "uniform"
-        normal_ = "normal"
-        constant_ = "constant"
-        eye_ = "eye"
-        dirac_ = "dirac"
-        xavier_uniform_ = "xavier_uniform"
-        xavier_normal_ = "xavier_normal"
-        kaiming_uniform_ = "kaiming_uniform"
-        kaiming_normal_ = "kaiming_normal"
-        orthogonal_ = "orthogonal"
-        sparse_ = "sparse"
-
-    def __init__(
-        self,
-        *,
-        z_out: int,
-        nnum: int,
-        kernel2d: int = 3,
-        conv_per_block2d: int = 2,
-        c_res2d: Sequence[Union[int, str]] = (488, 488, "u244", 244),
-        last_kernel2d: int = 1,
-        c_in_3d: int = 7,
-        kernel3d: int = 3,
-        conv_per_block3d: int = 2,
-        c_res3d: Sequence[str] = (7, "u7", 7, 7),
-        init_fn: Union[InitName, str] = InitName.xavier_uniform_.value,
-        final_activation: Optional[str] = None,
-    ):
-        super().__init__()
-        self.channel_from_lf = ChannelFromLightField(nnum=nnum)
-        init_fn = self.InitName(init_fn)
-
-        init_fn = getattr(nn.init, init_fn.value)
-        self.c_res2d = list(c_res2d)
-        self.c_res3d = list(c_res3d)
-        c_res3d = c_res3d
-        self.nnum = nnum
-        self.z_out = z_out
-        if kernel3d != 3:
-            raise NotImplementedError("z_out expansion for other res3d kernel")
-
-        dz = 2 * conv_per_block3d * (kernel3d // 2)
-        for c in c_res3d:
-            if isinstance(c, int) or not c.startswith("u"):
-                z_out += dz
-
-        # z_out += 4 * (len(c_res3d) - 2 * sum([layer == "u" for layer in c_res3d]))  # add z_out for valid 3d convs
-
-        assert c_res2d[-1] != "u", "missing # output channels for upsampling in 'c_res2d'"
-        assert c_res3d[-1] != "u", "missing # output channels for upsampling in 'c_res3d'"
-
-        res2d = []
-        c_in = nnum**2
-        c_out = c_in
-        for i in range(len(c_res2d)):
-            if not isinstance(c_res2d[i], int) and c_res2d[i].startswith("u"):
-                c_out = int(c_res2d[i][1:])
-                res2d.append(
-                    nn.ConvTranspose2d(
-                        in_channels=c_in, out_channels=c_out, kernel_size=2, stride=2, padding=0, output_padding=0
-                    )
-                )
-            else:
-                c_out = int(c_res2d[i])
-                res2d.append(
-                    ResnetBlock(
-                        in_n_filters=c_in,
-                        n_filters=c_out,
-                        kernel_size=(kernel2d, kernel2d),
-                        valid=False,
-                        conv_per_block=conv_per_block2d,
-                    )
-                )
-
-            c_in = c_out
-
-        self.res2d = nn.Sequential(*res2d)
-
-        if "gain" in inspect.signature(init_fn).parameters:
-            init_fn_conv2d = partial(init_fn, gain=nn.init.calculate_gain("relu"))
-        else:
-            init_fn_conv2d = init_fn
-
-        init = Initialization(weight_initializer=init_fn_conv2d, bias_initializer=Constant(0.0))
-        self.conv2d = Conv2D(c_out, z_out * c_in_3d, last_kernel2d, activation="ReLU", initialization=init)
-
-        self.c2z = lambda ipt, ip3=c_in_3d: ipt.view(ipt.shape[0], ip3, z_out, *ipt.shape[2:])
-
-        res3d = []
-        c_in = c_in_3d
-        c_out = c_in
-        for i in range(len(c_res3d)):
-            if not isinstance(c_res3d[i], int) and c_res3d[i].startswith("u"):
-                c_out = int(c_res3d[i][1:])
-                res3d.append(
-                    nn.ConvTranspose3d(
-                        in_channels=c_in,
-                        out_channels=c_out,
-                        kernel_size=(3, 2, 2),
-                        stride=(1, 2, 2),
-                        padding=(1, 0, 0),
-                        output_padding=0,
-                    )
-                )
-            else:
-                c_out = int(c_res3d[i])
-                res3d.append(
-                    ResnetBlock(
-                        in_n_filters=c_in,
-                        n_filters=c_out,
-                        kernel_size=(kernel3d, kernel3d, kernel3d),
-                        valid=True,
-                        conv_per_block=conv_per_block3d,
-                    )
-                )
-
-            c_in = c_out
-
-        self.res3d = nn.Sequential(*res3d)
-
-        if "gain" in inspect.signature(init_fn).parameters:
-            init_fn_conv3d = partial(init_fn, gain=nn.init.calculate_gain("linear"))
-        else:
-            init_fn_conv3d = init_fn
-
-        init = Initialization(weight_initializer=init_fn_conv3d, bias_initializer=Constant(0.0))
-        self.conv3d = ValidConv3D(c_out, 1, (1, 1, 1), initialization=init)
-
-        if final_activation is None:
-            self.final_activation = None
-        elif final_activation == "sigmoid":
-            self.final_activation = nn.Sigmoid()
-        else:
-            raise NotImplementedError(final_activation)
-
-    def forward(self, x):
-        x = self.channel_from_lf(x)
-        x = self.res2d(x)
-        x = self.conv2d(x)
-        x = self.c2z(x)
-        x = self.res3d(x)
-        x = self.conv3d(x)
-
-        if self.final_activation is not None:
-            x = self.final_activation(x)
-
-        return x
-
-    def get_scale(self, ipt_shape: Optional[Tuple[int, int]] = None) -> int:
-        s = max(1, 2 * sum(isinstance(res2d, str) and res2d.startswith("u") for res2d in self.c_res2d)) * max(
-            1, 2 * sum(isinstance(res3d, str) and res3d.startswith("u") for res3d in self.c_res3d)
-        )
-        return s
-
-    def get_shrink(self, ipt_shape: Optional[Tuple[int, int]] = None) -> int:
-        s = 0
-        for res in self.c_res3d:
-            if isinstance(res, str) and res.startswith("u"):
-                s *= 2
-            else:
-                s += 2
-
-        return s
-
-    def get_output_shape(self, ipt_shape: Tuple[int, int]) -> Tuple[int, int, int]:
-        scale = self.get_scaling(ipt_shape)
-        shrink = self.get_shrink(ipt_shape)
-        return (self.z_out,) + tuple(i * scale - 2 * shrink for i in ipt_shape)
+# type: ignore
+import inspect
+from enum import Enum
+from functools import partial
+from typing import Optional, Sequence, Tuple, Union
+
+import numpy as np
+import torch.nn as nn
+
+### Inferno parts (adapted from inferno 0.4.2)
+
+
+def assert_(condition, message="", exception_type=AssertionError):
+    """Like assert, but with arbitrary exception types."""
+    if not condition:
+        raise exception_type(message)
+
+
+# proxy for generated classes in inferno
+generated_inferno_classes = {}
+
+
+def partial_cls(base_cls, name, fix=None, default=None):
+
+    # helper function
+    def insert_if_not_present(dict_a, dict_b):
+        for kw, val in dict_b.items():
+            if kw not in dict_a:
+                dict_a[kw] = val
+        return dict_a
+
+    # helper function
+    def insert_call_if_present(dict_a, dict_b, callback):
+        for kw, val in dict_b.items():
+            if kw not in dict_a:
+                dict_a[kw] = val
+            else:
+                callback(kw)
+        return dict_a
+
+    # helper class
+    class PartialCls(object):
+        def __init__(self, base_cls, name, fix=None, default=None):
+
+            self.base_cls = base_cls
+            self.name = name
+            self.fix = [fix, {}][fix is None]
+            self.default = [default, {}][default is None]
+
+            if self.fix.keys() & self.default.keys():
+                raise TypeError("fix and default share keys")
+
+            # remove binded kw
+            self._allowed_kw = self._get_allowed_kw()
+
+        def _get_allowed_kw(self):
+
+            argspec = inspect.getfullargspec(base_cls.__init__)
+            args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, annotations = (
+                argspec
+            )
+
+            if varargs is not None:
+                raise TypeError(
+                    "partial_cls can only be used if __init__ has no varargs"
+                )
+
+            if varkw is not None:
+                raise TypeError("partial_cls can only be used if __init__ has no varkw")
+
+            if kwonlyargs is not None and kwonlyargs != []:
+                raise TypeError("partial_cls can only be used without kwonlyargs")
+
+            if args is None or len(args) < 1:
+                raise TypeError("seems like self is missing")
+
+            return [kw for kw in args[1:] if kw not in self.fix]
+
+        def _build_kw(self, args, kwargs):
+            # handle *args
+            if len(args) > len(self._allowed_kw):
+                raise TypeError("to many arguments")
+
+            all_args = {}
+            for arg, akw in zip(args, self._allowed_kw):
+                all_args[akw] = arg
+
+            # handle **kwargs
+            intersection = self.fix.keys() & kwargs.keys()
+            if len(intersection) >= 1:
+                kw = intersection.pop()
+                raise TypeError(
+                    "`{}.__init__` got unexpected keyword argument '{}'".format(
+                        name, kw
+                    )
+                )
+
+            def raise_cb(kw):
+                raise TypeError(
+                    "{}.__init__ got multiple values for argument '{}'".format(name, kw)
+                )
+
+            all_args = insert_call_if_present(all_args, kwargs, raise_cb)
+
+            # handle fixed arguments
+            def raise_cb(kw):
+                raise TypeError()
+
+            all_args = insert_call_if_present(all_args, self.fix, raise_cb)
+
+            # handle defaults
+            all_args = insert_if_not_present(all_args, self.default)
+
+            # handle fixed
+            all_args.update(self.fix)
+
+            return all_args
+
+        def build_cls(self):
+
+            def new_init(self_of_new_cls, *args, **kwargs):
+                combined_args = self._build_kw(args=args, kwargs=kwargs)
+
+                # call base cls init
+                super(self_of_new_cls.__class__, self_of_new_cls).__init__(
+                    **combined_args
+                )
+
+            return type(name, (self.base_cls,), {"__init__": new_init})
+
+    return PartialCls(
+        base_cls=base_cls, name=name, fix=fix, default=default
+    ).build_cls()
+
+
+def register_partial_cls(base_cls, name, fix=None, default=None):
+    generatedClass = partial_cls(base_cls=base_cls, name=name, fix=fix, default=default)
+    generated_inferno_classes[generatedClass.__name__] = generatedClass
+
+
+class Initializer(object):
+    """
+    Base class for all initializers.
+    """
+
+    # TODO Support LSTMs and GRUs
+    VALID_LAYERS = {
+        "Conv1d",
+        "Conv2d",
+        "Conv3d",
+        "ConvTranspose1d",
+        "ConvTranspose2d",
+        "ConvTranspose3d",
+        "Linear",
+        "Bilinear",
+        "Embedding",
+    }
+
+    def __call__(self, module):
+        module_class_name = module.__class__.__name__
+        if module_class_name in self.VALID_LAYERS:
+            # Apply to weight and bias
+            try:
+                if hasattr(module, "weight"):
+                    self.call_on_weight(module.weight.data)
+            except NotImplementedError:
+                # Don't cry if it's not implemented
+                pass
+
+            try:
+                if hasattr(module, "bias"):
+                    self.call_on_bias(module.bias.data)
+            except NotImplementedError:
+                pass
+
+        return module
+
+    def call_on_bias(self, tensor):
+        return self.call_on_tensor(tensor)
+
+    def call_on_weight(self, tensor):
+        return self.call_on_tensor(tensor)
+
+    def call_on_tensor(self, tensor):
+        raise NotImplementedError
+
+    @classmethod
+    def initializes_weight(cls):
+        return "call_on_tensor" in cls.__dict__ or "call_on_weight" in cls.__dict__
+
+    @classmethod
+    def initializes_bias(cls):
+        return "call_on_tensor" in cls.__dict__ or "call_on_bias" in cls.__dict__
+
+
+class Initialization(Initializer):
+    def __init__(self, weight_initializer=None, bias_initializer=None):
+        if weight_initializer is None:
+            self.weight_initializer = Initializer()
+        else:
+            if isinstance(weight_initializer, Initializer):
+                assert weight_initializer.initializes_weight()
+                self.weight_initializer = weight_initializer
+            elif isinstance(weight_initializer, str):
+                init_function = getattr(nn.init, weight_initializer, None)
+                assert init_function is not None
+                self.weight_initializer = WeightInitFunction(
+                    init_function=init_function
+                )
+            else:
+                # Provison for weight_initializer to be a function
+                assert callable(weight_initializer)
+                self.weight_initializer = WeightInitFunction(
+                    init_function=weight_initializer
+                )
+
+        if bias_initializer is None:
+            self.bias_initializer = Initializer()
+        else:
+            if isinstance(bias_initializer, Initializer):
+                assert bias_initializer.initializes_bias
+                self.bias_initializer = bias_initializer
+            elif isinstance(bias_initializer, str):
+                init_function = getattr(nn.init, bias_initializer, None)
+                assert init_function is not None
+                self.bias_initializer = BiasInitFunction(init_function=init_function)
+            else:
+                assert callable(bias_initializer)
+                self.bias_initializer = BiasInitFunction(init_function=bias_initializer)
+
+    def call_on_weight(self, tensor):
+        return self.weight_initializer.call_on_weight(tensor)
+
+    def call_on_bias(self, tensor):
+        return self.bias_initializer.call_on_bias(tensor)
+
+
+class WeightInitFunction(Initializer):
+    def __init__(self, init_function, *init_function_args, **init_function_kwargs):
+        super(WeightInitFunction, self).__init__()
+        assert callable(init_function)
+        self.init_function = init_function
+        self.init_function_args = init_function_args
+        self.init_function_kwargs = init_function_kwargs
+
+    def call_on_weight(self, tensor):
+        return self.init_function(
+            tensor, *self.init_function_args, **self.init_function_kwargs
+        )
+
+
+class BiasInitFunction(Initializer):
+    def __init__(self, init_function, *init_function_args, **init_function_kwargs):
+        super(BiasInitFunction, self).__init__()
+        assert callable(init_function)
+        self.init_function = init_function
+        self.init_function_args = init_function_args
+        self.init_function_kwargs = init_function_kwargs
+
+    def call_on_bias(self, tensor):
+        return self.init_function(
+            tensor, *self.init_function_args, **self.init_function_kwargs
+        )
+
+
+class TensorInitFunction(Initializer):
+    def __init__(self, init_function, *init_function_args, **init_function_kwargs):
+        super(TensorInitFunction, self).__init__()
+        assert callable(init_function)
+        self.init_function = init_function
+        self.init_function_args = init_function_args
+        self.init_function_kwargs = init_function_kwargs
+
+    def call_on_tensor(self, tensor):
+        return self.init_function(
+            tensor, *self.init_function_args, **self.init_function_kwargs
+        )
+
+
+class Constant(Initializer):
+    """Initialize with a constant."""
+
+    def __init__(self, constant):
+        self.constant = constant
+
+    def call_on_tensor(self, tensor):
+        tensor.fill_(self.constant)
+        return tensor
+
+
+class NormalWeights(Initializer):
+    """
+    Initialize weights with random numbers drawn from the normal distribution at
+    `mean` and `stddev`.
+    """
+
+    def __init__(self, mean=0.0, stddev=1.0, sqrt_gain_over_fan_in=None):
+        self.mean = mean
+        self.stddev = stddev
+        self.sqrt_gain_over_fan_in = sqrt_gain_over_fan_in
+
+    def compute_fan_in(self, tensor):
+        if tensor.dim() == 2:
+            return tensor.size(1)
+        else:
+            return np.prod(list(tensor.size())[1:])
+
+    def call_on_weight(self, tensor):
+        # Compute stddev if required
+        if self.sqrt_gain_over_fan_in is not None:
+            stddev = self.stddev * np.sqrt(
+                self.sqrt_gain_over_fan_in / self.compute_fan_in(tensor)
+            )
+        else:
+            stddev = self.stddev
+        # Init
+        tensor.normal_(self.mean, stddev)
+
+
+class OrthogonalWeightsZeroBias(Initialization):
+    def __init__(self, orthogonal_gain=1.0):
+        # This prevents a deprecated warning in Pytorch 0.4+
+        orthogonal = getattr(nn.init, "orthogonal_", nn.init.orthogonal)
+        super(OrthogonalWeightsZeroBias, self).__init__(
+            weight_initializer=partial(orthogonal, gain=orthogonal_gain),
+            bias_initializer=Constant(0.0),
+        )
+
+
+class KaimingNormalWeightsZeroBias(Initialization):
+    def __init__(self, relu_leakage=0):
+        # This prevents a deprecated warning in Pytorch 0.4+
+        kaiming_normal = getattr(nn.init, "kaiming_normal_", nn.init.kaiming_normal)
+        super(KaimingNormalWeightsZeroBias, self).__init__(
+            weight_initializer=partial(kaiming_normal, a=relu_leakage),
+            bias_initializer=Constant(0.0),
+        )
+
+
+class SELUWeightsZeroBias(Initialization):
+    def __init__(self):
+        super(SELUWeightsZeroBias, self).__init__(
+            weight_initializer=NormalWeights(sqrt_gain_over_fan_in=1.0),
+            bias_initializer=Constant(0.0),
+        )
+
+
+class ELUWeightsZeroBias(Initialization):
+    def __init__(self):
+        super(ELUWeightsZeroBias, self).__init__(
+            weight_initializer=NormalWeights(sqrt_gain_over_fan_in=1.5505188080679277),
+            bias_initializer=Constant(0.0),
+        )
+
+
+class BatchNormND(nn.Module):
+    def __init__(
+        self,
+        dim,
+        num_features,
+        eps=1e-5,
+        momentum=0.1,
+        affine=True,
+        track_running_stats=True,
+    ):
+        super(BatchNormND, self).__init__()
+        assert dim in [1, 2, 3]
+        self.bn = getattr(nn, "BatchNorm{}d".format(dim))(
+            num_features=num_features,
+            eps=eps,
+            momentum=momentum,
+            affine=affine,
+            track_running_stats=track_running_stats,
+        )
+
+    def forward(self, x):
+        return self.bn(x)
+
+
+class ConvActivation(nn.Module):
+    """Convolutional layer with 'SAME' padding by default followed by an activation."""
+
+    def __init__(
+        self,
+        in_channels,
+        out_channels,
+        kernel_size,
+        dim,
+        activation,
+        stride=1,
+        dilation=1,
+        groups=None,
+        depthwise=False,
+        bias=True,
+        deconv=False,
+        initialization=None,
+        valid_conv=False,
+    ):
+        super(ConvActivation, self).__init__()
+        # Validate dim
+        assert_(
+            dim in [1, 2, 3],
+            "`dim` must be one of [1, 2, 3], got {}.".format(dim),
+        )
+        self.dim = dim
+        # Check if depthwise
+        if depthwise:
+
+            # We know that in_channels == out_channels, but we also want a consistent API.
+            # As a compromise, we allow that out_channels be None or 'auto'.
+            out_channels = (
+                in_channels if out_channels in [None, "auto"] else out_channels
+            )
+            assert_(
+                in_channels == out_channels,
+                "For depthwise convolutions, number of input channels (given: {}) "
+                "must equal the number of output channels (given {}).".format(
+                    in_channels, out_channels
+                ),
+                ValueError,
+            )
+            assert_(
+                groups is None or groups == in_channels,
+                "For depthwise convolutions, groups (given: {}) must "
+                "equal the number of channels (given: {}).".format(groups, in_channels),
+            )
+            groups = in_channels
+        else:
+            groups = 1 if groups is None else groups
+        self.depthwise = depthwise
+        if valid_conv:
+            self.conv = getattr(nn, "Conv{}d".format(self.dim))(
+                in_channels=in_channels,
+                out_channels=out_channels,
+                kernel_size=kernel_size,
+                stride=stride,
+                dilation=dilation,
+                groups=groups,
+                bias=bias,
+            )
+        elif not deconv:
+            # Get padding
+            padding = self.get_padding(kernel_size, dilation)
+            self.conv = getattr(nn, "Conv{}d".format(self.dim))(
+                in_channels=in_channels,
+                out_channels=out_channels,
+                kernel_size=kernel_size,
+                padding=padding,
+                stride=stride,
+                dilation=dilation,
+                groups=groups,
+                bias=bias,
+            )
+        else:
+            self.conv = getattr(nn, "ConvTranspose{}d".format(self.dim))(
+                in_channels=in_channels,
+                out_channels=out_channels,
+                kernel_size=kernel_size,
+                stride=stride,
+                dilation=dilation,
+                groups=groups,
+                bias=bias,
+            )
+        if initialization is None:
+            pass
+        elif isinstance(initialization, Initializer):
+            self.conv.apply(initialization)
+        else:
+            raise NotImplementedError
+
+        if isinstance(activation, str):
+            self.activation = getattr(nn, activation)()
+        elif isinstance(activation, nn.Module):
+            self.activation = activation
+        elif activation is None:
+            self.activation = None
+        else:
+            raise NotImplementedError
+
+    def forward(self, input):
+        conved = self.conv(input)
+        if self.activation is not None:
+            activated = self.activation(conved)
+        else:
+            # No activation
+            activated = conved
+        return activated
+
+    def _pair_or_triplet(self, object_):
+        if isinstance(object_, (list, tuple)):
+            assert len(object_) == self.dim
+            return object_
+        else:
+            object_ = [object_] * self.dim
+            return object_
+
+    def _get_padding(self, _kernel_size, _dilation):
+        assert isinstance(_kernel_size, int)
+        assert isinstance(_dilation, int)
+        assert _kernel_size % 2 == 1
+        return ((_kernel_size - 1) // 2) * _dilation
+
+    def get_padding(self, kernel_size, dilation):
+        kernel_size = self._pair_or_triplet(kernel_size)
+        dilation = self._pair_or_triplet(dilation)
+        padding = [
+            self._get_padding(_kernel_size, _dilation)
+            for _kernel_size, _dilation in zip(kernel_size, dilation)
+        ]
+        return tuple(padding)
+
+
+# for consistency
+ConvActivationND = ConvActivation
+
+
+class _BNReLUSomeConv(object):
+    def forward(self, input):
+        normed = self.batchnorm(input)
+        activated = self.activation(normed)
+        conved = self.conv(activated)
+        return conved
+
+
+class BNReLUConvBaseND(_BNReLUSomeConv, ConvActivation):
+    def __init__(
+        self,
+        in_channels,
+        out_channels,
+        kernel_size,
+        dim,
+        stride=1,
+        dilation=1,
+        deconv=False,
+    ):
+
+        super(BNReLUConvBaseND, self).__init__(
+            in_channels=in_channels,
+            out_channels=out_channels,
+            kernel_size=kernel_size,
+            dim=dim,
+            stride=stride,
+            activation=nn.ReLU(inplace=True),
+            dilation=dilation,
+            deconv=deconv,
+            initialization=KaimingNormalWeightsZeroBias(0),
+        )
+        self.batchnorm = BatchNormND(dim, in_channels)
+
+
+def _register_bnr_conv_cls(conv_name, fix=None, default=None):
+    if fix is None:
+        fix = {}
+    if default is None:
+        default = {}
+    for dim in [1, 2, 3]:
+
+        cls_name = "BNReLU{}ND".format(conv_name)
+        register_partial_cls(BNReLUConvBaseND, cls_name, fix=fix, default=default)
+
+        for dim in [1, 2, 3]:
+            cls_name = "BNReLU{}{}D".format(conv_name, dim)
+
+            register_partial_cls(
+                BNReLUConvBaseND, cls_name, fix={**fix, "dim": dim}, default=default
+            )
+
+
+def _register_conv_cls(conv_name, fix=None, default=None):
+    if fix is None:
+        fix = {}
+    if default is None:
+        default = {}
+
+    # simple conv activation
+    activations = ["ReLU", "ELU", "Sigmoid", "SELU", ""]
+    init_map = {"ReLU": KaimingNormalWeightsZeroBias, "SELU": SELUWeightsZeroBias}
+    for activation_str in activations:
+        cls_name = cls_name = "{}{}ND".format(conv_name, activation_str)
+        initialization_cls = init_map.get(activation_str, OrthogonalWeightsZeroBias)
+        if activation_str == "":
+            activation = None
+            _fix = {**fix}
+            _default = {"activation": None}
+        elif activation_str == "SELU":
+            activation = nn.SELU(inplace=True)
+            _fix = {**fix, "activation": activation}
+            _default = {**default}
+        else:
+            activation = activation_str
+            _fix = {**fix, "activation": activation}
+            _default = {**default}
+
+        register_partial_cls(
+            ConvActivation,
+            cls_name,
+            fix=_fix,
+            default={**_default, "initialization": initialization_cls()},
+        )
+        for dim in [1, 2, 3]:
+            cls_name = "{}{}{}D".format(conv_name, activation_str, dim)
+            register_partial_cls(
+                ConvActivation,
+                cls_name,
+                fix={**_fix, "dim": dim},
+                default={**_default, "initialization": initialization_cls()},
+            )
+
+
+_register_conv_cls("Conv")
+_register_conv_cls("ValidConv", fix=dict(valid_conv=True))
+
+Conv2D = generated_inferno_classes["Conv2D"]
+ValidConv3D = generated_inferno_classes["ValidConv3D"]
+
+
+### HyLFM architecture
+class Crop(nn.Module):
+    def __init__(self, *slices: slice):
+        super().__init__()
+        self.slices = slices
+
+    def extra_repr(self):
+        return str(self.slices)
+
+    def forward(self, input):
+        return input[self.slices]
+
+
+class ChannelFromLightField(nn.Module):
+    def __init__(self, nnum: int):
+        super().__init__()
+        self.nnum = nnum
+
+    def forward(self, tensor):
+        assert len(tensor.shape) == 4, tensor.shape
+        b, c, x, y = tensor.shape
+        assert c == 1
+        assert x % self.nnum == 0, (x, self.nnum)
+        assert y % self.nnum == 0, (y, self.nnum)
+        return (
+            tensor.reshape(b, x // self.nnum, self.nnum, y // self.nnum, self.nnum)
+            .transpose(1, 2)
+            .transpose(2, 4)
+            .transpose(3, 4)
+            .reshape(b, self.nnum**2, x // self.nnum, y // self.nnum)
+        )
+
+
+class ResnetBlock(nn.Module):
+    def __init__(
+        self,
+        in_n_filters,
+        n_filters,
+        kernel_size=(3, 3),
+        batch_norm=False,
+        conv_per_block=2,
+        valid: bool = False,
+        activation: str = "ReLU",
+    ):
+        super().__init__()
+        if batch_norm and activation != "ReLU":
+            raise NotImplementedError("batch_norm with non ReLU activation")
+
+        assert isinstance(kernel_size, tuple), kernel_size
+        assert conv_per_block >= 2
+        self.debug = False  #  sys.gettrace() is not None
+
+        Conv = generated_inferno_classes[
+            f"{'BNReLU' if batch_norm else ''}{'Valid' if valid else ''}Conv{'' if batch_norm else activation}{len(kernel_size)}D"
+        ]
+        FinalConv = generated_inferno_classes[
+            f"{'BNReLU' if batch_norm else ''}{'Valid' if valid else ''}Conv{len(kernel_size)}D"
+        ]
+
+        layers = []
+        layers.append(
+            Conv(
+                in_channels=in_n_filters,
+                out_channels=n_filters,
+                kernel_size=kernel_size,
+            )
+        )
+
+        for _ in range(conv_per_block - 2):
+            layers.append(Conv(n_filters, n_filters, kernel_size))
+
+        layers.append(FinalConv(n_filters, n_filters, kernel_size))
+
+        self.block = nn.Sequential(*layers)
+
+        if n_filters != in_n_filters:
+            ProjConv = generated_inferno_classes[f"Conv{len(kernel_size)}D"]
+            self.projection_layer = ProjConv(in_n_filters, n_filters, kernel_size=1)
+        else:
+            self.projection_layer = None
+
+        if valid:
+            crop_each_side = [conv_per_block * (ks // 2) for ks in kernel_size]
+            self.crop = Crop(..., *[slice(c, -c) for c in crop_each_side])
+        else:
+            self.crop = None
+
+        self.relu = nn.ReLU()
+
+        # determine shrinkage
+        # self.shrinkage = (1, 1) + tuple([conv_per_block * (ks - 1) for ks in kernel_size])
+
+    def forward(self, input):
+        x = self.block(input)
+        if self.crop is not None:
+            input = self.crop(input)
+
+        if self.projection_layer is None:
+            x = x + input
+        else:
+            projected = self.projection_layer(input)
+            x = x + projected
+
+        x = self.relu(x)
+        return x
+
+
+class HyLFM_Net(nn.Module):
+    class InitName(str, Enum):
+        uniform_ = "uniform"
+        normal_ = "normal"
+        constant_ = "constant"
+        eye_ = "eye"
+        dirac_ = "dirac"
+        xavier_uniform_ = "xavier_uniform"
+        xavier_normal_ = "xavier_normal"
+        kaiming_uniform_ = "kaiming_uniform"
+        kaiming_normal_ = "kaiming_normal"
+        orthogonal_ = "orthogonal"
+        sparse_ = "sparse"
+
+    def __init__(
+        self,
+        *,
+        z_out: int,
+        nnum: int,
+        kernel2d: int = 3,
+        conv_per_block2d: int = 2,
+        c_res2d: Sequence[Union[int, str]] = (488, 488, "u244", 244),
+        last_kernel2d: int = 1,
+        c_in_3d: int = 7,
+        kernel3d: int = 3,
+        conv_per_block3d: int = 2,
+        c_res3d: Sequence[str] = (7, "u7", 7, 7),
+        init_fn: Union[InitName, str] = InitName.xavier_uniform_.value,
+        final_activation: Optional[str] = None,
+    ):
+        super().__init__()
+        self.channel_from_lf = ChannelFromLightField(nnum=nnum)
+        init_fn = self.InitName(init_fn)
+
+        if hasattr(nn.init, f"{init_fn.value}_"):
+            # prevents deprecation warning
+            init_fn = getattr(nn.init, f"{init_fn.value}_")
+        else:
+            init_fn = getattr(nn.init, init_fn.value)
+
+        self.c_res2d = list(c_res2d)
+        self.c_res3d = list(c_res3d)
+        c_res3d = c_res3d
+        self.nnum = nnum
+        self.z_out = z_out
+        if kernel3d != 3:
+            raise NotImplementedError("z_out expansion for other res3d kernel")
+
+        dz = 2 * conv_per_block3d * (kernel3d // 2)
+        for c in c_res3d:
+            if isinstance(c, int) or not c.startswith("u"):
+                z_out += dz
+
+        # z_out += 4 * (len(c_res3d) - 2 * sum([layer == "u" for layer in c_res3d]))  # add z_out for valid 3d convs
+
+        assert (
+            c_res2d[-1] != "u"
+        ), "missing # output channels for upsampling in 'c_res2d'"
+        assert (
+            c_res3d[-1] != "u"
+        ), "missing # output channels for upsampling in 'c_res3d'"
+
+        res2d = []
+        c_in = nnum**2
+        c_out = c_in
+        for i in range(len(c_res2d)):
+            if not isinstance(c_res2d[i], int) and c_res2d[i].startswith("u"):
+                c_out = int(c_res2d[i][1:])
+                res2d.append(
+                    nn.ConvTranspose2d(
+                        in_channels=c_in,
+                        out_channels=c_out,
+                        kernel_size=2,
+                        stride=2,
+                        padding=0,
+                        output_padding=0,
+                    )
+                )
+            else:
+                c_out = int(c_res2d[i])
+                res2d.append(
+                    ResnetBlock(
+                        in_n_filters=c_in,
+                        n_filters=c_out,
+                        kernel_size=(kernel2d, kernel2d),
+                        valid=False,
+                        conv_per_block=conv_per_block2d,
+                    )
+                )
+
+            c_in = c_out
+
+        self.res2d = nn.Sequential(*res2d)
+
+        if "gain" in inspect.signature(init_fn).parameters:
+            init_fn_conv2d = partial(init_fn, gain=nn.init.calculate_gain("relu"))
+        else:
+            init_fn_conv2d = init_fn
+
+        init = Initialization(
+            weight_initializer=init_fn_conv2d, bias_initializer=Constant(0.0)
+        )
+        self.conv2d = Conv2D(
+            c_out,
+            z_out * c_in_3d,
+            last_kernel2d,
+            activation="ReLU",
+            initialization=init,
+        )
+
+        self.c2z = lambda ipt, ip3=c_in_3d: ipt.view(
+            ipt.shape[0], ip3, z_out, *ipt.shape[2:]
+        )
+
+        res3d = []
+        c_in = c_in_3d
+        c_out = c_in
+        for i in range(len(c_res3d)):
+            if not isinstance(c_res3d[i], int) and c_res3d[i].startswith("u"):
+                c_out = int(c_res3d[i][1:])
+                res3d.append(
+                    nn.ConvTranspose3d(
+                        in_channels=c_in,
+                        out_channels=c_out,
+                        kernel_size=(3, 2, 2),
+                        stride=(1, 2, 2),
+                        padding=(1, 0, 0),
+                        output_padding=0,
+                    )
+                )
+            else:
+                c_out = int(c_res3d[i])
+                res3d.append(
+                    ResnetBlock(
+                        in_n_filters=c_in,
+                        n_filters=c_out,
+                        kernel_size=(kernel3d, kernel3d, kernel3d),
+                        valid=True,
+                        conv_per_block=conv_per_block3d,
+                    )
+                )
+
+            c_in = c_out
+
+        self.res3d = nn.Sequential(*res3d)
+
+        if "gain" in inspect.signature(init_fn).parameters:
+            init_fn_conv3d = partial(init_fn, gain=nn.init.calculate_gain("linear"))
+        else:
+            init_fn_conv3d = init_fn
+
+        init = Initialization(
+            weight_initializer=init_fn_conv3d, bias_initializer=Constant(0.0)
+        )
+        self.conv3d = ValidConv3D(c_out, 1, (1, 1, 1), initialization=init)
+
+        if final_activation is None:
+            self.final_activation = None
+        elif final_activation == "sigmoid":
+            self.final_activation = nn.Sigmoid()
+        else:
+            raise NotImplementedError(final_activation)
+
+    def forward(self, x):
+        x = self.channel_from_lf(x)
+        x = self.res2d(x)
+        x = self.conv2d(x)
+        x = self.c2z(x)
+        x = self.res3d(x)
+        x = self.conv3d(x)
+
+        if self.final_activation is not None:
+            x = self.final_activation(x)
+
+        return x
+
+    def get_scale(self, ipt_shape: Optional[Tuple[int, int]] = None) -> int:
+        s = max(
+            1,
+            2
+            * sum(
+                isinstance(res2d, str) and res2d.startswith("u")
+                for res2d in self.c_res2d
+            ),
+        ) * max(
+            1,
+            2
+            * sum(
+                isinstance(res3d, str) and res3d.startswith("u")
+                for res3d in self.c_res3d
+            ),
+        )
+        return s
+
+    def get_shrink(self, ipt_shape: Optional[Tuple[int, int]] = None) -> int:
+        s = 0
+        for res in self.c_res3d:
+            if isinstance(res, str) and res.startswith("u"):
+                s *= 2
+            else:
+                s += 2
+
+        return s
+
+    def get_output_shape(self, ipt_shape: Tuple[int, int]) -> Tuple[int, int, int]:
+        scale = self.get_scale(ipt_shape)
+        shrink = self.get_shrink(ipt_shape)
+        return (self.z_out,) + tuple(i * scale - 2 * shrink for i in ipt_shape)
+
+
+if __name__ == "__main__":
+    # Example usage
+    model = HyLFM_Net(
+        z_out=9,
+        nnum=5,
+        kernel2d=3,
+        conv_per_block2d=2,
+        c_res2d=(12, 14, "u14", 8),
+        last_kernel2d=1,
+        c_in_3d=7,
+        kernel3d=3,
+        conv_per_block3d=2,
+        c_res3d=(7, "u7", 7, 7),
+        init_fn="xavier_uniform",
+        final_activation="sigmoid",
+    )
+    print(model)
+    print(model.get_output_shape((64, 64)))