Skip to content
Snippets Groups Projects
Commit 34050b45 authored by ROUPPERT Leo's avatar ROUPPERT Leo
Browse files

Update 7 files

- /images/DOT_3_simplified_curve.jpg
- /images/DOT_3_simplified.jpg
- /README.md
- /requirements.txt
- /ot-image-restoration.ipynb
- /model_fashion_mnist.pt
- /Deep_Optimal_Transport_Presentation.pdf
parents
Branches
No related tags found
No related merge requests found
File added
# README
### Team : Fatma Zahra Zaghouani, Islem Chaari, Léo Rouppert
## Description
The paper "Deep Optimal Transport: A Practical Algorithm for Photo-realistic Image Restoration" presents an image restoration algorithm that allows control over the perceptual quality and/or mean square error (MSE) of any pre-trained model, enabling a trade-off between these aspects during testing. This few-shot algorithm requires only about a dozen images restored by the model to significantly enhance the perceptual quality or MSE for new images without additional training.
The approach is based on theoretical findings that connect the minimum MSE predictor with the predictor that minimizes MSE under a perfect perceptual quality constraint, achieved by optimally transporting the output of the former to match the source data distribution. This optimal transport is approximated by a linear transformation in the latent space of a variational autoencoder, computed in closed-form using empirical means and covariances.
## Installation
Make sure Python is installed on your machine. Then, install the required dependencies with:
```sh
pip install -r requirements.txt
```
## Usage
Run the notebook with the parameters you want to test. A pretrained VAE model on the Fashion-MNIST dataset is provided.
## Difference Between the Original and Simplified Transport Operator
The original optimal transport operator based on the paper has been modified for computational simplification while retaining its fundamental properties.
In the original version, the transport operator is defined as:
$$
T_{p_x \longrightarrow p_y}(x) = \Sigma_x^{-\frac{1}{2}} \left(\Sigma_x^{\frac{1}{2}} \Sigma_{y} \Sigma_x^{\frac{1}{2}}\right)^{\frac{1}{2}} \Sigma_x^{-\frac{1}{2}} (x - \mu_{x}) + \mu_y
$$
where $x \sim \mathcal{N}(\mu_{x}, \Sigma_x)$ and $y \sim \mathcal{N}(\mu_{y}, \Sigma_y)$.
In the simplified version, the transport operator is rewritten by replacing the successive multiplications with simpler transform :
$$
T_{p_{x} \longrightarrow p_{y}}(x) = \Sigma_y^{\frac{1}{2}} \Sigma_x^{-\frac{1}{2}} (x - \mu_x) + \mu_y
$$
This reformulation reduces the number of matrix multiplications and facilitates the inversion of matrices, improving the numerical stability of the model while maintaining a transformation close to the original. This approach led to some successful results whereas the original transport operator failed to improve perceptual quality.
## Example Results
![Results](images/DOT_3_simplified.jpg)
![Curve](images/DOT_3_simplified_curve.jpg)
## References
- [Deep Optimal Transport: A Practical Algorithm for Photo-realistic Image Restoration](https://arxiv.org/pdf/2306.02342).
## Author
This code was developed by Léo Rouppert.
\ No newline at end of file
images/DOT_3_simplified.jpg

143 KiB

images/DOT_3_simplified_curve.jpg

62 KiB

File added
Source diff could not be displayed: it is too large. Options to address this: view the blob.
torch
torchvision
numpy
pandas
matplotlib
pillow
scipy
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment