Noise model training CAREamist

[CatEek]

Description

Note

tldr:

High level API noise model training discussion

Implementation - how did you implement the changes?

Currently NM training is a separate step and the weight are passed to the MultiChannelNMConfig as paths to .npy files, 1 NM per channel

To train a NM a user has to provide either a noisy/clean image pairs or get the latter with N2V(or possibly any other denoising algo)

in CAREamist API we would want to abstract the whole NM training step away. But that would require esssentially calling CAREamist from within itself(e.g. to train N2V) and/or to provide very convoluted configuration object.

Proposed solution:

#N2V training step
config = create_n2v_configuration(...)
careamist = CAREamist(source=config, work_dir="noise_models")
careamist.train(train_source=nm_input, val_minimum_split=5)
prediction = careamist.predict(nm_input, tile_size=(256, 256))

#MS config & training
config = create_microsplit_configuration( # here we don't provide paths anymore
     ...,
    noise_model_training_params={} 
)
careamist = CAREamist(config)
careamist.train_noise_model(
    clean_data=clean_images, 
    noisy_data=noisy_images,
    n_epochs=1000,
    learning_rate=0.05
)
careamist.train(train_source=training_data)

How has this been tested?

After 1st round of discussion I will create a corresponding notebook.

Related Issues

• Resolves High level Noise Model interface #536

---

Please ensure your PR meets the following requirements:

• Code builds and passes tests locally, including doctests
• New tests have been added (for bug fixes/features)
• Pre-commit passes
• PR to the documentation exists (for bug fixes / features)

[jdeschamps]

Thanks @CatEek! Here are some thoughts:

1. The API looks ok to me. I understand that there is dependency between the MicroSplit and NM parameters, but I have a slight preference for the following pipeline:

# N2V training step
config = create_n2v_configuration(...)
careamist = CAREamist(source=config, work_dir="noise_models")
careamist.train(train_source=nm_input, val_minimum_split=5)
prediction = careamist.predict(nm_input, tile_size=(256, 256))


# Train noise model
noise_model = train_noise_model(
    clean=...,
    noisy=...,
    n_channels=...,
    ...
)

# MS config & training
config = create_microsplit_configuration(...)
careamist = CAREamist(config, noise_model=...)
careamist.train(train_source=training_data)

2. I don't understand what would happen in _update_config_with_trained_noise_models. What would change in the configuration?

3. If train_noise_model stays in CAREamist then the loop over channels and saving NM should be refactored outside of that function.

[CatEek]

Thx for the review.
2. This might not be needed, just in general after training the NM the weights should be stored somewhere. If we put the NM training inside CAREamist, all the params for training it can be saved in the config, which seem to be more natural.
3. Saving I agree, also saving step becomes optional if we train it inside. But looping over, why?

[jdeschamps]

But looping over, why?

It is part of the generation of the noise models, so it should be folded into a NM-specific model. I don't think it should not be the responsibility of the CAREamist to loop over the channels.