-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a baseline CFIS dataset #40
Conversation
Super easy to use: import autometacal
import tensorflow_dataset as tfds
import tensorflow as tf
# We create a function to add noise on the fly for data augmentation
@tf.function
def add_noise(example):
im_noise = example['obs'] + example['noise_std'] * tf.random.normal([51,51])
return im_noise, example
dset = tfds.load('CFIS/parametric_shear_1k', split='train')
dset = dset.map(add_noise)
for im, example in dset:
# im contains the noisy observation
# example contains psf image, observation without noise, and galaxy magnitude
.... |
I checked the notebook, I am a bit confused by the results:
|
Dont look too closely at the notebook. The response computed by autometacal is wrong, cf #26. The main point of this PR is to add the dataset, the notebook is to illustrate how to load and use the dataset, but it's optional, if you think it's not useful I can drop it for now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a few comments
'psf': tfds.features.Tensor(shape=[self.builder_config.stamp_size, | ||
self.builder_config.stamp_size], | ||
dtype=tf.float32), | ||
# 'gal_kimage': tfds.features.Tensor(shape=[2, self.builder_config.kstamp_size, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what do you have these commented lines? If they are not needed we should remove to avoid cluttering the code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that;s a good point, I'll rmove this stuff for now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gtg
This PR adds a first pass of a tf dataset of CFIS-like images, this aims at solving #39.
This is a first pass because we will probably want to add additional variants on that dataset in the future, but it is a starting point