Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distinct samples of training data #16

Open
leowe opened this issue Jun 27, 2019 · 5 comments
Open

Distinct samples of training data #16

leowe opened this issue Jun 27, 2019 · 5 comments

Comments

@leowe
Copy link

leowe commented Jun 27, 2019

Is it possible to train the ffn with several distinct pieces of data with each having its own bounding box?

@subeeshvasu
Copy link

subeeshvasu commented Jun 28, 2019

I am also trying to do the same. After inspecting the code, I figured out the following solution, to train with more than one data volumes.
Let's assume that we have two training volumes at hand.

  • First, generate the coordinates for both volumes, but with different tags. i.e., while using 'build_coordinates.py' we could use "validation1" as the tag for volume 1 and "validation2" as the tag for volume 2.
    eg. python build_coordinates.py --partition_volumes validation1:address_to_volume1_af.h5:af_file,validation2:address_to_volume2_af.h5:af_file --coordinate_output ..
  • Next, change the arguments corresponding to input volume and corresponding labels:
    data_volumes
    from
    validation1:address_to_volume_1.h5:raw
    to
    validation1:address_to_volume_1.h5:raw,validation2:address_to_volume_2.h5:raw

Repeat the above step for
label_volumes
too

  • Now run the training code as usual.

@mk123qwe
Copy link

mk123qwe commented Jul 4, 2019

How to save the model according to the set conditions, not periodically.
I noticed that MonitoredSession is a very inefficient method.

@mk123qwe
Copy link

mk123qwe commented Jul 30, 2019

I am also trying to do the same. After inspecting the code, I figured out the following solution, to train with more than one data volumes.
Let's assume that we have two training volumes at hand.

  • First, generate the coordinates for both volumes, but with different tags. i.e., while using 'build_coordinates.py' we could use "validation1" as the tag for volume 1 and "validation2" as the tag for volume 2.
    eg. python build_coordinates.py --partition_volumes validation1:address_to_volume1_af.h5:af_file,validation2:address_to_volume2_af.h5:af_file --coordinate_output ..
  • Next, change the arguments corresponding to input volume and corresponding labels:
    data_volumes
    from
    validation1:address_to_volume_1.h5:raw
    to
    validation1:address_to_volume_1.h5:raw,validation2:address_to_volume_2.h5:raw

Repeat the above step for
label_volumes
too

  • Now run the training code as usual.

Hello, I'm confused that this program has no validation set.

@subeeshvasu
Copy link

I am also trying to do the same. After inspecting the code, I figured out the following solution, to train with more than one data volumes.
Let's assume that we have two training volumes at hand.

  • First, generate the coordinates for both volumes, but with different tags. i.e., while using 'build_coordinates.py' we could use "validation1" as the tag for volume 1 and "validation2" as the tag for volume 2.
    eg. python build_coordinates.py --partition_volumes validation1:address_to_volume1_af.h5:af_file,validation2:address_to_volume2_af.h5:af_file --coordinate_output ..
  • Next, change the arguments corresponding to input volume and corresponding labels:
    data_volumes
    from
    validation1:address_to_volume_1.h5:raw
    to
    validation1:address_to_volume_1.h5:raw,validation2:address_to_volume_2.h5:raw

Repeat the above step for
label_volumes
too

  • Now run the training code as usual.

Hello, I'm confused that this program has no verification set.
I didn't understand. You meant: "No setup/data to check monitor the improvement in segmentation?". If yes, I agree. I would also like to figure out a way to save the network weights based on some sort of quantitative evaluation. I haven't explored that direction yet. Let me know if you figure out a way to do it. Thanks.

@mk123qwe
Copy link

I am also trying to do the same. After inspecting the code, I figured out the following solution, to train with more than one data volumes.
Let's assume that we have two training volumes at hand.

  • First, generate the coordinates for both volumes, but with different tags. i.e., while using 'build_coordinates.py' we could use "validation1" as the tag for volume 1 and "validation2" as the tag for volume 2.
    eg. python build_coordinates.py --partition_volumes validation1:address_to_volume1_af.h5:af_file,validation2:address_to_volume2_af.h5:af_file --coordinate_output ..
  • Next, change the arguments corresponding to input volume and corresponding labels:
    data_volumes
    from
    validation1:address_to_volume_1.h5:raw
    to
    validation1:address_to_volume_1.h5:raw,validation2:address_to_volume_2.h5:raw

Repeat the above step for
label_volumes
too

  • Now run the training code as usual.

Hello, I'm confused that this program has no verification set.
I didn't understand. You meant: "No setup/data to check monitor the improvement in segmentation?". If yes, I agree. I would also like to figure out a way to save the network weights based on some sort of quantitative evaluation. I haven't explored that direction yet. Let me know if you figure out a way to do it. Thanks.

I don't understand enough about FFN training process, which is different from what I know about segmentation tasks.
"MonitoredTrainingSession" is different from Session.You can try "Supervisor".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants