Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorFlow record files are corrupted #26

Open
griffbad opened this issue Jul 14, 2020 · 0 comments
Open

TensorFlow record files are corrupted #26

griffbad opened this issue Jul 14, 2020 · 0 comments

Comments

@griffbad
Copy link

griffbad commented Jul 14, 2020

I'm trying to train an FFN, and the first 2 steps (partition and build the coordinate file) seem to go fine, but training throws Key Value errors. On further inspection (using TFRecord Viewer), I get this error:
tensorflow.python.framework.errors_impl.DataLossError: corrupted record at 0.

Any help would be super appreciated!

TF version 1.13.2, and here are the exact calls I'm making:

For computing partitions:
python ../../ffn/compute_partitions.py \ --input_volume ../training_data_img.h5:label \ --output_volume training_data2.h5:af \ --thresholds 0.025,0.05,0.075,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9 \ --lom_radius 3,3,3 \ --min_size 10000

For building the TFRecord file:
python ../../ffn/build_coordinates.py \ --partition_volumes validation:training_data2.h5:af \ --coordinate_output tf_record_fileyw \ --margin 3,3,3

and for training:
python ../../ffn/train.py \ --train_coords tf_record_fileyw \ --data_volumes validation1:../training_data_img.h5:image \ --label_volumes validation1:../training_data_img.h5:label \ --model_name convstack_3d.ConvStack3DFFNModel \ --model_args "{\"depth\": 2, \"fov_size\": [2, 2, 2], \"deltas\": [2, 2, 2]}" \ --image_mean 72 \ --image_stddev 33

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant