Feat/include new profiles

[redur]

Added new set of borehole profiles, and with it, multilingual support.

• Adjusted GroundTruth to work with new format. --> old format will not work any more. There is a functionality in the notebook that converts old ground_truth.json into the new one.
• Added language detection.
• Translated including and excluding expression into french.
• Also comes with the EDA based on a different branch. Probably should be dropped here. Kept for the conversion script and the generation of test/train/eval splits. Can be dropped after the review.

[stijnvermeeren-swisstopo]

I would indeed remove the notebook and not merge it into the main code base.

What's the easiest way for us to share our train/validation/test splits? Do you recommend that I execute the notebook myself, using the exact same random seed, or is there a more convenient way?

[stijnvermeeren-swisstopo]

Does this keep accented letter such as öäüéàè that might be important for detecting German/French, or only ASCII?

[redur]

Good question. I'll have to check.

[redur]

öäüéàè and the like are kept.

[redur]

I would indeed remove the notebook and not merge it into the main code base.

What's the easiest way for us to share our train/validation/test splits? Do you recommend that I execute the notebook myself, using the exact same random seed, or is there a more convenient way?

I would upload them to a bucket and source data from there directly.

Other from that, it's probably good to execute the notebook with the same seed.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
src/stratigraphy
extract.py	189	189	0%	3–460
line_detection.py	29	29	0%	3–88
main.py	78	78	0%	3–174
src/stratigraphy/util
dataclasses.py	30	3	90%	34–36
depthcolumn.py	204	67	67%	26, 30, 51, 57, 60–61, 85, 88, 95, 102, 110–111, 121, 138–154, 187, 210, 226–234, 244, 249, 256, 263, 268, 286, 296, 299–306, 321–322, 365–407
depthcolumnentry.py	20	4	80%	12, 15, 27, 34
description_block_splitter.py	70	2	97%	24, 139
draw.py	73	73	0%	3–199
find_depth_columns.py	82	4	95%	57–58, 149–150
find_description.py	39	6	85%	27–34, 111–114
geometric_line_utilities.py	123	37	70%	74–88, 111–115, 214–237, 261, 311
interval.py	101	52	49%	24–27, 31–34, 39, 44, 47, 90–136, 156, 161–177
language_detection.py	18	18	0%	3–43
line.py	49	26	47%	25, 42, 51, 65–95, 98
plot_utils.py	44	44	0%	3–111
textblock.py	68	7	90%	31, 43, 55, 78, 99, 107, 135
util.py	40	22	45%	15–18, 22, 26, 40–47, 61–63, 87–88, 100–105
TOTAL	1262	661	48%

Tests	Skipped	Failures	Errors	Time
45	0 💤	0 ❌	0 🔥	1.027s ⏱️

[redur]

TBD: Remove Notebook and create scripts for the data generation.

[stijnvermeeren-swisstopo]

LGTM.

For now, it's good to have the convert_ground_script here for reproducibility, but in future we can probable remove that script again, as we won't need to deal with the old format at all anymore?

[redur]

LGTM.

For now, it's good to have the convert_ground_script here for reproducibility, but in future we can probable remove that script again, as we won't need to deal with the old format at all anymore?

I agree with this statement.