Feat/improve is valid

[redur]

Improve is_valid criterion.

The noise check within the is_valid criterion is now adjusted to check
short depth columns (i.e. few entries) more strictly than longer depth columns (i.e. more entries).
This is achieved by applying a quadratic behavior onto the number of entries.

F1 improvement on Zurich dataset by 0.6%.
Similar F1 on geoquat dataset.

[redur]

We now support numbers such as .40 that sometimes occur.

[stijnvermeeren-swisstopo]

The input .40 is now extracted with value 40, not as 0.40. Is that really what we want?

I would also suggest adding this as a test case in test_find_depth_columns.py.

[redur]

I checked again, and actually it's rather that sometimes a '-' is recognized as a '.' in older borehole profiles that have this "handwritten style". Then the behavior is totally desired.

But I also found an occurrence of '.80'. See here: A531.pdf

For our dataset, it is for now better to use the current behaviour.

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
src/stratigraphy
__init__.py	8	1	88%	11
extract.py	210	210	0%	3–506
get_files.py	21	21	0%	3–48
line_detection.py	26	26	0%	3–76
main.py	91	91	0%	3–232
src/stratigraphy/util
coordinate_extraction.py	116	20	83%	25, 45, 49, 53, 57–65, 86, 171, 191, 280, 283–284, 288, 300
dataclasses.py	32	3	91%	37–39
depthcolumn.py	208	67	68%	26, 30, 51, 57, 60–61, 85, 88, 95, 102, 110–111, 121, 138–154, 199, 238, 254–262, 274, 279, 286, 313, 323, 352, 373, 376–387, 402–403, 448–490
depthcolumnentry.py	20	4	80%	12, 15, 27, 34
description_block_splitter.py	70	2	97%	24, 139
draw.py	73	73	0%	3–225
duplicate_detection.py	51	51	0%	3–146
find_depth_columns.py	89	6	93%	41–42, 70, 82, 175–176
find_description.py	63	28	56%	27–35, 50–63, 79–95, 172–175
geometric_line_utilities.py	86	2	98%	82, 132
interval.py	107	52	51%	25–28, 32–35, 40, 45, 48, 100–146, 167, 172–188
language_detection.py	18	18	0%	3–43
layer_identifier_column.py	91	91	0%	3–227
line.py	49	26	47%	25, 42, 51, 65–95, 98
linesquadtree.py	46	1	98%	76
plot_utils.py	43	43	0%	3–120
predictions.py	186	186	0%	3–386
textblock.py	74	8	89%	27, 51, 63, 75, 98, 119, 127, 155
util.py	40	22	45%	15–18, 22, 26, 40–47, 61–63, 87–88, 100–105
TOTAL	1818	1052	42%

Tests	Skipped	Failures	Errors	Time
58	0 💤	0 ❌	0 🔥	0.611s ⏱️

[redur]

This is bothering me for the moment. Right now you need to adjust the code to extend the extraction to other languages. This should be doable from the config files. I believe there are other places where language is hard-coded in form of keywords. (e.g. coordinate extraction)

I will open an issue for it.

[stijnvermeeren-swisstopo]

The input .40 is now extracted with value 40, not as 0.40. Is that really what we want?

I would also suggest adding this as a test case in test_find_depth_columns.py.