Add motivation to README

README.md

@@ -1,16 +1,61 @@
 # Boreholes Data Extraction
 
-Boreholes Data Extraction is a data extraction pipeline that extracts depth layers with their corresponding material description from borehole profiles in form of pdfs.
+Boreholes Data Extraction is a pipeline to extract structured data from borehole profiles in PDF files. Extracted properties (currently coordinates, depths and associated material descriptions) are returned in JSON format, and (optionally) visualized as PNG images. This project was initiated by the Swiss Federal Office of Topography [swisstopo](https://www.swisstopo.admin.ch/), and is developed with support from [Visium](https://www.visium.ch/).
 
-## Limitations
+## Motivation
 
-Note that the project is under active development and there is no release to this date, nor has the project reached a maturity such that it could be used.
+The Federal Office of Topography swisstopo is Switzerland's geoinformation centre. The Swiss Geological Survey at swisstopo is the federal competence centre for the collection, analysis, storage and provision of geological data of national interest. 
 
-The current extractions are focused on the depths of the upper and lower limits of each layer, on the material descriptions of the layers and the coordinates.
+Data from boreholes is an essential source for our knowledge about the subsurface. In order to manage and publish borehole data of national interest, swisstopo has developed the application boreholes.swissgeol.ch (currently for internal use only), part of the [swissgeol.ch](https://swissgeol.ch) platform. As of August 2024, over 30.000 boreholes are registered in the application database, a number that is rapidly increasing thanks to an improved data exchange with cantonal offices, other government agencies and federal corporations such as the Swiss Federal Railways SBB. In the coming years, the number of boreholes in the database is expected to keep increasing to a multiple of the current size. Data is being added from both boreholes that were recently constructued and documented, as well as from older boreholes that were until now only documented in separate databases or in analogue archives. Data from older boreholes can still be very relevant, as geology only changes very slowly, and newer data is often unavailable (and expensive to collect).
 
-The coordinate types LV95 as well as the older LV03 are supported. More information about the swiss coordinate systems [here](https://opendata.swiss/de/dataset/bezugsrahmenwechsel-lv03-lv95-koordinatenanderung-lv03-lv95) and [here](https://de.wikipedia.org/wiki/Schweizer_Landeskoordinaten).
+In order to use the collected borehole data efficiently, it is critical that both metadata as well as geological information is digitally stored in a structured database. However, the relevant data for most boreholes that are received by swisstopo, is contained in PDF-files that lack a standardized structure. Older data is often only available in the form of a scanned image, obtained from a printed document or from a microfiche. Manually entering all the relevant data from these various sources is not feasible, given the large amount of boreholes and the continuous influx of new data.
 
-Only German and French borehole profiles are supported as of now.
+Therefore, the goal of this project is to automate the extraction of structured data from borehole profiles as much as possible. As far as swisstopo is concerned, the use case is to integrate the data extraction pipeline with the application boreholes.swissgeol.ch ([Github](https://github.com/swisstopo/swissgeol-boreholes-suite)), where a user interface for efficient quality control of the automatically extracted data will also be implemented.
+
+All code and documentation is published in this Github repository as open source software. All other persons, companies or agencies who manage borehole data of their own, are welcome to use the data extraction pipeline on their own data and to contribute to the project with their own improvements/additions.
+
+### Extracted properties
+
+Below is a list of the most relevant properties for the extraction of structure data from borehole profiles. Properties that can be automatically extracted by the current pipeline are in **bold**. Properties for which the implementation of automatic extraction is actively being worked on, are in _italics_.
+
+* Metadata
+  * **Coordinates**
+  * _Date_
+  * _Drilling method_
+* Lithology / stratigraphy
+  * **Depths** (upper and lower bound of each layer)
+  * **Material descriptions** (as plain text)
+  * USCS classification, color, consistency, plasticity...
+  * Geological interpretations
+* Other
+  * _Hydrogeology (ground water levels)_
+  * Instrumentation
+  * Casing
+  * Borehole geometry
+  * ...
+
+
+### Related work
+
+Existing work related to this project is mostly focussed on the extraction and classification of specific properties from textual geological descriptions.  Notable examples include [GEOBERTje](https://www.arxiv.org/abs/2407.10991) (Belgium), [geo-ner-model](https://github.com/BritishGeologicalSurvey/geo-ner-model) (UK), [GeoVec](https://www.sciencedirect.com/science/article/pii/S0098300419306533) und [dh2loop](https://github.com/Loop3D/dh2loop) (Australia). The scope of this project is considerable wider, in particular regarding the goal of understanding borehole profiles in various languages and with an unknown layout, where the document structure first needs to be understood, before the relevant text fragments can be identified and extracted.
+
+The automatic data extraction pipeline can be considered to belong to the field or [automatic/intelligent document processing](https://en.wikipedia.org/wiki/Document_processing). As such, it involves a combination of methods from multiple fields in data science and machine learning, in particular computer vision (e.g. object detection, line detection) and natural language processing (large language models, named entity recognition). Some of these have already been implemented (e.g. the [Line Segment Detector](https://docs.opencv.org/3.4/db/d73/classcv_1_1LineSegmentDetector.html) algorithm), others are planned as future work.
+
+### Limitations
+
+The project is under active development and there is no release to this date. The quality/accuracy of the results may vary strongly depending on the documents that are used as input.
+
+The input PDF files must contain digital text content. For PDF files that are not _digitally-born_ (e.g. scanned documents), this means that OCR must be performed, and the OCR results stored in the PDF file, before using the file as an input for this data extraction pipeline. The quality of the extracted data is dependent on the quality of the OCR. At swisstopo, we use the [AWS Textract](https://aws.amazon.com/textract/) service together with our own code from the [swissgeol-ocr](https://github.com/swisstopo/swissgeol-ocr) repository for this purpose.
+
+The pipeline has been optimized for and tested on boreholes profiles from Switzerland that have been written in German or (to a more limited extent) in French.
+
+With regard to the extraction of coordinates, the [Swiss coordinate systems](https://de.wikipedia.org/wiki/Schweizer_Landeskoordinaten) LV95 as well as the older LV03 are supported ([visualization of the differences](https://opendata.swiss/de/dataset/bezugsrahmenwechsel-lv03-lv95-koordinatenanderung-lv03-lv95)).
+
+## Main contributors
+
+* Stijn Vermeeren [@stijnvermeeren-swisstopo](https://www.github.com/stijnvermeeren-swisstopo) (swisstopo) - Project Lead
+* David Cleres [@dcleres](https://www.github.com/dcleres) (Visium)
+* Renato Durrer [@redur](https://www.github.com/redur) (Visium)
 
 ## Installation
 We use pip to manage the packages dependencies. We recommend using a virtual environment within which to install all dependencies.