QPP-ELIG4350 add berkeley sc (#49)

* Add and Update missed documentation

* Update EUC files with Berkeley County SC

* Update tests for Berkeley County

* Update line endings to LF from CRLF

* Add crosswalk clarifying link

DEVELOPMENT.md

@@ -2,13 +2,20 @@ This page outlines the development guidelines for updating and publishing data f
 
 ## System Requirements
 
-- Python 3.9.x
+- Python 3.11.x
 
 ## Installation
 - `pipenv install --dev --ignore-pipfile` - will install all the libraries
 - `pipenv run pytest --junitxml=coverage/test-report.xml --cov-report html --cov-report xml --cov=processors tests/` - will execute the tests with coverage reports
 
+## Generating Crosswalk
+Once the staging steps below are completed run the following command to
+generate the [crosswalk](https://en.wikipedia.org/wiki/Schema_crosswalk) file
+```
+pipenv run python main.py -p generator -y <year>
+```
 
+## Overview
 Obtaining EUC zipcode crosswalk is a 4 stage process:
 1. Create a file containing county names and state from EUC policy (`euc_counties.csv`).
 2. Enrich each county with FIPS code. This information is obtained from National Counties from [Census.gov](https://www.census.gov/geographies/reference-files/time-series/geo/gazetteer-files.html) and by matching the county and state.
@@ -39,7 +46,10 @@ The example is from `2021\euc_counties.csv` which shows the structure :-
 |KY        |clay       |
 
 #### Common Gotchas
-- The fact sheet is often riddled with leading and trailing whitespace. These need to be stripped in the final `euc_counties.csv` file.
+- The fact sheet is often riddled with leading and trailing whitespace. These
+  need to be stripped in the final `euc_counties.csv` file.
+- Watch out for line endings on your files. They probably don't matter, but
+  some `CRLF`s have crept in.
 - Louisiana calls their county equivalents a `parish`.  The fact sheet may
 report just the parish name, but the Census data marks the areas as parishes.
 So you must manually add `parish` to any Louisiana 'counties', eg
@@ -70,3 +80,4 @@ The column definitions are:
 
 ### euc_counties_zip_crosswalk.csv
 This file in `data` folder contains the counties identified in EUC along with zipcodes.
+

README.md

@@ -2,16 +2,17 @@
 
 The automatic extreme and uncontrollable circumstances (EUC) policy identifies counties affected by natural disasters and public health emergencies.
 A fact sheet that summarizes policy and CMS designated counties for each performance year is available for download via [QPP resource library](https://qpp.cms.gov/resources/resource-library).
-There are a few downstream data processing systems within Quality Payment Program (QPP) that require county data enriched with zip codes. 
+There are a few downstream data processing systems within Quality Payment Program (QPP) that require county data enriched with zip codes.
 This repository publishes the county-zipcode crosswalk data in various machine-readable formats and the backoffice programs that generate them.
 
-Refer [Developer Guide](./DEVELOPMENT.md) for more information. 
+Refer [Developer Guide](./DEVELOPMENT.md) for more information.
 
 ## EUC Zipcode Crosswalk Publications
 | File                                                                                  | Details                                      |
 |---------------------------------------------------------------------------------------|----------------------------------------------|
 | [2022\euc_county_zipcode_crosswalk.csv](./data/2022/euc_county_zipcode_crosswalk.csv) | The 2022 EUC County Zipcode Crosswalk file.  |
 | [2023\euc_county_zipcode_crosswalk.csv](./data/2023/euc_county_zipcode_crosswalk.csv) | The 2023 EUC County Zipcode Crosswalk file.  |
+| [2024\euc_county_zipcode_crosswalk.csv](./data/2024/euc_county_zipcode_crosswalk.csv) | The 2024 EUC County Zipcode Crosswalk file.  |
 
 
 

data/2024/euc_county_zipcode_crosswalk.csv

@@ -4535,6 +4535,26 @@ SC,beaufort,29945
 SC,beaufort,29926
 SC,beaufort,29928
 SC,beaufort,29940
+SC,berkeley,29492
+SC,berkeley,29406
+SC,berkeley,29479
+SC,berkeley,29466
+SC,berkeley,29450
+SC,berkeley,29456
+SC,berkeley,29431
+SC,berkeley,29468
+SC,berkeley,29483
+SC,berkeley,29059
+SC,berkeley,29436
+SC,berkeley,29434
+SC,berkeley,29472
+SC,berkeley,29469
+SC,berkeley,29486
+SC,berkeley,29410
+SC,berkeley,29476
+SC,berkeley,29453
+SC,berkeley,29445
+SC,berkeley,29461
 SC,calhoun,29118
 SC,calhoun,29047
 SC,calhoun,29160

staging/2023/euc_counties.csv

@@ -1,109 +1,109 @@
-state_code,county_name
-FL,Alachua
-FL,Baker
-FL,Bay
-FL,Bradford
-FL,Brevard
-FL,Calhoun
-FL,Charlotte
-FL,Citrus
-FL,Clay
-FL,Collier
-FL,Columbia
-FL,DeSoto
-FL,Dixie
-FL,Duval
-FL,Flagler
-FL,Franklin
-FL,Gadsden
-FL,Gilchrist
-FL,Gulf
-FL,Hamilton
-FL,Hardee
-FL,Hernando
-FL,Hillsborough
-FL,Jefferson
-FL,Lafayette
-FL,Lake
-FL,Lee
-FL,Leon
-FL,Levy
-FL,Liberty
-FL,Madison
-FL,Manatee
-FL,Marion
-FL,Nassau
-FL,Orange
-FL,Osceola
-FL,Pasco
-FL,Pinellas
-FL,Polk
-FL,Putnam
-FL,Sarasota
-FL,Seminole
-FL,St. Johns
-FL,Sumter
-FL,Suwannee
-FL,Taylor
-FL,Union
-FL,Volusia
-FL,Wakulla
-GA,Appling
-GA,Atkinson
-GA,Bacon
-GA,Berrien
-GA,Brantley
-GA,Brooks
-GA,Bulloch
-GA,Burke
-GA,Camden
-GA,Candler
-GA,Charlton
-GA,Clinch
-GA,Coffee
-GA,Colquitt
-GA,Cook
-GA,Echols
-GA,Emanuel
-GA,Glynn
-GA,Jeff Davis
-GA,Jenkins
-GA,Lanier
-GA,Lowndes
-GA,Montgomery
-GA,Pierce
-GA,Screven
-GA,Tattnall
-GA,Thomas
-GA,Tift
-GA,Toombs
-GA,Treutlen
-GA,Ware
-GA,Wayne
-GU,Agana Heights
-GU,Agat
-GU,Asan
-GU,Barrigada
-GU,Chalan Pago
-GU,Dededo
-GU,Hagatna
-GU,Inarajan
-GU,Maite
-GU,Mangilao
-GU,Merizo
-GU,Ordot
-GU,Santa Rita
-GU,Sinajana
-GU,Talofofo
-GU,Tamuning
-GU,Yigo
-GU,Yona
-HI,Hawaii
-HI,Maui
-MS,Carroll
-MS,Humphreys
-MS,Monroe
-MS,Montgomery
-MS,Panola
-MS,Sharkey
+state_code,county_name
+FL,Alachua
+FL,Baker
+FL,Bay
+FL,Bradford
+FL,Brevard
+FL,Calhoun
+FL,Charlotte
+FL,Citrus
+FL,Clay
+FL,Collier
+FL,Columbia
+FL,DeSoto
+FL,Dixie
+FL,Duval
+FL,Flagler
+FL,Franklin
+FL,Gadsden
+FL,Gilchrist
+FL,Gulf
+FL,Hamilton
+FL,Hardee
+FL,Hernando
+FL,Hillsborough
+FL,Jefferson
+FL,Lafayette
+FL,Lake
+FL,Lee
+FL,Leon
+FL,Levy
+FL,Liberty
+FL,Madison
+FL,Manatee
+FL,Marion
+FL,Nassau
+FL,Orange
+FL,Osceola
+FL,Pasco
+FL,Pinellas
+FL,Polk
+FL,Putnam
+FL,Sarasota
+FL,Seminole
+FL,St. Johns
+FL,Sumter
+FL,Suwannee
+FL,Taylor
+FL,Union
+FL,Volusia
+FL,Wakulla
+GA,Appling
+GA,Atkinson
+GA,Bacon
+GA,Berrien
+GA,Brantley
+GA,Brooks
+GA,Bulloch
+GA,Burke
+GA,Camden
+GA,Candler
+GA,Charlton
+GA,Clinch
+GA,Coffee
+GA,Colquitt
+GA,Cook
+GA,Echols
+GA,Emanuel
+GA,Glynn
+GA,Jeff Davis
+GA,Jenkins
+GA,Lanier
+GA,Lowndes
+GA,Montgomery
+GA,Pierce
+GA,Screven
+GA,Tattnall
+GA,Thomas
+GA,Tift
+GA,Toombs
+GA,Treutlen
+GA,Ware
+GA,Wayne
+GU,Agana Heights
+GU,Agat
+GU,Asan
+GU,Barrigada
+GU,Chalan Pago
+GU,Dededo
+GU,Hagatna
+GU,Inarajan
+GU,Maite
+GU,Mangilao
+GU,Merizo
+GU,Ordot
+GU,Santa Rita
+GU,Sinajana
+GU,Talofofo
+GU,Tamuning
+GU,Yigo
+GU,Yona
+HI,Hawaii
+HI,Maui
+MS,Carroll
+MS,Humphreys
+MS,Monroe
+MS,Montgomery
+MS,Panola
+MS,Sharkey
 MS,Washington

staging/2024/README.md

@@ -15,6 +15,11 @@ To generate the `euc_county_zipcode_crosswalk.csv` file, you'll need the followi
   > This file is normally a `.pdf` file downloaded from the qpp documents page, but
   this year it was provided as a `.docx` emailed to Shane Dougherty.
 
+  > [!IMPORTANT]
+  > Berkeley County, SC was added as a valid county after initial data release.
+    A second data load was done to include this county.  However, the policy `.docx`
+    fact sheet does not include this Berkeley County.
+
 - **[National Counties Gazetteer File](./2024_Gaz_counties_national.txt): FIPS Codes**
 
   Enrich each county with its corresponding FIPS code. Obtain the National