Fix for issue with parsing CSVs with mixed newline delimiters #64
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
One line description of your change (less than 72 characters)
Adds the newline param to the config for papaparse during csv parsing
Problem
Issue described here
Basically if a file has the first line delimited by
\r\n
but the rest of the file is\n
, then papaparse has trouble inferring which delimieter to use.Solution
This PR uses the suggestion here from the papaparse maintainer to set the
newline
config param.This uses input from the user (provided from
hpt-validator-cli
) to addnewline
to the config. If nonewline
option is provided, it is not added to the config and papaparse should continue behaving as before (detecting the newline on its own)For files with mixed line endings, the papaparse maintainers say you can set
\n
as the newline delimiter and then subsequently strip out the\r
from the resulting rows, but I am not sure the latter part is necessary for this implementation.Result
Solves issue 29
Test Plan
(Write your test plan here. If you changed any code, please provide us with
clear instructions on how you verified your changes work.)
To verify change works:
https://msc.rochesterregional.org/practitionerPortal/docs/161012691_canton-potsdam-hospital_standardcharges.csv
that is linked in the above issueSteps taken to verify this change does not alter working validations:
To do for more confidence