Skip to content

Commit

Permalink
Version 4.0.0 (#37)
Browse files Browse the repository at this point in the history 
* Alpha version with support for encrypted ZIP files

* Update README.md

* Base command to convert from an encrypted ZIP file

* Update CHANGELOG.md

* Update README.md

* Update README.md

* Alpha version with support for encrypted ZIP files

Added 7za logging + check return code

* Version 4.0.0 - Read README updated install + use

# New features and changes
- Added support to convert from an encrypted Huawei Health ZIP file. As of late october 2020, the Huawei Health app obligates to provide a password to encrypt the requested data with. You must provide this password in the new command line argument --password. The used encryption method also requires the installation of a 7-Zip stand-alone version to be able to decrypt the data. For more information, see the updated installation and conversion procedures in the README file.

# Known limitations
- Currently, conversion from an encrypted Huawei Health ZIP file is supported on Windows operating systems only.

* Update CHANGELOG.md
  • Loading branch information
@CTHRU
CTHRU authored Oct 30, 2020
1 parent c16effe commit b1b4095
Show file tree
Hide file tree
Showing 4 changed files with 118 additions and 41 deletions.
11 changes: 11 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,17 @@
All notable changes to this project are documented in this file.

## Release Notes
### Version 4.0.0 (build 2010.2902)
#### New features and changes
- Added support to convert from an encrypted Huawei Health ZIP file. As of late october 2020, the Huawei Health app
obligates to provide a password to encrypt the requested data with. You must provide this password in the new command
line argument --password. The used encryption method also requires the installation of a 7-Zip stand-alone version to be
able to decrypt the data. For more information, see the updated installation and conversion procedures in the README
file.

#### Known limitations
- Currently, conversion from an encrypted Huawei Health ZIP file is supported on Windows operating systems only.

### Version 3.6.1 (build 2009.1901)
#### Solved Issues
- Conversion stopped with an error when an empty swim activity is encountered (0 distance and no swim segments).
Expand Down
86 changes: 54 additions & 32 deletions Hitrava.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import operator
import os
import re
import subprocess
import sys
import tarfile
import tempfile
Expand Down Expand Up @@ -47,11 +48,11 @@

# Global Constants
PROGRAM_NAME = 'Hitrava'
PROGRAM_MAJOR_VERSION = '3'
PROGRAM_MINOR_VERSION = '6'
PROGRAM_PATCH_VERSION = '1'
PROGRAM_MAJOR_BUILD = '2009'
PROGRAM_MINOR_BUILD = '1901'
PROGRAM_MAJOR_VERSION = '4'
PROGRAM_MINOR_VERSION = '0'
PROGRAM_PATCH_VERSION = '0'
PROGRAM_MAJOR_BUILD = '2010'
PROGRAM_MINOR_BUILD = '2902'

OUTPUT_DIR = './output'
GPS_TIMEOUT = dts_delta(seconds=10)
Expand Down Expand Up @@ -1081,34 +1082,50 @@ def __del__(self):

class HiZip:
@staticmethod
def extract_json(zip_filename: str, output_dir: str = OUTPUT_DIR):
def extract_json(zip_filename: str, output_dir: str = OUTPUT_DIR, password: str = None):
_MOTION_PATH_JSON_FILENAME = 'data/Motion path detail data & description/motion path detail data.json'
_MOTION_PATH_JSON_FILENAME_ALT = 'Motion path detail data & description/motion path detail data.json'
if zipfile.is_zipfile(zip_filename):
with ZipFile(zip_filename, 'r') as hi_zip:
if _MOTION_PATH_JSON_FILENAME in hi_zip.namelist():
zip_json_filename = _MOTION_PATH_JSON_FILENAME
elif _MOTION_PATH_JSON_FILENAME_ALT in hi_zip.namelist():
zip_json_filename = _MOTION_PATH_JSON_FILENAME_ALT
else:
logging.getLogger(PROGRAM_NAME).warning('Could not find JSON file <%s> in ZIP file <%s>. \
Nothing to convert.',
_MOTION_PATH_JSON_FILENAME, zip_filename)
raise Exception('Could not find file <data/motion path detail data.json> in ZIP file <%s>. \
Nothing to convert.', zip_filename)
_UNZIP_CMD = '7za x -aoa "-o%s" -bb0 -bse0 -bsp2 "-p%s" -sccUTF-8 "%s" -- "%s"'

try:
hi_zip.extract(zip_json_filename, output_dir)
json_filename = output_dir + '/' + zip_json_filename
return json_filename
except Exception as e:
logging.getLogger(PROGRAM_NAME).error('Error extracting JSON file <%s> from ZIP file <%s>\n%s',
zip_json_filename, zip_filename, e)
raise Exception('Error extracting JSON file <%s> from ZIP file <%s>',
zip_json_filename, zip_filename)
if zipfile.is_zipfile(zip_filename):
if password is not None:
zip_json_filename = _MOTION_PATH_JSON_FILENAME_ALT
unzip_cmd = _UNZIP_CMD % (output_dir, password, zip_filename, zip_json_filename)
completed_process = subprocess.run(unzip_cmd,
universal_newlines=True,
stderr=subprocess.STDOUT,
stdout=subprocess.PIPE)
logging.getLogger(PROGRAM_NAME).info(completed_process.stdout)
if completed_process.returncode != 0:
logging.getLogger(PROGRAM_NAME).error('Error extracting JSON file <%s> from encrypted ZIP file <%s>. Return code was %s',
zip_json_filename, zip_filename, completed_process.returncode)
raise Exception('Error extracting JSON file <%s> from encrypted ZIP file <%s>. Return code was %s',
zip_json_filename, zip_filename, completed_process.returncode)
else:
with ZipFile(zip_filename, 'r', True) as hi_zip:
if _MOTION_PATH_JSON_FILENAME in hi_zip.namelist():
zip_json_filename = _MOTION_PATH_JSON_FILENAME
elif _MOTION_PATH_JSON_FILENAME_ALT in hi_zip.namelist():
zip_json_filename = _MOTION_PATH_JSON_FILENAME_ALT
else:
logging.getLogger(PROGRAM_NAME).warning('Could not find JSON file <%s> in ZIP file <%s>. \
Nothing to convert.',
_MOTION_PATH_JSON_FILENAME, zip_filename)
raise Exception('Could not find file <data/motion path detail data.json> in ZIP file <%s>. \
Nothing to convert.', zip_filename)

try:
hi_zip.extract(zip_json_filename, output_dir)
except Exception as e:
logging.getLogger(PROGRAM_NAME).error('Error extracting JSON file <%s> from ZIP file <%s>\n%s',
zip_json_filename, zip_filename, e)
raise Exception('Error extracting JSON file <%s> from ZIP file <%s>',
zip_json_filename, zip_filename)
json_filename = output_dir + '/' + zip_json_filename
return json_filename
else:
logging.getLogger(PROGRAM_NAME).error('Invalid ZIP file <%s>', zip_filename)
raise Exception('Invalid ZIP file <%s>', zip_filename)
logging.getLogger(PROGRAM_NAME).error('Invalid ZIP file or ZIP file not found <%s>', zip_filename)
raise Exception('Invalid ZIP file or ZIP file not found <%s>', zip_filename)


class HiJson:
Expand Down Expand Up @@ -1837,6 +1854,9 @@ def _init_argument_parser() -> argparse.ArgumentParser:
The JSON file will be extracted to the directory in the --output_dir \
argument and conversion will be performed.')

json_group.add_argument('-p', '--password', help='The password of the encrypted Huawei Cloud ZIP file. \
Required for encrypted ZIP files only.')

json_group.add_argument('-j', '--json', help='The filename of a Huawei Cloud JSON file containing the motion path \
detail data to convert or the filename of the Huawei Cloud ZIP file \
containing the JSON file with the motion path detail data (this will \
Expand Down Expand Up @@ -1938,14 +1958,16 @@ def main():
else:
_init_logging()

args_string = str(sys.argv[1:])
args_string = re.sub("('--password', '\w*')|('-p', '\w*')", "'--password', '********'", args_string)
logging.getLogger(PROGRAM_NAME).info("%s version %s.%s.%s (build %s.%s) started with arguments %s",
PROGRAM_NAME,
PROGRAM_MAJOR_VERSION,
PROGRAM_MINOR_VERSION,
PROGRAM_PATCH_VERSION,
PROGRAM_MAJOR_BUILD,
PROGRAM_MINOR_BUILD,
str(sys.argv[1:]))
args_string)
logging.getLogger(PROGRAM_NAME).info("Running on Python version %s.%s.%s",
sys.version_info[0],
sys.version_info[1],
Expand Down Expand Up @@ -1998,9 +2020,9 @@ def main():
logging.getLogger(PROGRAM_NAME).info('Converted %s', hi_activity)
elif args.json or args.zip:
if args.zip:
json_filename = HiZip.extract_json(args.zip, args.output_dir)
json_filename = HiZip.extract_json(args.zip, args.output_dir, args.password)
elif args.json and zipfile.is_zipfile(args.json):
json_filename = HiZip.extract_json(args.json, args.output_dir)
json_filename = HiZip.extract_json(args.json, args.output_dir, args.password)
else:
json_filename = args.json
hi_json = HiJson(json_filename, args.output_dir, args.json_export)
Expand Down
60 changes: 51 additions & 9 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ learn more on [https://cthru.hopto.org](https://cthru.hopto.org/hitrava-web).

## Features
- Recognizes and converts the following activity types from Huawei Health to Strava:
- Running: both outdoor and indoor running (threadmill run)
- Running: both outdoor and indoor running (treadmill run)
- Cycling: both outdoor and indoor cycling
- Swimming: both pool swimming and open water swimming
- Walking
Expand All @@ -62,6 +62,8 @@ To use Hitrava, you need:
- Python 3.7.6 is the lowest recommended version (developed and tested on this version).
- Python 3.5.1 is the lowest minimum required version (compatibility tested on this version).
- A Huawei account to request your health data.
- 7-Zip stand-alone version to convert directly from an encrypted Huawei Health ZIP file. Currently, this method is only
supported on Windows operating systems.

### Installation Procedure
#### Step 1 - Install Python
Expand All @@ -76,12 +78,28 @@ Only required if you don't have a (suitable) python installation on your system.
- Save the ZIP file with the sources on your system.
- Extract all contents of the ZIP file with the sources to a location of your choice on your system.

#### Step 3 - Download and Extract Stand-alone 7-Zip
NOTE: This step is required to convert **encrypted** Huawei Health ZIP files.

- Download the latest 7-zip **stand-alone** console version from the [`7-Zip website`](https://www.7-zip.org/download.html).
The stand-alone version can be identified by the description below:
> 7-Zip Extra: standalone console version, 7z DLL, Plugin for Far Manager
- Extract the file _7za.exe_ from the downloaded archive and place it in your Hitrava installation folder from step 2
above. Your Hitrava installation folder should now contain at least the following files.
> Hitrava.py
> 7za.exe
> Run_Hitrava_Decrypt.cmd
## How to convert your health activities and import them in Strava
All users can use conversion from a **[ZIP](#ZIP-conversion-procedure)** file or a **[JSON](#JSON-file-conversion-example)** file.
All users can use conversion from a **[ZIP](#Encrypted-ZIP-conversion-procedure)** file or a **[JSON](#JSON-file-conversion-example)** file.
For users with rooted phones, legacy **[file](#single-file-conversion-examples)** and
**[tar](#tar-file-conversion-examples)** options are still available.

### ZIP conversion procedure
### Encrypted ZIP conversion procedure
NOTE: As of late October 2020, the latest version of the Huawei Health app obliges you to provide a password with which
your data in the ZIP file will be encrypted. If you need to convert from an older non-encrypted ZIP file, please refer
to the **[corresponding example](ZIP-file-conversion-example)**.

Activities can be mass converted using the data in a ZIP file that you can request in the Huawei Health app.

The procedure below assumes that you [installed Hitrava](#installation) and are logged in with your Huawei account in
Expand All @@ -93,7 +111,10 @@ the Huawei Health app. If you don't have a Huawei account, you can create one in
- Now tap on your account name on top of the screen.
- Tap on **'Privacy Center'**.
- Tap **'Request Your Data'**.
- Select **'Health'** from the list. Confirm your selection and follow the in-app instructions.
- Select **'Health'** from the list and confirm your selection.
- You will be obligated to enter a password to encrypt the requested data with. You will need this password later in
step 3 below.
- Follow any further in-app instructions.
- Wait for the mail from Huawei to arrive with a link to download the data (ZIP file).

#### Step 2 - Download your requested data
Expand All @@ -103,15 +124,20 @@ the Huawei Health app. If you don't have a Huawei account, you can create one in
#### Step 3 - Convert the data with Hitrava

>**Tip**: If you're on Windows and you're not familiar with the Command Prompt or just want to do a quick
> conversion with default arguments, double-click the _Run_Hitrava.cmd_ file in the installation folder of Hitrava.
> conversion with default arguments, you can use the _Run_Hitrava_Decrypt.cmd_ batch file.
>- Open the _Run_Hitrava_Decrypt.cmd_ file with a text editor and change the password 123456 to the password you
>provided in step 2 above.
>- double-click the _Run_Hitrava_Decrypt.cmd_ file in the installation folder of Hitrava.
> This will convert all available activities in the ZIP file from the previous step.
- Open a Command Prompt (Windows) or Terminal (Linux / Mac OS) and change the directory to the installation folder of
Hitrava.
- In the Command Prompt, run Hitrava.py with the --zip command line argument. You can use the default example
command below or [add / change command line arguments](#command-line-arguments-overview) as you need.
- In the Command Prompt, run Hitrava.py with the --zip command line argument. You can start from the default example
command below or [add / change command line arguments](#command-line-arguments-overview) as you need.

**IMPORTANT**: You must replace the password 123456 with the password you provided in step 2 above.
```
Hitrava.py --zip HiZip.zip --json_export
Hitrava.py --zip HiZip.zip --password 123456 --json_export
```
The above command will generate both the original HiTrack files and the converted TCX files for ALL activities to the
_output_ subfolder of the Hitrava installation folder. In this folder:
Expand All @@ -129,7 +155,7 @@ extension, up to 25 at once) to upload.
## Usage
### Command Line Arguments Overview
```
usage: Hitrava.py [-h] [-z ZIP] [-j JSON] [--json_export] [-f FILE]
usage: Hitrava.py [-h] [-z ZIP] [-p PASSWORD] [-j JSON] [--json_export] [-f FILE]
[-s {Walk,Run,Cycle,Swim_Pool,Swim_Open_Water}] [-t TAR]
[--from_date FROM_DATE] [--pool_length POOL_LENGTH]
[--tcx_insert_altitude_data] [--output_dir OUTPUT_DIR]
Expand All @@ -149,6 +175,9 @@ JSON options:
convert. The JSON file will be extracted to the
directory in the --output_dir argument and conversion
will be performed.
-p PASSWORD, --password PASSWORD
The password of the encrypted Huawei Cloud ZIP file.
Required for encrypted ZIP files only.
-j JSON, --json JSON The filename of a Huawei Cloud JSON file containing
the motion path detail data to convert or the filename
of the Huawei Cloud ZIP file containing the JSON file
Expand Down Expand Up @@ -216,6 +245,19 @@ OUTPUT options:
```
### Usage Examples
#### Encrypted ZIP file conversion example
Use the command below to convert all activities available in the **encrypted** ZIP file with the Huawei
Privacy data (here with filename _HiZip.zip_) that were started on October, 3rd, 2019 or later.
The ZIP file was encrypted with password 123456 provided in the Huawei Health app.
The following files will be generated in folder _./my_output_dir_:
- JSON files with the raw JSON data of a single activity (_.json_ file extension).
- HiTrack files with the unconverted source data of a single activity (no extension, filenames start with _HiTrack__).
- Converted TCX files for upload to Strava (_.tcx_ file extension).

```
python Hitrava.py --zip HiZip.zip --password 123456 --json_export --from_date 2019-10-03 --output_dir my_output_dir
```

#### ZIP file conversion example
Use the command below to convert all activities available in the ZIP file with the Huawei
Privacy data (here with filename _HiZip.zip_) that were started on October, 3rd, 2019 or later. The following files will
Expand Down
2 changes: 2 additions & 0 deletions Run_Hitrava_Decrypt.cmd
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Hitrava.py --zip HiZip.zip --password 123456 --json_export
pause

0 comments on commit b1b4095

Please sign in to comment.