Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add float16 support with configurable conversion behavior #49

Merged
merged 2 commits into from
Jan 13, 2025

Conversation

eek
Copy link
Contributor

@eek eek commented Jan 6, 2025

This PR adds support for NumPy's float16 data type with flexible handling options:

Key Features:

  • Add float16 dtype support using Uint16Array
  • Implement automatic float16 to float32 conversion
  • Add convertFloat16 constructor option for configurable behavior
    • true (default): auto-convert to float32 for ease of use
    • false: keep raw uint16 values for custom handling

Technical Details:

  • Add new dtype "<f2" for float16 support
  • Implement conversion utilities:
    • float16ToFloat32Array for array conversion
    • float16ToFloat32 for single value conversion
  • Update TypeScript definitions for float16 handling
  • Add comprehensive test suite for float16 functionality

Infrastructure Improvements:

  • Enhance test data generation and consistency
  • Improve git diff visibility for test records

also updated the README

eek added 2 commits January 6, 2025 00:54
- Introduced a new dtype "<f2" for float16 with size 16 and Uint16Array as its array constructor.
- Implemented `float16ToFloat32Array` method to convert Uint16Array (float16) to Float32Array.
- Added static method `float16ToFloat32` for converting individual float16 values to float32.
- Updated the `Parsed` type to handle float16 conversion when necessary.

This update improves the handling of float16 data types in the npyjs library.
- Add `convertFloat16` constructor option to control float16 conversion behavior
- Fix test data generation script to use correct paths
- Update records.json format for better git diff visibility
- Fix float16 test timeout issues
- Ensure test data consistency between runs

The `convertFloat16` option (defaults to true) allows users to:
- true: automatically convert float16 to float32 (default)
- false: keep raw uint16 values for custom handling
@eek
Copy link
Contributor Author

eek commented Jan 6, 2025

had an issue with reading float16 last night and did a fix without checking if there was already another PR done (#44 & #45).

I've also needed my values to be converted not be opaque so this PR includes (by default and via an option the automatically conversion to float32 arrays)

cc @j6k4m8

@j6k4m8
Copy link
Member

j6k4m8 commented Jan 13, 2025

This is great — LOVE that it can be turned off configurably. Pulling down to test, and then I'll merge and deploy!

@j6k4m8 j6k4m8 merged commit f03b25c into aplbrain:master Jan 13, 2025
1 check passed
@j6k4m8
Copy link
Member

j6k4m8 commented Jan 13, 2025

Thank you again @eek — this is now on npm at v0.7.0!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants