Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vectorize-pr #856

Open
wants to merge 88 commits into
base: branch-24.03
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 84 commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
e7f6efd
towards using vectorize
ipdemes Jan 26, 2023
02b5ffc
making simple test work on CPUs
ipdemes Feb 2, 2023
8b8e601
making simple GPU function work
ipdemes Feb 3, 2023
d330016
changing isort version in pre-commmit due to some errors in the older…
ipdemes Feb 3, 2023
6199a41
adding type stubs for numba
ipdemes Feb 4, 2023
23b80c5
clean-up
ipdemes Feb 7, 2023
3e07363
forcing partition by the first dimention for vectorize
ipdemes Feb 8, 2023
67b5675
adding caching to vectorize
ipdemes Feb 9, 2023
5b634fb
adding support for otypes
ipdemes Feb 11, 2023
844a29f
requiring exact instance creation for EVAL_UDF task
ipdemes Feb 11, 2023
7285498
fixing logic for generating CPU functions
ipdemes Feb 11, 2023
f1192e6
some clean-up
ipdemes Feb 14, 2023
e8b544f
adding logic for scalar arguments
ipdemes Feb 15, 2023
37a7281
refactoring cectorize kernel to compute point
ipdemes Feb 16, 2023
024e943
making GPU kernel work with sparse arrays
ipdemes Feb 17, 2023
24b16c3
some clean-up
ipdemes Feb 21, 2023
c76b2fe
hashing CUDA kernel for user function
ipdemes Feb 22, 2023
c8e8847
removing PTX from task arguments in the case the function was hashed
ipdemes Feb 22, 2023
392b4ee
fixing Torchswe test
ipdemes Feb 22, 2023
4278be6
fixing Torchswe test
ipdemes Feb 22, 2023
ff0782b
fixing Torchswe test
ipdemes Feb 23, 2023
c772522
adding debug output
ipdemes Feb 23, 2023
958a7b8
adding debug output
ipdemes Feb 23, 2023
1f35532
removing debug output
ipdemes Feb 23, 2023
63265c3
do not pass string if PTX is hashed
ipdemes Feb 24, 2023
798289f
split Cuda kerenel generation with kernel execution
ipdemes Feb 24, 2023
6dbc77a
Merge remote-tracking branch 'origin/branch-23.03' into vectorize
ipdemes Feb 27, 2023
591fe8e
fixing errors after merge
ipdemes Feb 27, 2023
7b56a5c
fixing logic for generating numba function
ipdemes Feb 28, 2023
57bc2b3
adding support for upper-case Letters in argument names in UDF
ipdemes Feb 28, 2023
cd6d025
fixing CUfunction caching logic
ipdemes Feb 28, 2023
c7b9a08
adding debug output
ipdemes Mar 1, 2023
295bf45
adding explicit omp version for createCuKernel
ipdemes Mar 1, 2023
97949bf
Merge remote-tracking branch 'origin/branch-23.03' into vectorize
ipdemes Mar 1, 2023
3f8a307
changing the way we store CUfunctions
ipdemes Mar 1, 2023
fd060d1
refactoring GPU caching
ipdemes Mar 1, 2023
8557a3d
Merge remote-tracking branch 'origin/branch-23.03' into vectorize
ipdemes Mar 2, 2023
113dee7
moving map to store CUfunctions to cudalibs.h
ipdemes Mar 2, 2023
074535e
Merge remote-tracking branch 'origin/branch-23.03' into vectorize
ipdemes Mar 2, 2023
42b8277
fixing dependency between creation and use of UDF function
ipdemes Mar 3, 2023
d8caf5f
fixing dependency between creation and use of UDF function
ipdemes Mar 3, 2023
cfdf8eb
futures do not add dependency between tasks
ipdemes Mar 3, 2023
8846242
some clean-up
ipdemes Mar 8, 2023
6e8975f
fixed dependency bug
ipdemes Mar 8, 2023
1c3ce9e
more clean-up
ipdemes Mar 8, 2023
5a5b061
removing created_array
ipdemes Mar 9, 2023
a80c128
fixing barrier
ipdemes Mar 9, 2023
e45d037
removing scipy
ipdemes Mar 9, 2023
30e01c1
Merge remote-tracking branch 'origin/branch-23.05' into vectorize
ipdemes Mar 10, 2023
dafbf1d
adding provenance
ipdemes Mar 10, 2023
9e09b8b
adding black_scholes example that uses vectorize
ipdemes Mar 15, 2023
9cd491a
removing special cases from black_scholes_greek
ipdemes Mar 15, 2023
0c2f9cd
cleaning up greeks test
ipdemes Mar 16, 2023
d6311ab
Merge remote-tracking branch 'origin/branch-23.05' into vectorize
ipdemes Mar 17, 2023
f6b8e67
making CPU UDF work with sparse arrays
ipdemes Mar 17, 2023
f3a704d
removed unused argument
ipdemes Mar 18, 2023
ac074fe
code clean-up and formatting
ipdemes Mar 20, 2023
99446de
some fixes for empty functions
ipdemes Mar 21, 2023
725e223
parsing python function for return arguments
ipdemes Mar 21, 2023
905fa95
removing dependency on six and some clean-up
ipdemes Mar 21, 2023
9bdefe9
updates for docs
bryevdv Mar 21, 2023
c0278f7
adding logic for parsing returns from UDF
ipdemes Mar 22, 2023
f6c5155
making vectorize to return arrays
ipdemes Mar 22, 2023
0924e4f
adding more tests + code clean-up
ipdemes Mar 23, 2023
00b39c4
fixing logic for caching
ipdemes Mar 23, 2023
e7e4e7a
adding more tests
ipdemes Mar 23, 2023
fe5219a
formatting
ipdemes Mar 23, 2023
7901dd8
Merge branch 'vectorize' of github.com:ipdemes/cunumeric into vectorize
ipdemes Mar 23, 2023
5b75297
small bugfux
ipdemes Mar 23, 2023
09b755f
clean-up + formatting
ipdemes Mar 24, 2023
22b217e
Merge remote-tracking branch 'origin/branch-23.05' into vectorize
ipdemes Mar 24, 2023
393aa1b
clean-up + formatting
ipdemes Mar 24, 2023
c8dd7fa
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 24, 2023
92170ff
Update cunumeric/vectorize.py
ipdemes Mar 27, 2023
a3196b4
addressing comments from Bryan
ipdemes Mar 27, 2023
fb7853e
making strings more readable
ipdemes Mar 28, 2023
f62ed37
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 28, 2023
2762123
replacing strings with f-strings
ipdemes Mar 28, 2023
d16c2b2
Merge branch 'vectorize' of github.com:ipdemes/cunumeric into vectorize
ipdemes Mar 28, 2023
0a0650c
arguments should be a copy of arrays to match the numpy logic
ipdemes Mar 28, 2023
98f15d6
adding missing type stubs for numpy
ipdemes Mar 28, 2023
8d118ca
checking return statememt of UDF for special characters
ipdemes Mar 30, 2023
889fb65
updating the map between pyarow types and legate types
ipdemes Mar 31, 2023
7df88a4
adding type stubs for pyarrow
ipdemes Mar 31, 2023
a65eea1
Update cunumeric/vectorize.py
ipdemes Apr 3, 2023
4dc584b
addressing comments from Bryan
ipdemes Apr 3, 2023
0446909
Merge remote-tracking branch 'origin/branch-23.07' into vectorize
ipdemes May 19, 2023
c123d63
fixed some typos
ipdemes May 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cunumeric/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
from cunumeric.logic import *
from cunumeric.window import bartlett, blackman, hamming, hanning, kaiser
from cunumeric.coverage import clone_module
from cunumeric.vectorize import vectorize

clone_module(_np, globals(), maybe_convert_to_np_ndarray)

Expand Down
2 changes: 1 addition & 1 deletion cunumeric/array.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@

import legate.core.types as ty
import numpy as np
import pyarrow # type: ignore [import]
import pyarrow
from legate.core import Array
from numpy.core.multiarray import ( # type: ignore [attr-defined]
normalize_axis_index,
Expand Down
4 changes: 4 additions & 0 deletions cunumeric/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,9 +143,11 @@ class _CunumericSharedLib:
CUNUMERIC_CONVERT_NAN_PROD: int
CUNUMERIC_CONVERT_NAN_SUM: int
CUNUMERIC_CONVOLVE: int
CUNUMERIC_CREATE_CU_KERNEL: int
CUNUMERIC_DIAG: int
CUNUMERIC_DOT: int
CUNUMERIC_EYE: int
CUNUMERIC_EVAL_UDF: int
CUNUMERIC_FFT: int
CUNUMERIC_FFT_C2C: int
CUNUMERIC_FFT_C2R: int
Expand Down Expand Up @@ -345,9 +347,11 @@ class CuNumericOpCode(IntEnum):
CONTRACT = _cunumeric.CUNUMERIC_CONTRACT
CONVERT = _cunumeric.CUNUMERIC_CONVERT
CONVOLVE = _cunumeric.CUNUMERIC_CONVOLVE
CREATE_CU_KERNEL = _cunumeric.CUNUMERIC_CREATE_CU_KERNEL
DIAG = _cunumeric.CUNUMERIC_DIAG
DOT = _cunumeric.CUNUMERIC_DOT
EYE = _cunumeric.CUNUMERIC_EYE
EVAL_UDF = _cunumeric.CUNUMERIC_EVAL_UDF
FFT = _cunumeric.CUNUMERIC_FFT
FILL = _cunumeric.CUNUMERIC_FILL
FLIP = _cunumeric.CUNUMERIC_FLIP
Expand Down
27 changes: 27 additions & 0 deletions cunumeric/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@

import legate.core.types as ty
import numpy as np
import pyarrow as pa

from .types import NdShape

Expand All @@ -43,6 +44,26 @@
np.float64: ty.float64,
np.complex64: ty.complex64,
np.complex128: ty.complex128,
complex: ty.complex128,
}

CUNUMERIC_TYPE_MAP = {
bool: ty.bool_,
int: ty.int64,
float: ty.float64,
complex: ty.complex128,
pa.bool_: ty.bool_,
pa.int8: ty.int8,
pa.int16: ty.int16,
pa.int32: ty.int32,
pa.int64: ty.int64, # np.int is int
pa.uint8: ty.uint8,
pa.uint16: ty.uint16,
pa.uint32: ty.uint32,
pa.uint64: ty.uint64, # np.uint is np.uint64
pa.float16: ty.float16,
pa.float32: ty.float32,
pa.float64: ty.float64,
Comment on lines +51 to +62
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pa.bool_: ty.bool_,
pa.int8: ty.int8,
pa.int16: ty.int16,
pa.int32: ty.int32,
pa.int64: ty.int64, # np.int is int
pa.uint8: ty.uint8,
pa.uint16: ty.uint16,
pa.uint32: ty.uint32,
pa.uint64: ty.uint64, # np.uint is np.uint64
pa.float16: ty.float16,
pa.float32: ty.float32,
pa.float64: ty.float64,

Python-level arithmetic values can only be bool/int/float/complex. You'll probably also need to remove the import pyarrow at the top.

}
manopapad marked this conversation as resolved.
Show resolved Hide resolved


Expand Down Expand Up @@ -99,6 +120,12 @@ def is_supported_dtype(dtype: Any) -> bool:
return dtype.type in SUPPORTED_DTYPES


def convert_to_cunumeric_dtype(dtype: Any) -> Any:
if dtype in CUNUMERIC_TYPE_MAP:
return CUNUMERIC_TYPE_MAP[dtype]
raise TypeError("dtype is not supported")
Comment on lines +117 to +120
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def convert_to_cunumeric_dtype(dtype: Any) -> Any:
if dtype in CUNUMERIC_TYPE_MAP:
return CUNUMERIC_TYPE_MAP[dtype]
raise TypeError("dtype is not supported")
def convert_to_cunumeric_dtype(dtype: type) -> ty.Dtype:
if dtype in CUNUMERIC_TYPE_MAP:
return CUNUMERIC_TYPE_MAP[dtype]
raise TypeError(f"{dtype} is not supported")



def calculate_volume(shape: NdShape) -> int:
if len(shape) == 0:
return 0
Expand Down
Loading