diff --git a/README.md b/README.md index 1c66181801..e7c498d595 100644 --- a/README.md +++ b/README.md @@ -60,31 +60,37 @@ Completed items are ☑check-marked. See [closed PRs](https://github.com/scikit- * [X] Fully implement `__getitem__` for int/slice/intarray/boolarray/tuple (placeholders for newaxis/ellipsis), with perfect agreement with [Numpy basic/advanced indexing](https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html), to all levels of depth. * [ ] Appendable arrays (a distinct phase from readable arrays, when the type is still in flux) to implement `awkward.fromiter` in C++. * [X] Implemented all types but records; tested all primitives and lists. + * [ ] Expose appendable arrays to Numba. * [ ] Implement appendable records. * [ ] Test all (requires array types for all). * [X] JSON → Awkward via header-only [RapidJSON](https://rapidjson.org) and `awkward.fromiter`. * [ ] Explicit broadcasting functions for jagged and non-jagged arrays and scalars. + * [ ] Structure-preserving ufunc-like operation on the C++ side that applies a lambda function to inner data. The Python `__array_ufunc__` implementation will _call_ this to preserve structure. * [ ] Extend `__getitem__` to take jagged arrays of integers and booleans (same behavior as old). * [ ] Full suite of array types: - * [ ] `EmptyArray`: 1-dimensional array with length 0 and unknown type (result of `UnknownFillable`, compatible with all types of arrays). + * [X] `EmptyArray`: 1-dimensional array with length 0 and unknown type (result of `UnknownFillable`, compatible with all types of arrays). * [X] `RawArray`: flat, 1-dimensional array type for pure C++ (header-only). * [X] `NumpyArray`: rectilinear, N-dimensional array type without Python/pybind11 dependencies, but intended for Numpy. * [X] `ListArray`: the new `JaggedArray`, based on `starts` and `stops` (i.e. fully general). * [X] `ListOffsetArray`: the `JaggedArray` case with no unreachable data between reachable data (gaps). - * [ ] `RecordArray`: the new `Table` _without_ lazy-slicing. * [ ] `RegularArray`: rectilinear, N-dimensional array of arbitrary contents, for putting jagged dimensions inside fixed dimensions (for example). - * [ ] `ChunkedArray`: same as the old version, except that the type is a union if chunks conflict, not an error, and knowledge of all chunk sizes is always required. (Maybe `AmorphousChunkedArray` would fill that role.) - * [ ] `RegularChunkedArray`: like a `ChunkedArray`, but all chunks are known to have the same size. + * [ ] `RecordArray`: the new `Table` _without_ lazy-slicing. * [ ] `MaskedArray`, `BitMaskedArray`, `IndexedMaskedArray`: same as the old versions. * [ ] `UnionArray`: same as the old version; `SparseUnionArray`: the additional case found in Apache Arrow. - * [ ] `SlicedArray`: lazy-slicing (from old `Table`) that can be applied to any type. * [ ] `IndexedArray`: same as the old version. + * [ ] `RedirectArray`: an explicit weak-reference to another part of the structure (no hard-linked cycles). Often used with an `IndexedArray`. + * [ ] `SlicedArray`: lazy-slicing (from old `Table`) that can be applied to any type. * [ ] `SparseArray`: same as the old version. + * [ ] `ChunkedArray`: same as the old version, except that the type is a union if chunks conflict, not an error, and knowledge of all chunk sizes is always required. (Maybe `AmorphousChunkedArray` would fill that role.) + * [ ] `RegularChunkedArray`: like a `ChunkedArray`, but all chunks are known to have the same size. * [ ] `VirtualArray`: same as the old version, including caching, but taking C++11 lambda functions for materialization, get-cache, and put-cache. The pybind11 layer will connect this to Python callables. - * [ ] `ObjectArray`: same as the old version, but taking a C++11 lambda function to produce its output. The pybind11 layer will connect this to Python callables. - * [ ] Describe high-level types using [datashape](https://datashape.readthedocs.io/en/latest/) and possibly also an in-house schema. (Emit datashape _strings_ from C++.) + * [ ] Derived classes with ufunc-defined `Methods` and Numba extensions: + * [ ] `StringArray`: a `ListArray`/`ListOffsetArray` of characters with special methods and an optional encoding. + * [ ] `PyVirtualArray`: takes a Python lambda (which gets carried into `VirtualArray`). + * [ ] `PyObjectArray`: same as the old version. + * [X] Describe high-level types using [datashape](https://datashape.readthedocs.io/en/latest/) and possibly also an in-house schema. (Emit datashape _strings_ from C++.) * [ ] Describe mid-level "persistence types" with no lengths, somewhat minimal JSON, optional dtypes/compression. - * [ ] Describe low-level layouts independently of filled arrays? + * [ ] Describe low-level layouts independently of filled arrays (JSON or something)? * [ ] Layer 1 interface `Array`: * [ ] Pass through to the layout classes in Python and Numba. * [ ] Pass through Numpy ufuncs using [NEP 13](https://www.numpy.org/neps/nep-0013-ufunc-overrides.html) (as before). @@ -94,7 +100,9 @@ Completed items are ☑check-marked. See [closed PRs](https://github.com/scikit- * [ ] Mechanism for adding user-defined `Methods` like `LorentzVector`, as before, but only on Layer 1. * [ ] Inerhit from Pandas so that all Layer 1 arrays can be DataFrame columns. * [ ] Full suite of operations: - * [X] `awkward.tolist`: invokes iterators to convert arrays to lists and dicts. + * [X] `awkward.tolist`: same as before. + * [X] `awkward.fromiter`: same as before. + * [X] `awkward.typeof`: reports the high-level type (accepting some non-awkward objects). * [ ] `awkward.tonumpy`: to force conversion to Numpy, if possible. Neither Layer 1 nor Layer 2 will have an `__array__` method; in the Numpy sense, they are not "array-like" or "array-compatible." * [ ] `awkward.topandas`: flattening jaggedness into `MultiIndex` rows and nested records into `MultiIndex` columns. This is distinct from the arrays' inheritance from Pandas, distinct from the natural ability to use any one of them as DataFrame columns. * [ ] `awkward.flatten`: same as old with an `axis` parameter. @@ -110,7 +118,6 @@ Completed items are ☑check-marked. See [closed PRs](https://github.com/scikit- * [ ] `awkward.choose` (and `awkward.argchoose`): to make combinations by choosing a fixed number from a single array; option to use `Identity` index and an option to include same-object combinations. * [ ] `awkward.join`: performs an inner join of multiple arrays; requires `Identity`. Because the `Identity` is a surrogate index, this is effectively a per-event intersection, zipping all fields. * [ ] `awkward.union`: performs an outer join of multiple arrays; requires `Identity`. Because the `Identity` is a surrogate index, this is effectively a per-event union, zipping fields where possible. - * [ ] Derived classes section with `StringArray` as its first member. Derived classes have ufunc-defined `Methods` and Numba extensions. ### Soon after (possibly within) the six-month timeframe diff --git a/VERSION_INFO b/VERSION_INFO index baa9837854..790629964e 100644 --- a/VERSION_INFO +++ b/VERSION_INFO @@ -1 +1 @@ -0.1.20 +0.1.21 diff --git a/awkward1/__init__.py b/awkward1/__init__.py index 80f4ecee1a..cd29b5cca5 100644 --- a/awkward1/__init__.py +++ b/awkward1/__init__.py @@ -4,5 +4,6 @@ import awkward1._numba from awkward1.operations.convert import * +from awkward1.operations.describe import * __version__ = awkward1.layout.__version__ diff --git a/awkward1/_numba/__init__.py b/awkward1/_numba/__init__.py index ab50f5cb90..73bd78400c 100644 --- a/awkward1/_numba/__init__.py +++ b/awkward1/_numba/__init__.py @@ -14,3 +14,4 @@ import awkward1._numba.array.numpyarray import awkward1._numba.array.listarray import awkward1._numba.array.listoffsetarray + import awkward1._numba.array.empty diff --git a/awkward1/_numba/array/empty.py b/awkward1/_numba/array/empty.py new file mode 100644 index 0000000000..fe1b4060b6 --- /dev/null +++ b/awkward1/_numba/array/empty.py @@ -0,0 +1,143 @@ +# BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +import operator + +import numpy +import numba + +import awkward1.layout +from ..._numba import cpu, util, content + +@numba.extending.typeof_impl.register(awkward1.layout.EmptyArray) +def typeof(val, c): + return EmptyArrayType(numba.typeof(val.id)) + +class EmptyArrayType(content.ContentType): + def __init__(self, idtpe): + super(EmptyArrayType, self).__init__(name="EmptyArrayType(id={})".format(idtpe.name)) + self.idtpe = idtpe + + @property + def ndim(self): + return 1 + + def getitem_int(self): + raise ValueError("cannot compile getitem for EmptyArray, which has unknown element type") + + def getitem_range(self): + return self + + def getitem_tuple(self, wheretpe): + if len(wheretpe.types) == 0: + return self + elif len(wheretpe.types) == 1 and isinstance(wheretpe.types[0], numba.types.SliceType): + return self + else: + raise ValueError("cannot compile getitem for EmptyArray, which has unknown element type") + + def getitem_next(self, wheretpe, isadvanced): + if len(wheretpe.types) == 0: + return self + else: + raise ValueError("cannot compile getitem for EmptyArray, which has unknown element type") + + def carry(self): + return self + + @property + def lower_len(self): + return lower_len + + @property + def lower_getitem_range(self): + return lower_getitem_range + + @property + def lower_getitem_next(self): + return lower_getitem_next + + @property + def lower_carry(self): + return lower_carry + +@numba.extending.register_model(EmptyArrayType) +class EmptyArrayModel(numba.datamodel.models.StructModel): + def __init__(self, dmm, fe_type): + members = [] + if fe_type.idtpe != numba.none: + members.append(("id", fe_type.idtpe)) + super(EmptyArrayModel, self).__init__(dmm, fe_type, members) + +@numba.extending.unbox(EmptyArrayType) +def unbox(tpe, obj, c): + proxyout = numba.cgutils.create_struct_proxy(tpe)(c.context, c.builder) + if tpe.idtpe != numba.none: + id_obj = c.pyapi.obj_getattr_string(obj, "id") + proxyout.id = c.pyapi.to_native_value(tpe.idtpe, id_obj).value + c.pyapi.decref(id_obj) + is_error = numba.cgutils.is_not_null(c.builder, c.pyapi.err_occurred()) + return numba.extending.NativeValue(proxyout._getvalue(), is_error) + +@numba.extending.box(EmptyArrayType) +def box(tpe, val, c): + EmptyArray_obj = c.pyapi.unserialize(c.pyapi.serialize_object(awkward1.layout.EmptyArray)) + proxyin = numba.cgutils.create_struct_proxy(tpe)(c.context, c.builder, value=val) + if tpe.idtpe != numba.none: + id_obj = c.pyapi.from_native_value(tpe.idtpe, proxyin.id, c.env_manager) + out = c.pyapi.call_function_objargs(EmptyArray_obj, (id_obj,)) + c.pyapi.decref(id_obj) + else: + out = c.pyapi.call_function_objargs(EmptyArray_obj, ()) + c.pyapi.decref(EmptyArray_obj) + return out + +@numba.extending.lower_builtin(len, EmptyArrayType) +def lower_len(context, builder, sig, args): + return context.get_constant(numba.intp, 0) + +@numba.extending.lower_builtin(operator.getitem, EmptyArrayType, numba.types.slice2_type) +def lower_getitem_range(context, builder, sig, args): + rettpe, (tpe, wheretpe) = sig.return_type, sig.args + val, whereval = args + if context.enable_nrt: + context.nrt.incref(builder, rettpe, val) + return val + +@numba.extending.lower_builtin(operator.getitem, EmptyArrayType, numba.types.BaseTuple) +def lower_getitem_tuple(context, builder, sig, args): + rettpe, (tpe, wheretpe) = sig.return_type, sig.args + val, whereval = args + if context.enable_nrt: + context.nrt.incref(builder, rettpe, val) + return val + +def lower_getitem_next(context, builder, arraytpe, wheretpe, arrayval, whereval, advanced): + if context.enable_nrt: + context.nrt.incref(builder, arraytpe, arrayval) + return arrayval + +def lower_carry(context, builder, arraytpe, carrytpe, arrayval, carryval): + if context.enable_nrt: + context.nrt.incref(builder, arraytpe, arrayval) + return arrayval + +@numba.typing.templates.infer_getattr +class type_methods(numba.typing.templates.AttributeTemplate): + key = EmptyArrayType + + def generic_resolve(self, tpe, attr): + if attr == "id": + if tpe.idtpe == numba.none: + return numba.optional(identity.IdentityType(numba.int32[:, :])) + else: + return tpe.idtpe + +@numba.extending.lower_getattr(EmptyArrayType, "id") +def lower_id(context, builder, tpe, val): + proxyin = numba.cgutils.create_struct_proxy(tpe)(context, builder, value=val) + if tpe.idtpe == numba.none: + return context.make_optional_none(builder, identity.IdentityType(numba.int32[:, :])) + else: + if context.enable_nrt: + context.nrt.incref(builder, tpe.idtpe, proxyin.id) + return proxyin.id diff --git a/awkward1/operations/describe.py b/awkward1/operations/describe.py new file mode 100644 index 0000000000..010b0efbbe --- /dev/null +++ b/awkward1/operations/describe.py @@ -0,0 +1,56 @@ +# BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +import numbers + +import numpy + +import awkward1.layout + +def typeof(array): + if array is None: + return awkward1.layout.UnknownType() + + elif isinstance(array, (bool, numpy.bool, numpy.bool_)): + return awkward1.layout.PrimitiveType("bool") + + elif isinstance(array, numbers.Integral): + return awkward1.layout.PrimitiveType("int64") + + elif isinstance(array, numbers.Real): + return awkward1.layout.PrimitiveType("float64") + + elif isinstance(array, (numpy.int8, numpy.int16, numpy.int32, numpy.int64, numpy.uint8, numpy.uint16, numpy.uint32, numpy.uint64, numpy.float32, numpy.float64)): + return awkward1.layout.PrimitiveType(typeof.dtype2primitive[array.dtype.type]) + + elif isinstance(array, numpy.generic): + raise ValueError("cannot describe {0} as a PrimitiveType".format(type(array))) + + elif isinstance(array, numpy.ndarray): + if len(array.shape) == 0: + return typeof(array.reshape((1,))[0]) + elif len(array.shape) == 1: + return awkward1.layout.ArrayType(array.shape[0], awkward1.layout.PrimitiveType(typeof.dtype2primitive[array.dtype.type])) + else: + return awkward1.layout.ArrayType(array.shape[0], awkward1.layout.RegularType(array.shape[1:], awkward1.layout.PrimitiveType(typeof.dtype2primitive[array.dtype.type]))) + + elif isinstance(array, awkward1.layout.FillableArray): + return array.type + + elif isinstance(array, awkward1.layout.Content): + return array.type + + else: + raise TypeError("unrecognized array type: {0}".format(repr(array))) + +typeof.dtype2primitive = { + numpy.int8: "int8", + numpy.int16: "int16", + numpy.int32: "int32", + numpy.int64: "int64", + numpy.uint8: "uint8", + numpy.uint16: "uint16", + numpy.uint32: "uint32", + numpy.uint64: "uint64", + numpy.float32: "float32", + numpy.float64: "float64", +} diff --git a/include/awkward/Content.h b/include/awkward/Content.h index ef3cbe46a8..21ee38b99b 100644 --- a/include/awkward/Content.h +++ b/include/awkward/Content.h @@ -9,6 +9,7 @@ #include "awkward/Identity.h" #include "awkward/Slice.h" #include "awkward/io/json.h" +#include "awkward/type/ArrayType.h" namespace awkward { class Content { @@ -21,6 +22,7 @@ namespace awkward { virtual void setid(const std::shared_ptr id) = 0; virtual const std::string tostring_part(const std::string indent, const std::string pre, const std::string post) const = 0; virtual void tojson_part(ToJson& builder) const = 0; + virtual std::shared_ptr type_part() const = 0; virtual int64_t length() const = 0; virtual const std::shared_ptr shallow_copy() const = 0; virtual void checksafe() const = 0; @@ -33,6 +35,7 @@ namespace awkward { virtual const std::shared_ptr carry(const Index64& carry) const = 0; virtual const std::pair minmax_depth() const = 0; + const ArrayType type() const; const std::string tostring() const; const std::string tojson(bool pretty, int64_t maxdecimals) const; void tojson(FILE* destination, bool pretty, int64_t maxdecimals, int64_t buffersize) const; diff --git a/include/awkward/array/EmptyArray.h b/include/awkward/array/EmptyArray.h new file mode 100644 index 0000000000..186781fd6d --- /dev/null +++ b/include/awkward/array/EmptyArray.h @@ -0,0 +1,43 @@ +// BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +#ifndef AWKWARD_EMPTYARRAY_H_ +#define AWKWARD_EMPTYARRAY_H_ + +#include +#include +#include +#include + +#include "awkward/cpu-kernels/util.h" +#include "awkward/Slice.h" +#include "awkward/Content.h" + +namespace awkward { + class EmptyArray: public Content { + public: + EmptyArray(const std::shared_ptr id): id_(id) { } + + virtual const std::string classname() const; + virtual const std::shared_ptr id() const { return id_; } + virtual void setid(); + virtual void setid(const std::shared_ptr id); + virtual const std::string tostring_part(const std::string indent, const std::string pre, const std::string post) const; + virtual void tojson_part(ToJson& builder) const; + virtual std::shared_ptr type_part() const; + virtual int64_t length() const; + virtual const std::shared_ptr shallow_copy() const; + virtual void checksafe() const; + virtual const std::shared_ptr getitem_at(int64_t at) const; + virtual const std::shared_ptr getitem_at_unsafe(int64_t at) const; + virtual const std::shared_ptr getitem_range(int64_t start, int64_t stop) const; + virtual const std::shared_ptr getitem_range_unsafe(int64_t start, int64_t stop) const; + virtual const std::shared_ptr getitem_next(const std::shared_ptr head, const Slice& tail, const Index64& advanced) const; + virtual const std::shared_ptr carry(const Index64& carry) const; + virtual const std::pair minmax_depth() const; + + private: + std::shared_ptr id_; + }; +} + +#endif // AWKWARD_EMPTYARRAY_H_ diff --git a/include/awkward/array/ListArray.h b/include/awkward/array/ListArray.h index f2cac9a424..b7ec1b2c79 100644 --- a/include/awkward/array/ListArray.h +++ b/include/awkward/array/ListArray.h @@ -30,6 +30,7 @@ namespace awkward { virtual void setid(const std::shared_ptr id); virtual const std::string tostring_part(const std::string indent, const std::string pre, const std::string post) const; virtual void tojson_part(ToJson& builder) const; + virtual std::shared_ptr type_part() const; virtual int64_t length() const; virtual const std::shared_ptr shallow_copy() const; virtual void checksafe() const; diff --git a/include/awkward/array/ListOffsetArray.h b/include/awkward/array/ListOffsetArray.h index 48037b314f..187f68d8dc 100644 --- a/include/awkward/array/ListOffsetArray.h +++ b/include/awkward/array/ListOffsetArray.h @@ -28,6 +28,7 @@ namespace awkward { virtual void setid(const std::shared_ptr id); virtual const std::string tostring_part(const std::string indent, const std::string pre, const std::string post) const; virtual void tojson_part(ToJson& builder) const; + virtual std::shared_ptr type_part() const; virtual int64_t length() const; virtual const std::shared_ptr shallow_copy() const; virtual void checksafe() const; diff --git a/include/awkward/array/NumpyArray.h b/include/awkward/array/NumpyArray.h index 59d1512bc4..214dd57e6c 100644 --- a/include/awkward/array/NumpyArray.h +++ b/include/awkward/array/NumpyArray.h @@ -47,6 +47,7 @@ namespace awkward { virtual void setid(const std::shared_ptr id); virtual const std::string tostring_part(const std::string indent, const std::string pre, const std::string post) const; virtual void tojson_part(ToJson& builder) const; + virtual std::shared_ptr type_part() const; virtual int64_t length() const; virtual const std::shared_ptr shallow_copy() const; virtual void checksafe() const; diff --git a/include/awkward/array/RawArray.h b/include/awkward/array/RawArray.h index 229bb57732..ebe5994aee 100644 --- a/include/awkward/array/RawArray.h +++ b/include/awkward/array/RawArray.h @@ -15,6 +15,7 @@ #include "awkward/cpu-kernels/util.h" #include "awkward/cpu-kernels/identity.h" #include "awkward/cpu-kernels/getitem.h" +#include "awkward/type/PrimitiveType.h" #include "awkward/util.h" #include "awkward/Slice.h" #include "awkward/Content.h" @@ -179,6 +180,42 @@ namespace awkward { } } + virtual std::shared_ptr type_part() const { + if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::float64)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::float32)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::int64)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint64)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::int32)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint32)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::int16)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint16)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::int8)); + } + else if (std::is_same::value) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint8)); + } + else { + throw std::invalid_argument(std::string("RawArrayOf<") + typeid(T).name() + std::string("> cannot be expressed as a PrimitiveType")); + } + } + virtual int64_t length() const { return length_; } virtual const std::shared_ptr shallow_copy() const { return std::shared_ptr(new RawArrayOf(id_, ptr_, offset_, length_, itemsize_)); } diff --git a/include/awkward/type/ListType.h b/include/awkward/type/ListType.h index 912e457bfa..7e08e67f7b 100644 --- a/include/awkward/type/ListType.h +++ b/include/awkward/type/ListType.h @@ -17,7 +17,7 @@ namespace awkward { const std::shared_ptr type() const; private: - std::shared_ptr type_; + const std::shared_ptr type_; }; } diff --git a/include/awkward/type/OptionType.h b/include/awkward/type/OptionType.h index 4735ead848..6c198bf2f7 100644 --- a/include/awkward/type/OptionType.h +++ b/include/awkward/type/OptionType.h @@ -17,7 +17,7 @@ namespace awkward { const std::shared_ptr type() const; private: - std::shared_ptr type_; + const std::shared_ptr type_; }; } diff --git a/include/awkward/type/PrimitiveType.h b/include/awkward/type/PrimitiveType.h index 077ee77d93..fe1df954dc 100644 --- a/include/awkward/type/PrimitiveType.h +++ b/include/awkward/type/PrimitiveType.h @@ -30,7 +30,7 @@ namespace awkward { virtual bool equal(std::shared_ptr other) const; private: - DType dtype_; + const DType dtype_; }; } diff --git a/include/awkward/type/RegularType.h b/include/awkward/type/RegularType.h new file mode 100644 index 0000000000..bcfc946dae --- /dev/null +++ b/include/awkward/type/RegularType.h @@ -0,0 +1,28 @@ +// BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +#ifndef AWKWARD_REGULARTYPE_H_ +#define AWKWARD_REGULARTYPE_H_ + +#include + +#include "awkward/type/Type.h" + +namespace awkward { + class RegularType: public Type { + public: + RegularType(const std::vector shape, const std::shared_ptr type): shape_(shape), type_(type) { } + + virtual std::string tostring_part(std::string indent, std::string pre, std::string post) const; + virtual const std::shared_ptr shallow_copy() const; + virtual bool equal(std::shared_ptr other) const; + + const std::vector shape() const; + const std::shared_ptr type() const; + + private: + const std::vector shape_; + const std::shared_ptr type_; + }; +} + +#endif // AWKWARD_REGULARTYPE_H_ diff --git a/include/awkward/type/UnionType.h b/include/awkward/type/UnionType.h index 5ee13bf708..7807f4b009 100644 --- a/include/awkward/type/UnionType.h +++ b/include/awkward/type/UnionType.h @@ -21,7 +21,7 @@ namespace awkward { const std::shared_ptr type(int64_t i) const; private: - std::vector> types_; + const std::vector> types_; }; } diff --git a/src/libawkward/Content.cpp b/src/libawkward/Content.cpp index 89a13827fd..15c7a4670d 100644 --- a/src/libawkward/Content.cpp +++ b/src/libawkward/Content.cpp @@ -1,10 +1,15 @@ // BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE #include "awkward/array/ListArray.h" +#include "awkward/type/ArrayType.h" #include "awkward/Content.h" namespace awkward { + const ArrayType Content::type() const { + return ArrayType(length(), type_part()); + } + const std::string Content::tostring() const { return tostring_part("", "", ""); } diff --git a/src/libawkward/array/EmptyArray.cpp b/src/libawkward/array/EmptyArray.cpp new file mode 100644 index 0000000000..de4d054d0f --- /dev/null +++ b/src/libawkward/array/EmptyArray.cpp @@ -0,0 +1,114 @@ +// BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +#include +#include +#include + +#include "awkward/type/UnknownType.h" + +#include "awkward/array/EmptyArray.h" + +namespace awkward { + const std::string EmptyArray::classname() const { + return "EmptyArray"; + } + + void EmptyArray::setid(const std::shared_ptr id) { + if (id.get() != nullptr && length() != id.get()->length()) { + util::handle_error(failure("content and its id must have the same length", kSliceNone, kSliceNone), classname(), id_.get()); + } + id_ = id; + } + + void EmptyArray::setid() { } + + const std::string EmptyArray::tostring_part(const std::string indent, const std::string pre, const std::string post) const { + std::stringstream out; + out << indent << pre << "<" << classname(); + if (id_.get() != nullptr) { + out << ">\n" << id_.get()->tostring_part(indent + std::string(" "), "", "\n") << indent << "" << post; + } + else { + out << "/>" << post; + } + return out.str(); + } + + void EmptyArray::tojson_part(ToJson& builder) const { + builder.beginlist(); + builder.endlist(); + } + + std::shared_ptr EmptyArray::type_part() const { + return std::shared_ptr(new UnknownType()); + } + + int64_t EmptyArray::length() const { + return 0; + } + + const std::shared_ptr EmptyArray::shallow_copy() const { + return std::shared_ptr(new EmptyArray(id_)); + } + + void EmptyArray::checksafe() const { } + + const std::shared_ptr EmptyArray::getitem_at(int64_t at) const { + util::handle_error(failure("index out of range", kSliceNone, at), classname(), id_.get()); + return std::shared_ptr(nullptr); // make Windows compiler happy + } + + const std::shared_ptr EmptyArray::getitem_at_unsafe(int64_t at) const { + util::handle_error(failure("index out of range", kSliceNone, at), classname(), id_.get()); + return std::shared_ptr(nullptr); // make Windows compiler happy + } + + const std::shared_ptr EmptyArray::getitem_range(int64_t start, int64_t stop) const { + return shallow_copy(); + } + + const std::shared_ptr EmptyArray::getitem_range_unsafe(int64_t start, int64_t stop) const { + return shallow_copy(); + } + + const std::shared_ptr EmptyArray::getitem_next(const std::shared_ptr head, const Slice& tail, const Index64& advanced) const { + if (head.get() == nullptr) { + return shallow_copy(); + } + + else if (SliceAt* at = dynamic_cast(head.get())) { + util::handle_error(failure("too many dimensions in slice", kSliceNone, kSliceNone), classname(), id_.get()); + return std::shared_ptr(nullptr); // make Windows compiler happy + } + + else if (SliceRange* range = dynamic_cast(head.get())) { + util::handle_error(failure("too many dimensions in slice", kSliceNone, kSliceNone), classname(), id_.get()); + return std::shared_ptr(nullptr); // make Windows compiler happy + } + + else if (SliceEllipsis* ellipsis = dynamic_cast(head.get())) { + return getitem_ellipsis(tail, advanced); + } + + else if (SliceNewAxis* newaxis = dynamic_cast(head.get())) { + return getitem_newaxis(tail, advanced); + } + + else if (SliceArray64* array = dynamic_cast(head.get())) { + util::handle_error(failure("too many dimensions in slice", kSliceNone, kSliceNone), classname(), id_.get()); + return std::shared_ptr(nullptr); // make Windows compiler happy + } + + else { + throw std::runtime_error("unrecognized slice item type"); + } + + throw std::runtime_error("unreachable because of EmptyArray::carry"); + } + + const std::shared_ptr EmptyArray::carry(const Index64& carry) const { + return shallow_copy(); + } + + const std::pair EmptyArray::minmax_depth() const { return std::pair(1, 1); } +} diff --git a/src/libawkward/array/ListArray.cpp b/src/libawkward/array/ListArray.cpp index d8ba03e5d2..945afdd2d6 100644 --- a/src/libawkward/array/ListArray.cpp +++ b/src/libawkward/array/ListArray.cpp @@ -5,6 +5,7 @@ #include "awkward/cpu-kernels/identity.h" #include "awkward/cpu-kernels/getitem.h" +#include "awkward/type/ListType.h" #include "awkward/Slice.h" #include "awkward/array/ListOffsetArray.h" @@ -156,6 +157,11 @@ namespace awkward { } } + template + std::shared_ptr ListArrayOf::type_part() const { + return std::shared_ptr(new ListType(content_.get()->type_part())); + } + template int64_t ListArrayOf::length() const { return starts_.length(); @@ -199,8 +205,14 @@ namespace awkward { if (start == stop) { start = stop = 0; } - if (!(0 <= start && start < lencontent) || !(start <= stop && stop <= lencontent)) { - util::handle_error(failure("not 0 <= starts[i] < len(content) or not starts[i] <= stops[i] <= len(content)", kSliceNone, at), classname(), id_.get()); + if (start < 0) { + util::handle_error(failure("starts[i] < 0", kSliceNone, at), classname(), id_.get()); + } + if (start > stop) { + util::handle_error(failure("starts[i] > stops[i]", kSliceNone, at), classname(), id_.get()); + } + if (stop > lencontent) { + util::handle_error(failure("starts[i] != stops[i] and stops[i] > len(content)", kSliceNone, at), classname(), id_.get()); } return content_.get()->getitem_range_unsafe(start, stop); } diff --git a/src/libawkward/array/ListOffsetArray.cpp b/src/libawkward/array/ListOffsetArray.cpp index 0257864065..1523ba60eb 100644 --- a/src/libawkward/array/ListOffsetArray.cpp +++ b/src/libawkward/array/ListOffsetArray.cpp @@ -5,6 +5,7 @@ #include "awkward/cpu-kernels/identity.h" #include "awkward/cpu-kernels/getitem.h" +#include "awkward/type/ListType.h" #include "awkward/Slice.h" #include "awkward/array/ListArray.h" @@ -169,6 +170,11 @@ namespace awkward { } } + template + std::shared_ptr ListOffsetArrayOf::type_part() const { + return std::shared_ptr(new ListType(content_.get()->type_part())); + } + template int64_t ListOffsetArrayOf::length() const { return offsets_.length() - 1; @@ -206,8 +212,14 @@ namespace awkward { if (start == stop) { start = stop = 0; } - if (!(0 <= start && start < lencontent) || !(start <= stop && stop <= lencontent)) { - util::handle_error(failure("not 0 <= offsets[i] < len(content) or not offsets[i] <= offsets[i + 1] <= len(content)", kSliceNone, at), classname(), id_.get()); + if (start < 0) { + util::handle_error(failure("offsets[i] < 0", kSliceNone, at), classname(), id_.get()); + } + if (start > stop) { + util::handle_error(failure("offsets[i] > offsets[i + 1]", kSliceNone, at), classname(), id_.get()); + } + if (stop > lencontent) { + util::handle_error(failure("offsets[i] != offsets[i + 1] and offsets[i + 1] > len(content)", kSliceNone, at), classname(), id_.get()); } return content_.get()->getitem_range_unsafe(start, stop); } diff --git a/src/libawkward/array/NumpyArray.cpp b/src/libawkward/array/NumpyArray.cpp index b68b40fe97..988f30dcc9 100644 --- a/src/libawkward/array/NumpyArray.cpp +++ b/src/libawkward/array/NumpyArray.cpp @@ -6,6 +6,8 @@ #include "awkward/cpu-kernels/identity.h" #include "awkward/cpu-kernels/getitem.h" +#include "awkward/type/PrimitiveType.h" +#include "awkward/type/RegularType.h" #include "awkward/array/NumpyArray.h" @@ -202,6 +204,7 @@ namespace awkward { } void NumpyArray::tojson_part(ToJson& builder) const { + assert(!isscalar()); if (ndim() == 1) { if (format_.compare("d") == 0) { tojson_real(builder, reinterpret_cast(byteptr()), length()); @@ -262,6 +265,65 @@ namespace awkward { } } + std::shared_ptr NumpyArray::type_part() const { + if (ndim() == 1) { + if (format_.compare("d") == 0) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::float64)); + } + else if (format_.compare("f") == 0) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::float32)); + } +#ifdef _MSC_VER + else if (format_.compare("q") == 0) { +#else + else if (format_.compare("l") == 0) { +#endif + return std::shared_ptr(new PrimitiveType(PrimitiveType::int64)); + } +#ifdef _MSC_VER + else if (format_.compare("Q") == 0) { +#else + else if (format_.compare("L") == 0) { +#endif + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint64)); + } +#ifdef _MSC_VER + else if (format_.compare("l") == 0) { +#else + else if (format_.compare("i") == 0) { +#endif + return std::shared_ptr(new PrimitiveType(PrimitiveType::int32)); + } +#ifdef _MSC_VER + else if (format_.compare("L") == 0) { +#else + else if (format_.compare("I") == 0) { +#endif + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint32)); + } + else if (format_.compare("h") == 0) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::int16)); + } + else if (format_.compare("H") == 0) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint16)); + } + else if (format_.compare("b") == 0) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::int8)); + } + else if (format_.compare("B") == 0 || format_.compare("c") == 0) { + return std::shared_ptr(new PrimitiveType(PrimitiveType::uint8)); + } + else { + throw std::invalid_argument(std::string("Numpy format \"") + format_ + std::string("\" cannot be expressed as a PrimitiveType")); + } + } + else { + NumpyArray tmp(id_, ptr_, std::vector({ 1 }), std::vector({ itemsize_ }), byteoffset_, itemsize_, format_); + std::vector shape(shape_.begin() + 1, shape_.end()); + return std::shared_ptr(new RegularType(shape, tmp.type_part())); + } + } + int64_t NumpyArray::length() const { if (isscalar()) { return -1; diff --git a/src/libawkward/fillable/UnknownFillable.cpp b/src/libawkward/fillable/UnknownFillable.cpp index b8b723423e..56e5e511e7 100644 --- a/src/libawkward/fillable/UnknownFillable.cpp +++ b/src/libawkward/fillable/UnknownFillable.cpp @@ -3,6 +3,7 @@ #include #include "awkward/Identity.h" +#include "awkward/array/EmptyArray.h" #include "awkward/type/UnknownType.h" #include "awkward/fillable/OptionFillable.h" #include "awkward/fillable/BoolFillable.h" @@ -26,7 +27,12 @@ namespace awkward { } const std::shared_ptr UnknownFillable::snapshot() const { - throw std::runtime_error("UnknownFillable::snapshot() needs EmptyArray"); + if (nullcount_ == 0) { + return std::shared_ptr(new EmptyArray(Identity::none())); + } + else { + throw std::runtime_error("UnknownFillable::snapshot() needs MaskedArray"); + } } Fillable* UnknownFillable::null() { diff --git a/src/libawkward/type/RegularType.cpp b/src/libawkward/type/RegularType.cpp new file mode 100644 index 0000000000..b31deed60b --- /dev/null +++ b/src/libawkward/type/RegularType.cpp @@ -0,0 +1,44 @@ +// BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +#include +#include + +#include "awkward/type/UnknownType.h" + +#include "awkward/type/RegularType.h" + +namespace awkward { + std::string RegularType::tostring_part(std::string indent, std::string pre, std::string post) const { + std::stringstream out; + out << indent << pre; + for (auto x : shape_) { + out << x << " * "; + } + out << type_.get()->tostring_part(indent, "", "") << post; + return out.str(); + } + + const std::shared_ptr RegularType::shallow_copy() const { + return std::shared_ptr(new RegularType(shape_, type_)); + } + + bool RegularType::equal(std::shared_ptr other) const { + if (UnknownType* t = dynamic_cast(other.get())) { + return true; + } + else if (RegularType* t = dynamic_cast(other.get())) { + return shape() == t->shape() && type().get()->equal(t->type()); + } + else { + return false; + } + } + + const std::vector RegularType::shape() const { + return shape_; + } + + const std::shared_ptr RegularType::type() const { + return type_; + } +} diff --git a/src/pyawkward.cpp b/src/pyawkward.cpp index 23b6c490c8..c4a86493cd 100644 --- a/src/pyawkward.cpp +++ b/src/pyawkward.cpp @@ -15,12 +15,14 @@ #include "awkward/array/NumpyArray.h" #include "awkward/array/ListArray.h" #include "awkward/array/ListOffsetArray.h" +#include "awkward/array/EmptyArray.h" #include "awkward/fillable/FillableOptions.h" #include "awkward/fillable/FillableArray.h" #include "awkward/type/Type.h" #include "awkward/type/ArrayType.h" #include "awkward/type/UnknownType.h" #include "awkward/type/PrimitiveType.h" +#include "awkward/type/RegularType.h" #include "awkward/type/ListType.h" #include "awkward/type/OptionType.h" #include "awkward/type/UnionType.h" @@ -47,15 +49,24 @@ py::object box(std::shared_ptr t) { if (ak::ArrayType* raw = dynamic_cast(t.get())) { return py::cast(*raw); } - else if (ak::PrimitiveType* raw = dynamic_cast(t.get())) { + else if (ak::ListType* raw = dynamic_cast(t.get())) { return py::cast(*raw); } else if (ak::OptionType* raw = dynamic_cast(t.get())) { return py::cast(*raw); } + else if (ak::PrimitiveType* raw = dynamic_cast(t.get())) { + return py::cast(*raw); + } + else if (ak::RegularType* raw = dynamic_cast(t.get())) { + return py::cast(*raw); + } else if (ak::UnionType* raw = dynamic_cast(t.get())) { return py::cast(*raw); } + else if (ak::UnknownType* raw = dynamic_cast(t.get())) { + return py::cast(*raw); + } else { throw std::runtime_error("missing boxer for Type subtype"); } @@ -95,6 +106,9 @@ py::object box(std::shared_ptr content) { else if (ak::ListOffsetArray64* raw = dynamic_cast(content.get())) { return py::cast(*raw); } + else if (ak::EmptyArray* raw = dynamic_cast(content.get())) { + return py::cast(*raw); + } else { throw std::runtime_error("missing boxer for Content subtype"); } @@ -128,6 +142,10 @@ std::shared_ptr unbox_type(py::handle obj) { return obj.cast()->shallow_copy(); } catch (py::cast_error err) { } + try { + return obj.cast()->shallow_copy(); + } + catch (py::cast_error err) { } try { return obj.cast()->shallow_copy(); } @@ -172,6 +190,10 @@ std::shared_ptr unbox_content(py::object obj) { return obj.cast()->shallow_copy(); } catch (py::cast_error err) { } + try { + return obj.cast()->shallow_copy(); + } + catch (py::cast_error err) { } throw std::invalid_argument("content argument must be a Content subtype"); } @@ -507,7 +529,7 @@ py::class_ make_FillableArray(py::handle m, std::string name) .def("__repr__", &ak::FillableArray::tostring) .def("__len__", &ak::FillableArray::length) .def("clear", &ak::FillableArray::clear) - .def("type", &ak::FillableArray::type) + .def_property_readonly("type", &ak::FillableArray::type) .def("snapshot", [](ak::FillableArray& self) -> py::object { return box(self.snapshot()); }) @@ -597,6 +619,16 @@ py::class_, ak::Type> make ); } +py::class_, ak::Type> make_RegularType(py::handle m, std::string name) { + return (py::class_, ak::Type>(m, name.c_str()) + .def(py::init, std::shared_ptr>()) + .def_property_readonly("shape", &ak::RegularType::shape) + .def_property_readonly("type", &ak::RegularType::type) + .def("__repr__", &ak::RegularType::tostring) + .def("__eq__", &ak::RegularType::equal) + ); +} + py::class_, ak::Type> make_ListType(py::handle m, std::string name) { return (py::class_, ak::Type>(m, name.c_str()) .def(py::init>()) @@ -697,7 +729,8 @@ py::class_ content(py::class_& x) { .def("__getitem__", &getitem) .def("__iter__", &iter) .def("tojson", &tojson_string, py::arg("pretty") = false, py::arg("maxdecimals") = py::none()) - .def("tojson", &tojson_file, py::arg("destination"), py::arg("pretty") = false, py::arg("maxdecimals") = py::none(), py::arg("buffersize") = 65536); + .def("tojson", &tojson_file, py::arg("destination"), py::arg("pretty") = false, py::arg("maxdecimals") = py::none(), py::arg("buffersize") = 65536) + .def_property_readonly("type", &ak::Content::type); } py::class_ make_Content(py::handle m, std::string name) { @@ -782,6 +815,16 @@ py::class_, ak::Content> make_ListOffsetArrayOf(py::han ); } +/////////////////////////////////////////////////////////////// EmptyArray + +py::class_ make_EmptyArray(py::handle m, std::string name) { + return content(py::class_(m, name.c_str()) + .def(py::init([](py::object id) -> ak::EmptyArray { + return ak::EmptyArray(unbox_id(id)); + }), py::arg("id") = py::none()) + ); +} + /////////////////////////////////////////////////////////////// module PYBIND11_MODULE(layout, m) { @@ -809,6 +852,7 @@ PYBIND11_MODULE(layout, m) { make_Type(m, "Type"); make_ArrayType(m, "ArrayType"); make_PrimitiveType(m, "PrimitiveType"); + make_RegularType(m, "RegularType"); make_UnknownType(m, "UnknownType"); make_ListType(m, "ListType"); make_OptionType(m, "OptionType"); @@ -826,6 +870,8 @@ PYBIND11_MODULE(layout, m) { make_ListOffsetArrayOf(m, "ListOffsetArrayU32"); make_ListOffsetArrayOf(m, "ListOffsetArray64"); + make_EmptyArray(m, "EmptyArray"); + m.def("fromjson", [](std::string source, int64_t initial, double resize, int64_t buffersize) -> py::object { bool isarray = false; for (char const &x: source) { diff --git a/tests/test_PR021_emptyarray.py b/tests/test_PR021_emptyarray.py new file mode 100644 index 0000000000..fce9d19e01 --- /dev/null +++ b/tests/test_PR021_emptyarray.py @@ -0,0 +1,157 @@ +# BSD 3-Clause License; see https://github.com/jpivarski/awkward-1.0/blob/master/LICENSE + +import sys +import os +import json + +import pytest +import numpy + +import awkward1 + +def test_unknown(): + a = awkward1.fromjson("[[], [], []]") + assert awkward1.tolist(a) == [[], [], []] + assert str(awkward1.typeof(a)) == "3 * var * ???" + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.UnknownType())) + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.PrimitiveType("float64"))) + assert awkward1.typeof(a) != awkward1.layout.ArrayType(3, awkward1.layout.PrimitiveType("float64")) + + a = awkward1.fromjson("[[], [[], []], [[], [], []]]") + assert awkward1.tolist(a) == [[], [[], []], [[], [], []]] + assert str(awkward1.typeof(a)) == "3 * var * var * ???" + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.ListType(awkward1.layout.UnknownType()))) + + a = awkward1.layout.FillableArray() + a.beginlist() + a.endlist() + a.beginlist() + a.endlist() + a.beginlist() + a.endlist() + assert awkward1.tolist(a) == [[], [], []] + assert str(awkward1.typeof(a)) == "3 * var * ???" + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.UnknownType())) + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.PrimitiveType("float64"))) + assert awkward1.typeof(a) != awkward1.layout.ArrayType(3, awkward1.layout.PrimitiveType("float64")) + + a = a.snapshot() + assert awkward1.tolist(a) == [[], [], []] + assert str(awkward1.typeof(a)) == "3 * var * ???" + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.UnknownType())) + assert awkward1.typeof(a) == awkward1.layout.ArrayType(3, awkward1.layout.ListType(awkward1.layout.PrimitiveType("float64"))) + assert awkward1.typeof(a) != awkward1.layout.ArrayType(3, awkward1.layout.PrimitiveType("float64")) + +def test_getitem(): + a = awkward1.fromjson("[[], [[], []], [[], [], []]]") + assert awkward1.tolist(a[2]) == [[], [], []] + + assert awkward1.tolist(a[2, 1]) == [] + with pytest.raises(ValueError) as excinfo: + a[2, 1, 0] + assert str(excinfo.value) == "in ListArray64 attempting to get 0, index out of range" + assert awkward1.tolist(a[2, 1][()]) == [] + with pytest.raises(ValueError) as excinfo: + a[2, 1][0] + assert str(excinfo.value) == "in EmptyArray attempting to get 0, index out of range" + assert awkward1.tolist(a[2, 1][100:200]) == [] + assert awkward1.tolist(a[2, 1, 100:200]) == [] + assert awkward1.tolist(a[2, 1][numpy.array([], dtype=int)]) == [] + assert awkward1.tolist(a[2, 1, numpy.array([], dtype=int)]) == [] + with pytest.raises(ValueError) as excinfo: + a[2, 1, numpy.array([0], dtype=int)] + assert str(excinfo.value) == "in ListArray64 attempting to get 0, index out of range" + with pytest.raises(ValueError) as excinfo: + a[2, 1][100:200, 0] + assert str(excinfo.value) == "in EmptyArray, too many dimensions in slice" + with pytest.raises(ValueError) as excinfo: + a[2, 1][100:200, 200:300] + assert str(excinfo.value) == "in EmptyArray, too many dimensions in slice" + with pytest.raises(ValueError) as excinfo: + a[2, 1][100:200, numpy.array([], dtype=int)] + assert str(excinfo.value) == "in EmptyArray, too many dimensions in slice" + + assert awkward1.tolist(a[1:, 1:]) == [[[]], [[], []]] + with pytest.raises(ValueError) as excinfo: + a[1:, 1:, 0] + assert str(excinfo.value) == "in ListArray64 attempting to get 0, index out of range" + +numba = pytest.importorskip("numba") +def test_numba(): + a = awkward1.fromjson("[[], [[], []], [[], [], []]]") + + @numba.njit + def f1(q): + return q[2, 1] + assert awkward1.tolist(f1(a)) == [] + + @numba.njit + def f2(q): + return q[2, 1][()] + assert awkward1.tolist(f2(a)) == [] + + @numba.njit + def f3(q): + return q[2, 1][100:200] + assert awkward1.tolist(f3(a)) == [] + + @numba.njit + def f4(q): + return q[2, 1, 0] + with pytest.raises(numba.errors.TypingError): + f4(a) + + @numba.njit + def f5(q): + return q[2, 1, 100:200] + assert awkward1.tolist(f5(a)) == [] + + @numba.njit + def f6a(q): + return q[2, 1, 100:200, 0] + with pytest.raises(numba.errors.TypingError): + f6a(a) + + @numba.njit + def f6b(q): + return q[2, 1, 100:200][0] + with pytest.raises(numba.errors.TypingError): + f6b(a) + + @numba.njit + def f7a(q): + return q[2, 1, 100:200, 200:300] + with pytest.raises(numba.errors.TypingError): + f7a(a) + + @numba.njit + def f7b(q): + return q[2, 1, 100:200][200:300] + assert awkward1.tolist(f7b(a)) == [] + + @numba.njit + def f7c(q): + return q[2, 1, 100:200][()] + assert awkward1.tolist(f7c(a)) == [] + + @numba.njit + def f8a(q): + return q[2, 1, 100:200, numpy.array([], dtype=numpy.int64)] + with pytest.raises(numba.errors.TypingError): + f8a(a) + + @numba.njit + def f8b(q, z): + return q[2, 1, z] + assert awkward1.tolist(f8b(a, numpy.array([], dtype=int))) == [] + + @numba.njit + def f8c(q, z): + return q[2, 1, z, z] + with pytest.raises(numba.errors.TypingError): + f8c(a, numpy.array([], dtype=int)) + + @numba.njit + def f8d(q, z): + return q[2, 1, z][()] + assert awkward1.tolist(f8d(a, numpy.array([], dtype=int))) == []