Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Cannot create large ndarray from numpy #13070

Closed
eric-haibin-lin opened this issue Oct 31, 2018 · 2 comments
Closed

Cannot create large ndarray from numpy #13070

eric-haibin-lin opened this issue Oct 31, 2018 · 2 comments

Comments

@eric-haibin-lin
Copy link
Member

>>> import mxnet as mx
>>> import numpy as np
>>> a = np.empty((5000000000,))
>>> a
array([0., 0., 0., ..., 0., 0., 0.])
>>> b = mx.nd.array(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/ndarray/utils.py", line 146, in array
    return _array(source_array, ctx=ctx, dtype=dtype)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 2488, in array
    arr = empty(source_array.shape, ctx, dtype)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 3877, in empty
    return NDArray(handle=_new_alloc_handle(shape, ctx, False, dtype))
  File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 134, in _new_alloc_handle
    c_array_buf(mx_uint, native_array('I', shape)),
OverflowError: unsigned int is greater than maximum
>>>
@wkcn
Copy link
Member

wkcn commented Nov 1, 2018

Similar Issue: #13036

@apeforest
Copy link
Contributor

This issue has been fixed. In 1.5.0 release, user need to build MXNet from source with the compilation flag USE_INT64_TENSOR_SIZE=1. We are working to make this flag on by default and available in pip package in next minor release. Closing this issue for now.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants