FileInput in databricks notebooks results in a timeout exception on read()

[ntn-rjdn]

I'm trying to use FileInput to read a file from my local machine in a databricks notebook.

I considered ipywidgets, but databricks doesn't seem to support the file upload feature in ipywidgets - https://docs.databricks.com/notebooks/ipywidgets.html#limitations

This is the code I tried. Picking the file works as expected.

from ipyvuetify.extra import FileInput
file_input = FileInput()
file_input

files = file_input.get_files()
data = files[0]['file_obj'].read()
data

I get this timeout exception when I try the read() above -

Exception                                 Traceback (most recent call last)
File <command-2037501063098494>:1
----> 1 data = files[0]['file_obj'].read()
      2 data

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-f391b237/lib/python3.9/site-packages/ipyvuetify/extra/file_input.py:147, in ClientSideFile.readall(self)
    146 def readall(self):
--> 147     return self.read(self.size - self.offset)

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-f391b237/lib/python3.9/site-packages/ipyvuetify/extra/file_input.py:143, in ClientSideFile.readinto(self, buffer)
    140         self.widget.update_stats(self.file_index, chunk_size)
    141         await process_messages()
--> 143 asyncio.run(read_all())
    144 return size

File /databricks/python/lib/python3.9/site-packages/nest_asyncio.py:35, in _patch_asyncio.<locals>.run(main, debug)
     33 task = asyncio.ensure_future(main)
     34 try:

Would really appreciate if you have any inputs about this.

[maartenbreddels]

Hi Nithin,

It is very difficult for us to debug this, and it might be because the databricks notebook does things slightly different.
We were in a similar situation with solara, and Jason Grout (ipywidgets maintainer and works at databricks), suggested to bring this up at Databricks first. You can reference this issue for if the people at databricks need our help, or more information.
See widgetti/solara#128 (comment)

Some ideas why this may not work are:

• we use nest_asyncio. Although I would expect asyncio.run to not execute if that failed to patch.
• databricks has their own widget manager, maybe the message from the frontend to the kernel is somehow not getting through.

Regards,

Maarten

[Chaoses-Ib]

I have a similar issue when using Solara in VS Code:

import solara

@solara.component
def Page():
    def on_file(file_info: solara.file_drop.FileInfo):
        print(file_info)
        print(file_info['file_obj'].read())
    solara.FileDrop("Drop file here", on_file=on_file)

Page()

Timeout
Traceback (most recent call last):
  File "c:\ProgramData\Miniconda3\lib\site-packages\solara\hooks\use_thread.py", line 84, in runner
    value = f()
  File "c:\ProgramData\Miniconda3\lib\site-packages\solara\components\file_drop.py", line 87, in handle_file
    wired_files[0]["data"] = wired_files[0]["file_obj"].read()
  File "c:\ProgramData\Miniconda3\lib\site-packages\ipyvuetify\extra\file_input.py", line 147, in readall
    return self.read(self.size - self.offset)
  File "c:\ProgramData\Miniconda3\lib\site-packages\ipyvuetify\extra\file_input.py", line 143, in readinto
    asyncio.run(read_all())
  File "c:\ProgramData\Miniconda3\lib\site-packages\nest_asyncio.py", line 35, in run
    return loop.run_until_complete(task)
  File "c:\ProgramData\Miniconda3\lib\site-packages\nest_asyncio.py", line 89, in run_until_complete
    return f.result()
  File "c:\ProgramData\Miniconda3\lib\asyncio\futures.py", line 201, in result
    raise self._exception
  File "c:\ProgramData\Miniconda3\lib\asyncio\tasks.py", line 256, in __step
    result = coro.send(None)
  File "c:\ProgramData\Miniconda3\lib\site-packages\ipyvuetify\extra\file_input.py", line 124, in read_all
    raise Exception("Timeout")
Exception: Timeout
Component react.component(solara.components.file_drop.FileDrop) raised exception Exception('Timeout')
Traceback (most recent call last):
  File "c:\ProgramData\Miniconda3\lib\site-packages\reacton\core.py", line 1675, in _render
    root_element = el.component.f(*el.args, **el.kwargs)
  File "c:\ProgramData\Miniconda3\lib\site-packages\solara\components\file_drop.py", line 94, in FileDrop
    raise result.error
  File "c:\ProgramData\Miniconda3\lib\site-packages\solara\hooks\use_thread.py", line 84, in runner
    value = f()
  File "c:\ProgramData\Miniconda3\lib\site-packages\solara\components\file_drop.py", line 87, in handle_file
    wired_files[0]["data"] = wired_files[0]["file_obj"].read()
  File "c:\ProgramData\Miniconda3\lib\site-packages\ipyvuetify\extra\file_input.py", line 147, in readall
    return self.read(self.size - self.offset)
  File "c:\ProgramData\Miniconda3\lib\site-packages\ipyvuetify\extra\file_input.py", line 143, in readinto
    asyncio.run(read_all())
  File "c:\ProgramData\Miniconda3\lib\site-packages\nest_asyncio.py", line 35, in run
    return loop.run_until_complete(task)
  File "c:\ProgramData\Miniconda3\lib\site-packages\nest_asyncio.py", line 89, in run_until_complete
    return f.result()
  File "c:\ProgramData\Miniconda3\lib\asyncio\futures.py", line 201, in result
    raise self._exception
  File "c:\ProgramData\Miniconda3\lib\asyncio\tasks.py", line 256, in __step
    result = coro.send(None)
  File "c:\ProgramData\Miniconda3\lib\site-packages\ipyvuetify\extra\file_input.py", line 124, in read_all
    raise Exception("Timeout")
Exception: Timeout

I tried to set lazy=False, but the result is the same.

Edit: It turns out that this is an issue of the Jupyter kernel and VS Code: microsoft/vscode-jupyter#13469. It's already fixed, but not yet released.