marimo
Log in

Get help from the marimo community

Updated 3 weeks ago

Notebook termination issue when using GPU

Bug
QQuill Healey
·
Hello!
I’ve been experiencing a bug when I have a large amount of data loaded onto my GPU. Specifically,
  • The host I’m working on is a Dell tower (with a NVIDIA GPU device) running Ubuntu 24.04.
  • The marimo notebook I’m running pulls data from a mongo host (on the order of gigabytes) and then loads this data onto device via CuPy
  • the notebook operates as expected (manipulating the data, transferring some back to host to generate plots, etc.)
  • HOWEVER, when I attempt to shut the notebook down via the red circle with an x, while the browser tab is quit out of, the marimo process continues to run from the command line, and even when I ctrl+C, the process still doesn’t quit. I have to kill -9 <pid> in order to actually stop the process. (The photos I attached are after I hit red x from the notebook editor, and then after I try to ctrl+C a few times.)
As a note, notebooks in this sensorenv environment where I'm working with solely the CPU or even fewer bytes on device memory terminate as expected.

Any thoughts on what I could be doing wrong or if this is a bug?

(Oh, and sorry for any potential misuse of dialect!)
Attachments
Screenshot_2025-02-21_at_10.17.58_AM.png
Screenshot_2025-02-21_at_9.37.47_AM.png
A
Q
6 comments
AAkshay
Interesting … let me try to reproduce on my machine. Thanks for reporting
AAkshay
I tried loading ~5GB onto my GPU and couldn't reproduce. I think the GPU usage is likely surfacing some other unrelated problem. Can you run with marimo -l debug edit post_processor_simple.py and paste the logs you see in the terminal when you click the red x button?
QQuill Healey
Ah, for some reason discord didn’t alert me to this message 🤦‍♂️ .
I left the office already, but will see if one of my coworkers can try this for me. If not then I’ll go back and try tomorrow. Thanks for trying to reproduce on your machine!
My guess at this point is the bug is probably how I’m connecting to the data server/potentially not closing that properly. But will run the suggested command and report back!
QQuill Healey
Sorry for all the horrendous red! Also, I have more of the log if you need it
Attachments
Screenshot_2025-02-21_at_6.37.59_PM_3.png
Screenshot_2025-02-21_at_6.33.00_PM.png
AAkshay
Interesting thanks. We had problems like this a while time ago, it’s possible that something regressed.
QQuill Healey
No problem — happy to help/get you more data around what I’m seeing!
Giving it some more thought, here is additional info around this termination, GPU issue:

  1. Three of my coworkers have also seen this issue
  2. It actually has happened not just when loading data from mongo, but also loading the same type of data (complex vectors with millions of components) from CSV files as well.
I’ll try to do some more formal testing next week at work. Excitingly, the company I work for is doing a migration to marimo, so I should be able to help determine if this is an us problem or if it is actually a bug/regression as the migratoin progresses
Add a reply
Sign up and join the conversation on Discord