Get help from the marimo community

Updated 3 months ago

SQL <> Python cell auto-conversion

At a glance

The community member was experiencing an issue where cells in their notebook were automatically converting from Python to SQL, causing downstream dependency issues. To mitigate this, they tried converting the SQL cells to Python, but found that the cells would revert back to SQL after restarting the kernel. They resolved this by making the dataframe local and creating a copy.

The community member also encountered an issue where cells with mo.sql and Python code would convert to SQL upon restart or rerun. They tried various methods to keep the cells in Python format, but ultimately had to break out the mo.sql code into a separate cell and display the resulting dataframe in an mo.md cell.

The community member recommended that ideally, cells would not auto-convert from Python to SQL, and that it should be possible to generate a dataframe with mo.sql and present it in an mo.md cell without the conversion issue. They suggested that if this is not desired behavior, an exception should be thrown to inform users.

In the comments, other community members provided some additional insights and suggestions, including that the SQL rendering of cells is a UI feature and the underlying code is still Python, and that there is an example of using the query result

Useful resources
problem: I was attempting to solve this issue and found that cells were automatically converting to different cell types.

Auto conversion from Python --> SQL
  • To mitigate some downstream dependency issues, I first tried converting the SQL cell to python (e.g, "df_result = mo.sql(f""" query """)") , and then would save the file, and then reset the Kernel. Post kernel reset, that cell that I converted to Python was back to being a SQL file.
  • I resolved this auto-conversion by making df_result local (_df_result) and then df_result = _df_result.copy().
  • Even though both were outputting the same DF, the python version seemed to work more extensively for dependencies.
Auto conversion from mo.sql + Python code --> SQL

  • I have data from an mo.sql outputting to a dataframe.
  • This dataframe is then used in am mo.md
  • Everything would be set up correctly - I would restart the kernel and the script would autorun
  • Upon restart/rerun, my cells with mo.sql + Python would convert to SQL. I tried many times to have the .py cell format stick.
  • To resolve this, I had to break out my mo.sql -> dataframe code (also unfortunately having to use the fix above) into a previous cell, and then display that resulting dataframe via an mo.md in the next cell.
Recommendation

Ideally, cells would not auto-convert from Python --> SQL. Some people might be more comfortable using mo.sql explicitly.

Additionally, ideally I could generate a DF with mo.sql in a cell, and also present that DF in mo.md. If this is undesired, then throwing an exception and stating this requirement would be helpful.
A
r
6 comments
Additionally, ideally I could generate a DF with mo.sql in a cell, and also present that DF in mo.md. If this is undesired, then throwing an exception and stating this requirement would be helpful.

This should be possible. If it's not working I think it's a bug. Let me try to give you an example.
Upon restart/rerun, my cells with mo.sql + Python would convert to SQL. I tried many times to have the .py cell format stick.

The SQL rendering of cells is UI sugar -- the code is still Python. You can click the Python icon in the top right of a cell to view the underlying Python. If the underlying Python code is not the same as what you had, then that is a bug, otherwise it is more of a UI issue. Does that make sense?
Here is an example of using the query result of a SQL cell in markdown:

https://marimo.app/?slug=zv70bc
cell 25 (previously cell 24, now split into 24 and 25**) is an example of where my python code was attempted to be converted into SQL when it should not have been. And when I try to switch it back to python is mis-converts it, so I have g to do it manually. https://marimo.io/p/@michael-day/michael-day-switchyards-case-oct-7
screenshot of the cell. Seems to cut off the top and bottom rows of text.

I'm going to have to fix it now as I am presenting tomorrow morning. Will fix it by breaking out the mo.sql function from the rest of the python code.
Attachment
image.png
Thank you, Iโ€™ll look into this ๐Ÿ™
Add a reply
Sign up and join the conversation on Discord