register data source / dataframe?

At a glance

The community member has a Python call that returns a DataFrame, and they want to register it as a data source without assigning it to a variable in the notebook. The community members discuss a few options:

1. As a workaround, the community member could put the DataFrame in a dictionary in mo.state and then use a dropdown to select the DataFrame.

2. The community members are interested in adding support for registering DataFrames as data sources, and one community member offers to contribute to the implementation. They mention that they would need to handle cleanup, possibly using weakrefs.

3. The community members discuss advanced tooling like mo.editor.register_datasource to support this feature.

4. The community members agree to work out the specification for this feature in a GitHub issue, and one community member files the issue at https://github.com/marimo-team/marimo/issues/2766.

There is no explicitly marked answer in the comments.

Useful resources

RRiya Sinha

Say I have a python call that returns a dataframe. without assigning it to a variable within the notebook itself, is there a way I can register it as a data source? like from a module i'm importing?

Example:

Plain Text

def run_agent(input):
   for event in agent.run(input):
      # if is tabular
      df = pd.DataFrame(event.content)
      **mo.state.register_datasource(df, event.name + event.tool_args + "_df") # <------ saves df to var "nearest_gene_tool_chr15_88569444_df"**
      mo.output.append(df)
      ...
    ...

9 comments

MMyles Scolnick

not currently.

As a workaround: you may be able to put this in a dict in mo.state({}) and then have a dropdown selected_df = mo.ui.dropdown(get_state()) and build your own "data source explorer"

RRiya Sinha

ok! is this something you would be interested in supporting in the longer term?

MMyles Scolnick

yea, i think we could add support for this, esepcially if you are open to contributing it. we would likely need a way to handle cleanup too (maybe just weakrefs)

MMyles Scolnick

i can see a few advanced tooling like mo.editor.register_dasource

RRiya Sinha

yes, i can def help implement it!!

RRiya Sinha

is a github issue the preferred place to work out the spec?

AAkshay

Yea GitHub issue is fine. Just to clarify, what's the main reason you'd like to register as a data source? Is it to see the schema in the data sources panel?

RRiya Sinha

the data sources panel is def one big one, helps summarize to a user what data an agent has fetched already in my use case; the other reason is to then be able to reference them using the @ syntax in the ai generated cells

RRiya Sinha

filed: https://github.com/marimo-team/marimo/issues/2766

Add a reply

Get help from the marimo community

register data source / dataframe?