bug: Errors using create_table and insert with overwrite=True

[augcollet]

What happened?

Hello,

I have noticed quite recurring problems when I activate the overwrite feature.

Exemple with create_table :

import ibis
from ibis import _
import os

table='test_table'
schema='test'

con=ibis.postgres.connect(
    user=os.getenv('POSTGRES_USER'),
    password=os.getenv('POSTGRES_PASSWORD'),
    host="postgres",
    port=os.getenv('POSTGRES_PORT'),
    database=os.getenv('POSTGRES_DB') 
)

ibis.set_backend(con)

t = ibis.memtable({
    'A': [1,2,3], 
})

con.create_table(table, obj=t, database=schema, overwrite=True)
con.table(table, database=schema).execute()

I systematically get error messages :

The only solution is to drop the table to recreate it without overwrite.

Exemple with insert :

import ibis
import os
from ibis import _

table='test_table'
schema='test_case'

con=ibis.postgres.connect(
    user=os.getenv('POSTGRES_USER'),
    password=os.getenv('POSTGRES_PASSWORD'),
    host="postgres",
    port=os.getenv('POSTGRES_PORT'),
    database=os.getenv('POSTGRES_DB') 
)

ibis.set_backend(con)

t = ibis.memtable({
    'key': [1,2,3,4,5], 
})

print('Create table with following data')
display(t.execute())
con.drop_table(table, force=True, database=schema)
con.create_table(table, obj=t, database=schema, overwrite=False)

print('Replace table data with')
t=t.filter(_['key']>3)
display(t.execute())

print(f'(Existing tables {con.list_tables(database=schema)})')
con.insert(table, obj=t, overwrite=True, database=schema)

Ibis fails because the table does not exist, while it does exist (it appears even if we list the tables).

Here too, I have to recreate the table to avoid the error.

At the same time, I occasionally observe other more obscure problems that I cannot reproduce on minimal cases, but I suspect them to be partly linked to the behaviors described above. It would be really great if a fix could be provided! I am ready to help if needed.

Thanks in advance.

Regards,

What version of ibis are you using?

Name: ibis-framework
Version: 9.5.0
Summary: The portable Python dataframe library
Home-page: https://ibis-project.org/
Author: Ibis Maintainers
Author-email: [email protected]
License: Apache-2.0
Location: /home/user/.local/lib/python3.12/site-packages
Requires: atpublic, parsy, python-dateutil, pytz, sqlglot, toolz, typing-extensions
Required-by:

What backend(s) are you using, if any?

PostgreSQL 16.4 (Debian 16.4-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit

Relevant log output

Code of Conduct

• I agree to follow this project's Code of Conduct