Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing indexes for foreign keys on postgres #3509

Merged
merged 1 commit into from
Nov 15, 2023
Merged

Conversation

johha
Copy link
Contributor

@johha johha commented Nov 14, 2023

Mysql automatically creates indexes for foreign keys. This makes the execution of joins, eager loading etc. faster.

Conversely, in Postgres indexes for for foreign keys aren't automatically created. This change adds the missing indexes for Postgres.

The migration has been created with the help of the following psql function which shows the foreign keys with missing indexes:

--
-- function:    missing_fk_indexes2
-- purpose:     List all foreing keys in the database without and index in the referencing table.
--              The listing contains create index sentences
-- author:      Based on the work of Laurenz Albe
-- see:         https://www.cybertec-postgresql.com/en/index-your-foreign-key/
--
create or replace function missing_fk_indexes2 ()
returns setof varchar
language sql as $$
  select
    -- create index sentence
    'create index on ' ||
    tc.conrelid::regclass ||
    '(' ||
    string_agg(ta.attname, ', ' order by tx.n) ||
    ')' as create_index

  from pg_catalog.pg_constraint tc

  -- enumerated key column numbers per foreign key
  cross join lateral unnest(tc.conkey) with ordinality as tx(attnum, n)

  -- name for each key column
  join pg_catalog.pg_attribute ta on ta.attnum = tx.attnum and ta.attrelid = tc.conrelid

  where not exists (
    -- is there ta matching index for the constraint?
    select 1 from pg_catalog.pg_index i
    where
      i.indrelid = tc.conrelid and
      -- the first index columns must be the same as the key columns, but order doesn't matter
      (i.indkey::smallint[])[0:cardinality(tc.conkey)-1] @> tc.conkey) and
      tc.contype = 'f'
    group by
      tc.conrelid,
      tc.conname,
      tc.confrelid
    order by
      pg_catalog.pg_relation_size(tc.conrelid) desc
$$;

Which can be executed like this:

select * from missing_fk_indexes2();
  • I have reviewed the contributing guide

  • I have viewed, signed, and submitted the Contributor License Agreement

  • I have made this pull request to the main branch

  • I have run all the unit tests using bundle exec rake

  • I have run CF Acceptance Tests

@johha johha marked this pull request as ready for review November 14, 2023 14:43
Copy link
Member

@FloThinksPi FloThinksPi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the current way this creates indices will do them in one transaction in PSQL

db/migrations/20231114130216_add_missing_fk_indexes.rb Outdated Show resolved Hide resolved
db/migrations/20231114130216_add_missing_fk_indexes.rb Outdated Show resolved Hide resolved
@johha johha force-pushed the missing-fk-indexes branch 2 times, most recently from f8173e6 to c9b6ca3 Compare November 15, 2023 11:39
Mysql automatically creates indexes for foreign keys.
This makes the execution of joins, eager loading etc. faster.

Conversely, in Postgres indexes for for foreign keys aren't automatically created.
This change adds the missing indexes for Postgres.

The migration has been created with the help of the following psql
function which shows the foreign keys with missing indexes:

```
--
-- function:    missing_fk_indexes2
-- purpose:     List all foreing keys in the database without and index in the referencing table.
--              The listing contains create index sentences
-- author:      Based on the work of Laurenz Albe
-- see:         https://www.cybertec-postgresql.com/en/index-your-foreign-key/
--
create or replace function missing_fk_indexes2 ()
returns setof varchar
language sql as $$
  select
    -- create index sentence
    'create index on ' ||
    tc.conrelid::regclass ||
    '(' ||
    string_agg(ta.attname, ', ' order by tx.n) ||
    ')' as create_index

  from pg_catalog.pg_constraint tc

  -- enumerated key column numbers per foreign key
  cross join lateral unnest(tc.conkey) with ordinality as tx(attnum, n)

  -- name for each key column
  join pg_catalog.pg_attribute ta on ta.attnum = tx.attnum and ta.attrelid = tc.conrelid

  where not exists (
    -- is there ta matching index for the constraint?
    select 1 from pg_catalog.pg_index i
    where
      i.indrelid = tc.conrelid and
      -- the first index columns must be the same as the key columns, but order doesn't matter
      (i.indkey::smallint[])[0:cardinality(tc.conkey)-1] @> tc.conkey) and
      tc.contype = 'f'
    group by
      tc.conrelid,
      tc.conname,
      tc.confrelid
    order by
      pg_catalog.pg_relation_size(tc.conrelid) desc
$$;

```

Which can be executed like this:

```
select * from missing_fk_indexes2();
```

Co-authored-by: Johannes Haass <[email protected]>
@johha johha force-pushed the missing-fk-indexes branch from c9b6ca3 to 5bb3fe9 Compare November 15, 2023 13:07
@johha johha merged commit b6ca2fb into main Nov 15, 2023
5 checks passed
@johha johha deleted the missing-fk-indexes branch November 15, 2023 13:26
johha added a commit that referenced this pull request Nov 20, 2023
In some cases it might happen that the returned results have the wrong
order, e.g. when running cc with postgres 10 and special foreign key
indexes (commit: 5bb3fe9 /  #3509)

This commit adds a '.order' for all user related models to ensure the
correct order.
johha added a commit that referenced this pull request Nov 20, 2023
In some cases it might happen that the returned results have the wrong
order, e.g. when running cc with postgres 10 and special foreign key
indexes (commit: 5bb3fe9 /  #3509)

This commit adds a '.order' for all user related models to ensure the
correct order.
@johha
Copy link
Contributor Author

johha commented Nov 20, 2023

This change indeed broke the unit tests on postgres 10 due to a random order in the result sets. While it's unclear why this is only happening on postgres 10, a valid fix is to explicitly order the results -> #3518

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants