Shared roster are duplicated and falls to Not Found after migrating to v21.01

[ibaklykov]

We upgraded our ejabberd from 18.12 to 21.01 and encountered a problem.
When we make some changes in a shared roster and save it, the roster are duplicated.
Both in our mysql database and in the web GUI duplicate lines appear and if you click on any of them you will see Not Found message. Also this shared roster disappears from client apps (became invisible).
Please advice how to fix it. Any ideas would be helpful.

[badlop]

Can you update your post detailing:

• the options you have in ejabberd.yml related to mod_shared_roster, or any other that may affect it
• what exact ejabberd version you are using: how did you install it, from where, and if you added custom source code changes, modules, etc that may affect it
• exact reproduction steps: how can i reproduce exactly what you do to reproduce the problem? do you use webadmin or ejabberdctl? what exact arguments?

[ibaklykov]

@badlop Yes, this is very similar to my case, but my ejabberd didn't crash.

[badlop]

Ah, sorry: when I said "ejabberd crashes" I wanted to say "ejabberd shows some error messages in the log file". Obviously it doesn't crash compltetely :)

PD: still compiling old ejabberd versions to check when the problem was introduced...

[badlop]

I'll continue investigating tomorrow.

ejabberd 20.07 seems to work correctly.
20.12 fails like this when clicking on "Submit" button on WebAdmin of a shared roster group options page:

2021-05-18 17:49:20.349863+02:00 [error]
Hook webadmin_page_host crashed when running mod_shared_roster:webadmin_page/3:
** exception error: bad argument
   in function  lists:keysearch/3
      called as lists:keysearch(label,1,error)
      *** argument 3: not a list
   in call from mod_shared_roster:get_opt/3 (/home/bernar/e/git/ejabberd/src/mod_shared_roster.erl, line 1153)
   in call from mod_shared_roster:shared_roster_group/4 (/home/bernar/e/git/ejabberd/src/mod_shared_roster.erl, line 979)
   in call from mod_shared_roster:webadmin_page/3 (/home/bernar/e/git/ejabberd/src/mod_shared_roster.erl, line 897)
   in call from ejabberd_hooks:safe_apply/4 (/home/bernar/e/git/ejabberd/src/ejabberd_hooks.erl, line 236)
   in call from ejabberd_hooks:run_fold1/4 (/home/bernar/e/git/ejabberd/src/ejabberd_hooks.erl, line 217)
   in call from ejabberd_web_admin:process_admin/3 (/home/bernar/e/git/ejabberd/src/ejabberd_web_admin.erl, line 551)
   in call from ejabberd_http:process/2 (/home/bernar/e/git/ejabberd/src/ejabberd_http.erl, line 373)

[ibaklykov]

ОК. Thank you. I'll be waiting for your conclusion.

[prefiks]

@badlop What group name you used? I can't reproduce that with master version. I can get this error when i change url to use unknown group name, but it works for me when using something else, maybe question of some normalization?

[prefiks]

Ok, i see what we get duplicate entries in sr_group, we don't have group name as primary key/unique index on it, so replace statement that we use to update doesn't work correctly here.

[prefiks]

I commited 95fa43a that adds those missing indexes. You can add them with

CREATE UNIQUE INDEX i_sr_group_name ON sr_group(name); 

or if you use new schema (when there is server_host column in sr_group table:)

CREATE UNIQUE INDEX i_sr_group_sh_name ON sr_group(server_host(191), name);

[prefiks]

And this did occur just now because, code that did upsert on mysql war changed from doing delete then insert to use replace statement, which needs primary keys to deteremine rows that need to be updated

[ibaklykov]

Thanks a lot! Adding an index (first one in my case) solved the problem.

[badlop]

Great! Your fix works for me, both updating the previous database, or creating a new one.

The commit that introduced this problem is 5b0f0d8 ? In that case, it didn't affect any ejabberd release.

Anyway, the SQL update procedure will be mentioned in the next release announcement.