2021-10-05 weekly planning

[kgodey]

Discussion topics

• Updates: General updates everyone should be aware of.
• Questions: Questions for the team to answer.
• Comms Liaison Schedule: Upcoming comms liaison schedule for the core team.
• Active Work: Work planned for the next week.
• Retrospective: Summary of last week's work.

About

Each Tuesday, we start a discussion about Mathesar's progress. Everyone is welcome to add a new topic to discuss or to comment on existing topics.

[kgodey]

Updates

Staging Server

We finally have a permanent Mathesar server up: https://staging.mathesar.org/. The latest Mathesar code is automatically deployed to it whenever a new pull request is merged.

The server is currently password-protected because we don't have any authentication in the product yet. Please talk to me if you want access.

Progress

We're continuing to make progress on our active milestones:

• Frontend work on [05] Editable Tables is almost complete. A design review of the "adding records" functionality last week resulted in some major design changes, but once those are done, we can close the milestone.
• Frontend, backend, and design work on the October stability release, [06] 2021-10 Stability is ongoing.
• Some frontend work on [07] Working with Tables by community contributors is in review and should be merged soon. All backend work has been completed.
• Backend work on [08] Initial Data Types is in progress. It's a little slower than anticipated because of issues with the functionality involved in setting default values, but those are in review now. We're making steady progress, though, with text types completed and URI types in review.
• Design work on [09] Working with Views is progressing nicely and will hopefully be wrapped up in the next week.

Hiring

Interviews for the backend engineer position are in full swing. We're still accepting applications and referrals are appreciated.

Design

We now have a new repository: https://github.com/centerofci/mathesar-design/ for HTML/CSS design prototypes. For features involving a lot of user interactions, working with HTML prototypes was easier than Figma. We will be incorporating these designs into our in-review and future design specs.

You can see the current state of the prototype at https://mathesar-prototype.netlify.app/. It is still a work in progress.

Community Comms Liaison

We've decided to designate a single person on the core team each work to be on point for community-related communications (Matrix messages, pull request reviews, issue comments, etc.). This person will pull in other people on the team as needed. This arrangement will allow us to respond to community messages in a timely manner without needing everyone on the team to pay attention to messages all the time.

We'll use the weekly discussions to designate the comms liaison each week.

[kgodey]

Questions

@mathemancer, @ghislaineguerin, @pavish: Your responses are encouraged. Others are also welcome to respond.

• What went well over the past week?
• What could have gone better over the past week?
• Does your workload for this week look okay?
• Are there any improvements you'd like to suggest?

[ghislaineguerin]

What went well over the past week?

I've been focusing on design problems for longer periods of time. Also, coordinating w Kriti to send the project list for students.

What could have gone better over the past week?

I still don't have a plan for delivering the UI styling.
I also wish I could contribute more to discussions on our design channel and do more community-facing activities.

Does your workload for this week look okay?

It does.

Are there any improvements you'd like to suggest?

None at the moment.

[pavish]

What went well over the past week?

We got the review completed for add records, which was blocking the PR.

What could have gone better over the past week?

Progress has been slow from my end, due to some unforeseen health concerns. I tried to work through that which delayed progress, it would have been faster and better if I had taken some time off and then worked.

Does your workload for this week look okay?

It does

Are there any improvements you'd like to suggest?

None, at the moment.

[mathemancer]

What went well over the past week?

I'm happy to have gotten some types implemented, and to have made good progress on the server default settings.

What could have gone better over the past week?

I'm trying to take a stance of being quicker to set things down and move to other implementation tasks when something gets blocked or needs clarification, but I think this is resulting in a stack of half-finished things. I'm currently managing these so they don't get to where the current work on them is unusable, but does add some complexity.

Does your workload for this week look okay?

Overall, it's a lot. If everything goes well, it's feasible, but there's at least one blocker since the TIME types probably should be done before the TIMESTAMP types are started (since they overlap quite a bit).

Are there any improvements you'd like to suggest?

None at the moment.

[kgodey]

What went well over the past week?

I'm glad to have the staging server up finally.

What could have gone better over the past week?

I haven't had a lot of time to focus on code or design reviews and the new Mathesar project automation is trickier than I thought it would be. I haven't gotten as much done as I'd like.

Does your workload for this week look okay?

I probably will not be able to get it all done, but that's fine.

Are there any improvements you'd like to suggest?

No.

[kgodey]

Comms Liaison Schedule

Here's our (randomized) comms liaison schedule for the next few weeks:

Week	Person
Oct 6 to Oct 12	@ghislaineguerin
Oct 13 to Oct 19	@mathemancer
Oct 20 to Oct 26	@kgodey
Oct 27 to Nov 2	@pavish

[kgodey]

Active Work

This post lists work planned for this week (see also: Active). It also includes a current summary of milestones involved in this week's work.

[2021-10-08] [05] Editable Tables

• 9 issues already closed.
• Issues currently in progress:
  • [@pavish] Implement adding new records #197

[2021-10-13] [06] 2021-10 Stability

• 7 issues already closed.
• Issues currently in progress:
  • [@ghislaineguerin] General styling updates #473
  • [@pavish] Refactor frontend stores for table-data #651
  • [@pavish] Set up integration testing environment #639
  • [@kgodey] Schema sqlalchemy engine caching is per-instance #280
  • [@kgodey] Refactor db code to work with existing connections #592
  • [@bohemia420] Deleting all columns in a table results in a NoSuchTableError #668
  • [@bohemia420] Table, Database, & Schema APIs should support sorting by ID & name #635
• Issues not yet started:
  • [@kgodey] Set up logging in the backend code #583
  • [@ghislaineguerin] Design for loading, error, and retry states #636
  • [@ghislaineguerin] Review existing UI and create tickets for changes needed #638
  • [@pavish] Write integration tests for some important user scenarios #640
  • [@pavish] Unit tests for stores #641
  • [@pavish] Unit tests for all reusable components #642
  • [@pavish] Enable TypeScript strict mode #619

[2021-10-15] [07] Working with Tables

• 22 issues already closed.
• Issues currently in progress:
  • [@mr-gabe49] Implement deleting a table #448
  • [@dmos62] Implement showing and changing a column's type #498
  • [@mr-gabe49] Implement deleting a column #496
• Issues remaining in backlog:
  • 10 frontend issues

[2021-10-22] [08] Initial Data Types

• 33 issues already closed.
• Issues currently in progress:
  • [@mathemancer] Create URI custom type and handle it in the backend. #412
  • [@mathemancer] Some server_default constants are not handled #682
  • [@mathemancer] Unknown server_default settings silently deleted #681
  • [@mathemancer] Backend work for Mathesar money custom type  #417
  • [@eito-fis] Handle TIME data type in the backend #426
  • [@mathemancer] Implement filtering options for Number types. #385
  • [@mathemancer] Handle BOOLEAN data type in the backend #387
• Issues not yet started:
  • [@mathemancer] Implement range based grouping for numeric types. #384
  • [@mathemancer] Handle TIMESTAMP data type in the backend #424
  • [@mathemancer] Handle INTERVAL type in the backend #430
  • [@mathemancer] Handle MONEY data type in the backend. #420
  • [@mathemancer] Implement filtering options for the email type #410
  • [@kgodey] Store icon, labels, and input options for types on the backend #661
  • [@kgodey] Set up Column Django model with support for display options #658
• Issues remaining in backlog:
  • 16 backend and database issues
  • 9 frontend issues
  • 8 product issues

[2021-10-31] [08A] 2021-10 improvements

• Issues currently in progress:
  • [@arvind-1] Column API should support creating columns without names  #555
  • [@kgodey] Move to new GitHub Project setup #685
  • [@silentninja] Handle non-unicode encoded datafiles #421
• Issues not yet started:
  • [@pavish] Move filter pane to dropdown #643
• Issues remaining in backlog:
  • 6 backend and database issues
  • 12 frontend issues
  • 3 infrastructure issues

[2021-11-05] [09] Working with Views

• 4 issues already closed.
• Issues currently in progress:
  • [@ghislaineguerin] Support for listing database views #455
  • [@ghislaineguerin] Design for many-to-many relationships between tables  #443
  • [@ghislaineguerin] Support for opening and interacting with a View #466
  • [@ghislaineguerin] Support for deleting a database view #454
  • [@ghislaineguerin] Design for usage of Foreign Key constraints #243
  • [@ghislaineguerin] Support for editing records within a view #456
• Issues remaining in backlog:
  • 7 backend and database issues
  • 2 product issues

[2021-11-12] [10] Data Modeling

• 4 issues already closed.
• Issues currently in progress:
  • [@rajeshkumarrs] Default primary key column should be id #394
• Issues remaining in backlog:
  • 3 backend and database issues
  • 2 frontend issues
  • 2 design issues

[2021-11-19] [11] Multiple Database Support

• 7 issues already closed.
• Issues currently in progress:
  • [@aaditkamat] Backend support for tables without single-column primary keys #395
• Issues remaining in backlog:
  • 3 backend and database issues
  • 3 frontend issues
  • 2 design issues

[kgodey]

Retrospective

This post lists issue updates from 2021-09-28 until now by milestone.

[05] Editable Tables

• Issues started, not yet finished:
  • [@pavish] Implement adding new records #197

[06] 2021-10 Stability

• Issues closed:
  • [@kgodey] Columns API is failing on staging #687
  • [@mr-gabe49] Cancel import does not eliminate table under the left pane #646
  • [@kgodey] Set up staging environment for Mathesar #225
  • [@pavish] Frontend unit test cases: Readonly table view #213
• Issues started, not yet finished:
  • [@ghislaineguerin] General styling updates #473
  • [@pavish] Refactor frontend stores for table-data #651
  • [@pavish] Set up integration testing environment #639
  • [@kgodey] Schema sqlalchemy engine caching is per-instance #280
  • [@kgodey] Refactor db code to work with existing connections #592
  • [@bohemia420] Deleting all columns in a table results in a NoSuchTableError #668
  • [@bohemia420] Table, Database, & Schema APIs should support sorting by ID & name #635
• New issues created:
  • Columns API is failing on staging #687, created by @kgodey

[07] Working with Tables

• Issues closed:
  • [@pegnott] Implement adding a new column #495
• Issues started, not yet finished:
  • [@mr-gabe49] Implement deleting a table #448
  • [@dmos62] Implement showing and changing a column's type #498
  • [@mr-gabe49] Implement deleting a column #496

[08] Initial Data Types

• Issues closed:
  • [@mathemancer] Handle CHAR data type in the backend. #407
  • [@mathemancer] Handle TEXT data type in the backend. #405
  • [@mathemancer] Handle VARCHAR data type in the backend. #404
• Issues started, not yet finished:
  • [@mathemancer] Create URI custom type and handle it in the backend. #412
  • [@mathemancer] Some server_default constants are not handled #682
  • [@mathemancer] Unknown server_default settings silently deleted #681
  • [@mathemancer] Backend work for Mathesar money custom type  #417
  • [@eito-fis] Handle TIME data type in the backend #426
  • [@mathemancer] Implement filtering options for Number types. #385
  • [@mathemancer] Handle BOOLEAN data type in the backend #387

[08A] 2021-10 improvements

• Issues started, not yet finished:
  • [@arvind-1] Column API should support creating columns without names  #555
  • [@kgodey] Move to new GitHub Project setup #685
  • [@silentninja] Handle non-unicode encoded datafiles #421
• New issues created:
  • Tests that depend on schema fixture in import/test_csv throws up integrity error when run individually #689, created by @silentninja
  • Move to new GitHub Project setup #685, created by @kgodey

[09] Working with Views

• Issues started, not yet finished:
  • [@ghislaineguerin] Support for listing database views #455
  • [@ghislaineguerin] Design for many-to-many relationships between tables  #443
  • [@ghislaineguerin] Support for opening and interacting with a View #466
  • [@ghislaineguerin] Support for deleting a database view #454
  • [@ghislaineguerin] Design for usage of Foreign Key constraints #243
  • [@ghislaineguerin] Support for editing records within a view #456

[10] Data Modeling

• Issues started, not yet finished:
  • [@rajeshkumarrs] Default primary key column should be id #394

[11] Multiple Database Support

• Issues started, not yet finished:
  • [@aaditkamat] Backend support for tables without single-column primary keys #395

[13A] 2021-11 improvements

• New issues created:
  • Bump tmpl to latest version when npm audit raises alert #683, created by @pavish