GSOC-2020: User Collection-BookBrainz : Prabal

prabalsingh24 · February 22, 2020, 10:16am

Personal information

Name: Prabal Singh
IRC Nick: prabal
MB username: prabalsingh24
Email: praba170107045@iitg.ac.in
Github: prabalsingh24
Time Zone. UTC+05:30

Proposal

Project overview

Bookbrainz lacks a feature which allows a person to make a collection of entities (edition, edition-group, work, author, publisher) for his/her future reference. This project is about introducing such feature - User Collections.
Music Library : Playlist :: Bookbrainz : User Collection

Features and UI prototype

Basic Features :

A user can create a collection of a particular entity type. Author-Collection, Work-Collection and can store the entities of that type in that collection.
A user can change the privacy of their collection thus controlling whether other people can see the collection.
A user can makes changes to collections created by themselves or collections whose collaborator they are.
User can add collaborators to his/her own collections - giving your friends access to edit and make changes in the collection.
A collaborator can edit the collection - add more entities, change the description, privacy, name but is not allowed to delete the collection neither is he/she allowed to add more collaborators. That can only be done by the owner of the collection.

UI Prototype :

There will be another tab in the profile page - collections, which will show all the collections of a user in a table. Since an user can have multiple many collections, the collections be displayed using pagination.

Create New Collection button will open the create new collection form which is described later.
Clicking on the row anywhere would redirect to that particular collection.

Inside a particular collection, you can see the entities in the tabular view with pagination. You have the option to Edit the collection, Delete the collection, and to Add/Remove the entities.
Each type of collection - author collection, work collection, edition collection - will have their releveant columns in the table

Work Collection

Edition Collection

Publisher Collection

Author Collection

Clicking on any of the row will redirect to that entity-view page
Clicking add author will open a pop up, in which one can search for the entity and add it to the collection

Clicking ‘Edit’ will open a page in which you can edit the information about the collection - Name, Type, Collaborator. This page will be same as Create New Collection page.

editcollection1519×856 77.6 KB

Adding entities to your collection

In order to add entities in the collection Add to Collection button will be placed in both search results and entity page.
By making more buttons we make it generic and we can add other things like - merge share etc
Bulk edition of entities will also be possible using the checkboxes

If the user is not logged in, - User will be redirected to login. ( Same happens with merge button )
When the user is logged in , a list of user’s existing collections will be shown filtered by the entity type. Clicking them will add the entity to that particular collection.
Also New Collection option will be there, clicking it will display a form for new colletion, and then clicking add will add that entity to that new collection

addToCollection1519×856 125 KB

CreateCollection21919×1079 168 KB
Since the space is less in this create collection form, we can omit adding collaborators here.

Stats in the profile page will be updated like this

There will be a page which will show all the Public Collections. /collections.
There will be a search collections bar in this page as well.

Database Changes

Three new tables will be added in the existing database like this.

user_collection : this table will have all the information about a particular collection. editor_id is the owner of the collection, the editor who created the collection.
user_collection_item : this table will store the collection_id and bbid which describes the entites in the collection.
user_collection_collaborator : this table stores collection_id and editor_id of their collaborator

CREATE TABLE bookbrainz.user_collection (
	collection_id UUID PRIMARY KEY DEFAULT public.uuid_generate_v4(),
	editor_id INT,
	name VARCHAR(80) NOT NULL CHECK (name <> ''),
	description TEXT NOT NULL CHECK (description <> ''),
	type bookbrainz.entity_type NOT NULL
	public BOOLEAN NOT NULL DEFAULT FALSE,
	created_at TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT timezone('UTC'::TEXT, now()),
       last_modified TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT timezone('UTC'::TEXT, now()),
	CONSTRAINT user_collection_fk FOREIGN KEY (editor_id) REFERENCES bookbrainz.editor(id)
);

CREATE TABLE bookbrainz.user_collection_item (
	collection_id UUID,
	bbid UUID,
        added_at TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT timezone('UTC'::TEXT, now()),
	CONSTRAINT user_collection_item_fk FOREIGN KEY (collection_id) REFERENCES bookbrainz.user_collection(collection_id) ON DELETE CASCADE,
	CONSTRAINT user_collection_item_fk1 FOREIGN KEY (bbid) REFERENCES bookbrainz.entity(bbid)
);

CREATE TABLE bookbrainz.user_collection_collaborator (
	collection_id UUID ,
	editor_id INT,
	CONSTRAINT user_collection_collaborator_fk FOREIGN KEY (collection_id) REFERENCES bookbrainz.user_collection(collection_id) ON DELETE CASCADE,
	CONSTRAINT user_collection_collaborator_fk1 FOREIGN KEY (editor_id) REFERENCES bookbrainz.editor(id)
);

ORM BookBrainz-Data Changes

Two new models will be added in BookBrainz-data
UserCollectionsItem, UserCollections , UserCollectionsCollaborator

const UserCollections = bookshelf.Model.extend({
		items(){
                       return this.hasMany('UserCollectionItem','collection_id');
                }
                collaborators(){
                       return this.hasMany('UserCollectionCollaborator', 'collection_id');
                }
                owner(){
                       return this.belongsTo('Editor','editor_id');
                }
		tableName: 'bookbrainz.user_collection'
	});

const UserCollectionItem = bookshelf.Model.extend({
                entity() {
			return this.belongsTo('Entity', 'bbid');
		},
		tableName: 'bookbrainz.user_collection_item'
	});

const UserCollectionCollaborator = bookshelf.Model.extend({
	collaborator() {
		return this.belongsTo('Editor', 'editor_id');
	},
	collections(){
		return this.belongsTo('UserCollection', 'collection_id');
	}
	tableName: 'bookbrainz.user_collection_collaborator'
});

Editor model will be modified little bit. These two relation will be added in it

collections() {
	return this.hasMany('UserCollections', 'editor_id');

},
collaborators(){
	return this.belongsToMany('UserCollectionCollaborator', 'editor_id');
}

Backend Server Changes

I will add a new router file in src/server/routes/collection. This new router file will have following routes.

Router.get(‘collection/:collectionId’) and Router.get('collection/:collectionId/paginate'):
Router.get(‘collection/:collectionId/delete’)
Router.post(‘collection/:collectionId/delete/handler’)
Router.get(‘collection/create’)
Router.post(‘collection/create/handler’)
Router.get(‘collection/:collectionId/edit’)
Router.post(‘collection/:collectionId/edit/handler’)
Router.post('collection/:collectionId/add')
Router.post('collection/:collectionId/remove')
Router.post('collection/:collectionId/collaborator/add)
Router.post('collection/:collectionId/collaborator/remove)
Router.get('collections') and Router.get('collections/collections')
Router.get('collections/search') and `Router.get(‘collections/search/search’)

Apart from this, /editor/:id/collections and editor/:id/collections/collections will be added in the editor route

Router.get(‘collection/:collectionId):

This route will display a particular collection.
First of all, user will be authenticated to check if user can see the collection ( user is either collaborator or owner or collection is public). For this a function-isAuthenticatedForCollection(req, res, collectionId) will be created.
The results will be in a table and in a paginated view – PagerElement -will be reused here.
Collections of different types will be having different columns in the table.
I plan to make only one table -CollectionView (different that CollectionsTable ) - and reuse it for all the collection-types. I will achieve this by using boolean field - ShowLanguage, ShowDateofBirth, etc. It will be similar to the revisions table I made earlier where I achieved similar results.

In the server side, data will be fetched like this

const entitiesArray = await new UserCollections()
			.where('collection_id', collectionId)
			.fetchPage({
				limit: size,
				offset: from
                                withRelated: ['items', "items.entity"]
			});

The entitesArray will be having bbid and the entity.
In case the entity is deleted, it’s name will be fetched like this.

if (!entity.dataId) {
	entity.deleted = true;
	const parentAlias = await orm.func.entity.getEntityParentAlias(
					orm, modelName, bbid);
	entity.parentAlias = parentAlias;
}

As for the frontend part, I will make a react component - CollectionsPage which will be in src/client/components/pages/collections
The UI would be similar to described in the earlier section

router.get(‘/editor/:id/collections’) and `router.get('editor/:id/collections/collections):

This route will display all the collections of a user (both owner and collaborator)
The first route will be send a markup whereas second route will be used for updating the data (pagination)
getCollectionsForEditor() function will be made which will be used in both these routes.
Inside this function, collections can be fetched like this

const collectionIds = await new UserCollectionCollaborator()
                                               .where('editor_id', editorId)
                                               .fetchAll({});

collectionIds array have collection_id of collections of which the editor is a collaborator

const collections = await new UserCollections()
   		.query((qb) => {
   			qb.where('editor_id', editorId)
                       .orWhere('collection_id', 'in', collectionIds);;
   		})
                .orderBy('created_at', 'DESC');
   		.fetchPage({
                     limit: size,
                     offset: from,
   		     withRelated: ['Items', 'Collaborators']
   		});

Since the number of collections of an user can be high, this table will be displayed with pagination. The PagerElement will be reused for this.
For the front-end part, I will make a corresponding page in src/client/components/pagers/editor-collection and also add corresponding file in controller to make JS file.
I will create a react component - CollectionsTable - to display the collections. I will be reusing this table in 'See All Collections` page also.
This CollectionsPage will be very much similar to RevisionsPage which I made in this PR

Router.get(‘collection/:collectionId/delete’) :

A DeletionForm will be displayed in this route.
I plan create CollectionDeletionForm which will be stored in src/client/components/forms/collectionDeletion
Depending on whether the user is owner or a collaborator, the details ( text of the form) will change.
Ths form will be confirmation of the delete and hitting delete will make post call to 'collection/:collectionId/delete’

Router.post('collection/:collectionId/delete/handler):

If user is owner of the collection, collection will be deleted,

await new UserCollections({collectionId: collectionId}).destroy();

Otherwise if the user is collaborator, the particular row in the user_collection_collaborator will be deleted - user will no longer be the collaborator of the collection .

await new UserCollectionCollaborator({
	collectionId: CollectionID,
	editorId: editorID
}).destroy();

Router.get(‘collection/create’) :

User must be logged in for this. Already defined isAuthenticated function from auth.js will be used here to check this
The Name , Description, Type and Privacy will be filled/edited in a form. This new CollectionForm will be created in src/client/components/forms/
The ‘Save’ button will make a post request to collection/create/handler

Router.get('collection/:collectionId/edit)

Just like in the above routes, isAuthenticated will be used to check if the user is logged in or not.
Because user needs to be either a collaborator or the owner of the collection, another function will be made isAuthenticatedForCollectionEditing(req, res, collectionId) which will be checking this
The ‘Save’ button will make a post request to collection/collectionId/edit/handler

Router.post(‘collection/create/handler’) and Router.post(‘collection/:collectionId/edit/handler’):

Both of these routes would be similar to each other, that they would use the same function createOrEditHandler in it. .
The params recieved from the post call will be used to make changes/create new collection

const collection = await UserCollections.forge({collectionId: collectionID}).fetch({require: true});
collection.set('name', name)
	.set('description', description)
	.set('public', privacy)
	.save()

Router.post('collection/:collectionId/collaborator/add) and Router.post('collection/:collectionId/collaborator/remove) :

User will be authenticated first to make sure if they’re the owner of the collection.
EditorId will be received as params and the change ( add/remove ) will be done. ( adding or removing a row in user_collection_collaborator table using the bookshelf model

Add to Collection button and Router.post('collection/:collectionId/add') :

For this a AddToCollection react component will be made which will be added in entity view and search result page (As shown in the UI ).
It will display all the collections of users( with entityType=entity).
These collections can be fetched like this

const collectionIds = UserCollectionCollaborator()
                                               .where('editor_id', editorId)
                                               .fetchAll({});

The collectionIds array will have all the collection_id of collections of which the editor is a collaborator.

const collections = await new UserCollection()
   		.query((qb) => {
   			qb.where('editor_id', editorId).orWhere('collection_id', 'in', collectionIds);
                               qb.where('type', entityType);
   		})
   		.fetch({
   			withRelated: ['items', 'collaborators']
   		});

Clicking Add button will then send a post request to 'collection/add' with bbid of the entity and collectionId of the collection, and it will return a message - success or failure ( in case of duplicate entities in collection, user not authorized ( isAuthenticatedForEditing) , etc)
Since user can also create new collection then and there also, clicking New Collection will display a Collection Form to create a collection.
If user creates a new collection, Collection Name and Collection Description will also be sent along with bbid in the post request call.

Remove selected entities button and Router.post('collection/:collectionId/remove') :

Clicking Remove selected entities in the collection-view page will make a post request to collection/:collectionId/remove.
The bbid of the entities to be removed will be sent as params.
In the Router.post('collection/:collectionId/remove) , the user will be authenticated to check if he/she is allowed to make the change (isAuthenticatedForCollectionEditing) and then entities will be removed from the collection.

Router.get('collections'') and `Router.get(‘collections/collections’) :

In this route, all the public collections will be displayed. This is gonna be very similar to the revisions page I created. Obviously pagination will be required hence Pager-Element will be reused.
The ‘collections’ route will be sending a markup whereas ‘collections/collections’ route will be for updating the data in the table ( for pagination).
Both Router.get('collections') and Router.get('collections/collections') will be using a function getOrderedCollections() to get the data.
Inside the getOrderedCollections() function, data will fetched something like this.

const collections = await new UserCollections()
                .where('public', true)
                .orderBy('last_modified', 'DESC')
		.fetchPage({
			limit: size,
			offset: from,
			withRelated: [
				'items', 'collaborators'
			]
		});

For the client side, a React Component CollectionsPage will be also be made.
Pager Element and CollectionsTable will be reused here.

Router.get('collections/search'') and `Router.get(‘collections/search/search’) :

This route will be used to display the search results.
The CollectionTable created in the above route (collections) will be used and Pager element will be used.
As for the search feature, I would be working along with the mentor ( Mr Monkey) to achieve this.
I expect this to be quite similar to existing search features in the website.

TIMELINE

Here is a more detailed week-by-week timeline of the 13 weeks GSoC coding period to keep me on track

Each of the following sub tasks/week-tasks will begin by first writing tests using chai and mocha and then these routes/functions will be written to pass those tests.
54870429-b6612e00-4dcc-11e9-8e08-b8ed902d523f
The complete documentation will be required in this project and will be done at the time of coding itself, we will use JSDoc to document the code

Each of the week would be write tests->write production code → do documentation

Pre-Community Bonding Period ( April ) :
I will solve url history bug that was discussed in irc a while ago.
Because when you are in page 5 of collection-view page and you make changes in the entities ( add or remove), you would ideally want to be in page 5 after the change as well. Right now the url history is not updated when you use the Pager Element

Community Bonding Period :

I will spend some time learning about test-driven development using frameworks like chai and mocha. I will get familiar with debugging tools and learn how to use the react devtools browser extension.

Week 1 :

I will begin with setting up database (Creating new table) and writing corresponding Models in bookbrainz-data.

Week 2 :

This week will be about finalizing UI mockups, making sure we don’t take any bad decision at the early stage

Week 3-4

Router.get(‘collection/create’)
Router.post(‘collection/create/handler’)

Phase 1 Evaluation here

Week 5-7 :

Router.get(‘collection/:collectionId/edit’)
Router.post(‘collection/:collectionId/edit/handler’)
Router.post('collection/:collectionId/collaborator/add')
Router.post('collectoin/:collectionId/collaborator/remove')

Week 8 :

Router.get(‘collection/:collectionId/delete’)
Router.post(‘collection/:collectionId/delete/handler’)

Phase 2 Evaluation here

Week 9 :

Router.get(‘/editor/:id/collections’).
Router.get(‘collection/:collectionId) :

I feel Week 9 will be similar to ‘router.get(’/editor/:id/revisions) which I’ve done in this PR

Week 10 :

Add to Collection feature in search page and entity page
Router.post('collection/:collectionID/add')
Remove Selected Entities feature in the collection-view page.
Router.post('collection/:collectionID/remove)

Week 11:

Collection Search Feature
Router.get(‘collections/search’)

Week 12

Router('collections') which displays all the public collections.
Adding Collection in user stats

Week 13:

Buffer Period

Stretch Goal

The addition of collection endpoints in the existing API.

About Me

I am 3rd year student in IIT Guwahati. I got to know about Metabrainz from one of my senior’s blog - Sambhav Kothari’s Blog. I got interested in BookBrainz because of my interest in reading story books and ofcource familiarity with the technologies used in the project. I have been contributing in BookBrainz project since January. Here is list of my PRs I’ve made so far.

Tell us about the computer(s) you have available for working on your SoC project!
I have an asus laptop with i7 processor with 8gb of ram.
When did you first start programming?
I wrote Hello World in my computer science class in high school.
What type of books do you read?
I enjoy reading detective novels - Agatha Christie novels are my favourite. I’ve also read Harry Potter, LOTR, The Hunger Games
Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
I’ve contributed to Bookbrainz only. I’ve made 35 PRs since January.
RevisionsPage, Editor-Revisions, Pager-Element, etc
How much time do you have available, and how would you plan to use it?
I am free most of the summer. I plan to give 35-40 hours a week in the project.
My classes will probably start in last 10 days. Work load will be less and I will be able to give 4-5 hours a day easily.

prabalsingh24 · February 22, 2020, 10:20am

Is there any way I can make this private till I finish writing this?

Freso · February 22, 2020, 11:33am

You can prepare it in a document off the forum and paste it to the forum when you’re ready.

Also, note that you’re not currently following the submission template.

prabalsingh24 · February 23, 2020, 1:34am

yeah thanks I will do that. Yeah I know about the submission template

prabalsingh24 · February 27, 2020, 12:24am

Hi, this is an initial draft of my proposal, Let me know what you think of this
@mr_monkey

mr_monkey · March 11, 2020, 1:15pm

Hi @prabalsingh24, thanks for your patience
Overall, it’s a pretty solid proposal !
A few sticking points which I detail below. I tried to collect by theme, but don’t hesitate to ask questions if any point needs clarification.

Pages design

I don’t think we need to show the collectionID when presenting a user’s collections.

collectionsView page:

Why show relationships? I’m not sure I understand what’s going on there.

I suppose it’s a collection of Authors and I should ignore the big Work icon?

I like the ability to add items to a collection from search page and entity display page, that’s good.

I also like the idea of a dropdown on the search results.

CreateCollection is missing a dropdown to select the collection’s entity type

I’d love for the collection page to allow adding and removing items directly on the page, without having to click on 'edit’.

I suppose in that case clicking ‘edit’ will allow only to edit the title, description and public/private flag.

The ability to remove items in bulk would also be nice.

I’m not sure I like the idea of the collection create/edit page looking like the relationship editor, but we could talk more about the design at the start of the project

It would be nice to have another paginated page showing all public collections (similar to the revisions page you made)

DB schema changes.

For collectionID Instead of serial number use BBID (like MusicBrainz does)

We use sets in the revision-based schema to describe a group of items (aliases, relationships, etc.) at a given revision.

Considering we don’t need revisions for the collections, I don’t think sets make sense here.

In any case, ‘user_collection_set’ seems like the wrong name for what you describe.

If I understand correctly it describes an entry in a collection, rather than a set of collections.

The collection itself is conceptually your ‘set’: a group of things.

You’ll want to rethink how you named your DB tables and ORM models. To me, something like “collection” and “collection_item” makes more sense

I would also like to see support for private and public collections (isPublic boolean in the schema, defaulting to false).

It would also be interesting to have multi-editor (collaborative) collections.

Both of these ideas come from the MusicBrainz collections, which I encourage you to have a good look at.

BookshelfJS (the library our ORM package is using) has a section about Collections, their way of representing and interacting with a set of items (the names are getting confusing here!). It would be good preparation to read it and start playing with them in a sandbox to get used to it: https://bookshelfjs.org/api.html#section-Collection

Timeline

I think Week 1 should only be about setting up database and ORM. It will take some time to write the tests for the ORM.

Week 2 should start with finalizing UI mockups, before getting to implementing the display pages. (/editor/:id/collections // collection/:collectionId).

That way we’re assured of the final look and you’ll have a clear reference as a guideline.

You currently have weeks 3 & 4 just for the delete routes, that does not seem right.

I’d like to see a bit more detail in weeks 9-13

Potential stretch goals

A feature that would be good to at least start implementing would be a nice page for users to search for public collections.

This is currently missing for MusicBrainz for example.

This might require a bit of help from my side if search is involved, to guide you on ho to work with ElasticSearch indexes.

I’d also like to see as a stretch goal the addition of collection API endpoints (at least to get collection content, and list a user’s public collections, not necessarily edit/delete)

I suspect you’re not familiar with the API code yet, but you’ll see it is well structured and easy to follow (much easier that the website codebase), and should be very similar to the website endpoints you’ll be implementing and reuse the same utilities.

prabalsingh24 · March 19, 2020, 4:32pm

@mr_monkey sorry I was busy in the past week. I will make changes very soon

prabalsingh24 · March 22, 2020, 6:37pm

Umm that’s the closest think we have to an entity description right? When you look at Agatha Christie for example you can see the stuff she wrote. That was my idea behind adding it. Should that be removed ?

Edit: I’ve planned to create different collection-table for each entities. With appropriate columns.

Yeah that icon is wrong. I will update it

This depends on whether SOLR search project gets done in the summer too right? That’s why I didn’t mention this thing

This will be in post-gsoc right? Or would you like this to be adjusted in the 13 weeks?

What do you think of checkboxes and ‘remove selected entities’ like in collection in MusicBrainz?
Other thing that can be done is add a ‘Remove icon (dustbin)’ which will be shown when you hover over the row. (Removing multiple entities won’t be possible in this case)

I guess you’re talking about create collection form which comes after the ‘add entity’ from the search result. Since the entity will directly be added to that collection, I plan to assign the entity type automatically to that collection. or do you prefer to have entity type field to be present but set to the entity which is about to be added, and make it unchangeable?

Would you prefer to have a name other than User Collection ?

mr_monkey · March 23, 2020, 12:18pm

Umm that’s the closest think we have to an entity description right? When you look at Agatha Christie for example you can see the stuff she wrote. That was my idea behind adding it. Should that be removed ?
Edit: I’ve planned to create different collection-table for each entities. With appropriate columns.

I’m curious to see what your plan is for each entity, but my gut feeling is that relationships shouldn’t be shown in there.
I think it’s out of scope for that table; if I’m adding Agatha Cristie into an Authors collection, I don’t expect to see her works in that author collection.

A feature that would be good to at least start implementing would be a nice page for users to search for public collections.

This depends on whether SOLR search project gets done in the summer too right? That’s why I didn’t mention this thing

No, we should be able to implement it using ElasticSearch. I would help you in that process.
We’ll have to define what eactly we would be searching for (collection title? any collections a certain entity is in?)

[…] stretch goal the addition of collection API endpoints[…]
This will be in post-gsoc right? Or would you like this to be adjusted in the 13 weeks?

I’d like to see it formalized as a stretch goal, meaning an extra item if you finish everything else earlier than anticipated.
You won’t be penalized if you don’t get to it during the GSOC period, but it gives a clear direction if you run out of things to do, or a good next step for after GSOC to round off the project.

I suppose in that case clicking ‘edit’ will allow only to edit the title, description and public/private flag.
The ability to remove items in bulk would also be nice
What do you think of checkboxes and ‘remove selected entities’ like in collection in MusicBrainz?
Other thing that can be done is add a ‘Remove icon (dustbin)’ which will be shown when you hover over the row. (Removing multiple entities won’t be possible in this case)

Yes, bulk remove is a must. We could combine both and have both the auto-hide dustbin button each row and a checkbox for bulk actions.
If we have to choose one, bulk actions would be prefered.

I guess you’re talking about create collection form which comes after the ‘add entity’ from the search result. Since the entity will directly be added to that collection, I plan to assign the entity type automatically to that collection. or do you prefer to have entity type field to be present but set to the entity which is about to be added, and make it unchangeable?

Automatically assigning the type is fine. I think it would be good the display it as well, but not editable as you say.
you could also shown it in the “Create New Collection” button (e.g. “Create New Author Collection”), just because you can

BookshelfJS (the library our ORM package is using) has a section about Collections, their way of representing and interacting with a set of items (the names are getting confusing here!).
Would you prefer to have a name other than User Collection ?

No, I think the name is right. I just wanted to make a point of differentiating the ORM Collection and the User Collection you propose to implement.
They ORM Collections will be instrumental, so it’s important we have a good way of talking about both without confusing them

prabalsingh24 · March 24, 2020, 12:13pm

@mr_monkey I’ve made some changes in my proposal. Have a look whenever you’re free

prabalsingh24 · March 24, 2020, 1:42pm

I haven’t mentioned much about search feature right now. I will be reading little bit about ElasticSearch first.

mr_monkey · March 24, 2020, 6:25pm

This is getting to a nice shape now!
Thanks for those modifications.

I have more nitpicks and questions to help solidify the proposal further:

UI:

This is pretty good for now, we can see in more detail when we come to it.

Search page would benefit from having bulk actions, I think, i.e. select multiple entities to add to collection or to select for merge (if same type)

Schema:

That looks much better now.

You could also store the last modification date for a user_collection. That would allow you for example to sort collections by latest modification in the recent collections display page you made.

For dates, have a look at how they’re defined in existing schema + with a default bookbrainz-site/sql/schemas/bookbrainz.sql at master · metabrainz/bookbrainz-site · GitHub

For name and description: bookbrainz-site/sql/schemas/bookbrainz.sql at master · metabrainz/bookbrainz-site · GitHub

Another possible addition would be storing the timestamp when an item was added to a collection (in user_collection_item).
That would allow to sort a collection’s items by their addition date-time.

ORM changes:

Still mentions in the beginning UserCollectionsSet and UserCollections

UserCollection model:

userCollectionItem and userCollectionCollaborator should use this.hasMany instead of belongsToMany.

I would also suggest renaming them to “items” and “collaborators” respectively. It read like this: 'UserCollection hasMany items’ and 'UserCollection hasMany collaborators’.

Perhaps renaming “editor” to “owner” would also be appropriate to clarify that concept (no need to change it in the schema)?

Editor model:
userCollection → userCollections (“Editor hasMany userCollections”)

Routes:

GET ‘/editor/:id/collections’

collectionIds array have collection_id of collections of which the editor is a collaborator

That’s not a bad way to go bout it, but you should be able to achieve the same using a subquery in await new UserCollection()…. You’ll need to dig a bit on http://knexjs.org/

GET ‘collection/:collectionId’

Router.get(‘collection/:collectionId’) and Router.get('collection/:collectionID’)`

You say this at the beginning of the routes section. Not sure I follow.

For isAuthenticatedForCollection you don’t mention public collections.

Collections of different types will be having different columns in the table.
I plan to make only one table CollectionView

It would be interesting to investigate if you could reuse the existing WorkTable and EditionTable components

The entitesArray will be having bbid of all the entities. Now to fill up the other data, these two already defined function will be reused […] orm.func.entity.getEntity

Using withRelated: ["UserCollectionItem.entity”] above should be enough, and load the entity.(however the rest concerning deleted entities is still relevant)

GET ‘collection/:collectionId/delete’

Also a function isAuthenticatedForCollection will be reused

Keep in mind that there are different scenarios here that need to be taken into account by that middleware (owner can view, add, remove and delete, collaborators can view, add, remove but not delete and others can view if collection is public). The middleware you’re basing this on only checks if user is signed in. I’d like to see a plan for how to accommodate for the different cases.
On that subject, there’s no mention of a route for adding/removing collaborators from a user collection…

Ths form will be confirmation of the delete and hitting delete will make post call to 'collection/:collectionId/delete’

The sentence could use some rephrasing. Also I assume you mean 'collection/:collectionId/delete/handler’. The next route title (the next sentence) should be corrected as well.

POST ‘collection/create/handler’

.set(‘privacy’, privacy)

This should be .set(‘public', … according to the schema.

POST 'collection/add’ and 'collection/remove`

Shouldn’t these routes have a :collectionId? Why depart from the norm of the other routes and have the collectionId be a param?
It will also be easier to reuse the isAuthenticatedForCollection middleware if the structure is the same as the other routes.

Timeline:

I have a request to add an item to the Community Bonding Period: if you’re not very familiar with debugging tools yet, it would be a great advantage for you to learn how to add breakpoints and step in, out and over code, and how to use the react devtools browser extension.
They are invaluable tools and will greatly increase your precision and capacity to analyze issues while saving you time.

Shamefully, I haven’t yet found the time to make any efforts in integrating the React-based design-system into BookBrainz.
As its integration could have unexpected time repercussions, I won’t count it as a requirement for the project.

I think it makes more sense to start with the create/edit/handlers routes instead of the display pages (after schema/ORM and UI mockups), from a conceptual point of view and also so that you can better test what follows (being able to create test collections using the website instead of editing the DB directly…)

mr_monkey · March 31, 2020, 2:08pm

Nice !

I have very two changes requests below. Time to post the proposal on the gsoc website !

For the Editor ORM model, collaborators() is a bit ambiguous. Maybe collaboratesOn would be clearer?

You didn’t address this point from my previous reply:

Ths form will be confirmation of the delete and hitting delete will make post call to 'collection/:collectionId/delete’

The sentence could use some rephrasing. Also I assume you mean 'collection/:collectionId/delete/handler’. The next route title (the next sentence) should be corrected as well.