GSoc 2019: BB User collection

Personal information

Name: Aniket Bansal
IRC Nick: baniket
MB username: projectescape
Email: escaperealityproject@gmail.com
Github: escaperealityproject
Time Zone. UTC+05:30

Proposal

Project overview

This project is aimed to design and implement a user collections feature, which would enable users to create, organize and retrieve collections. This will also help share various collections with other users.

Collections will have two access types, Private and Public, and will be having different entity types, like edition ,work, etc.
A collection will be able to have only one type of entities in them.

UI/UX and placement of react components

View Collections
The user will be able to see his/her collection and edit/remove them by going to the collections tab in the editor,

On which the user will be redirected to

/editor/:id/collections

A user can delete his/her created collection by pressing on the cross button.

A new react component will be required for this,

./src/client/components/pages/parts/editor-collection.js

On selecting a collection, user will be shown a page like this

where he/she will be easily be able to remove any entity they don’t want in the collection by pressing the cross button near the entity.

Edit/Add Collection
Pressing on the edit collection and add collection will redirect to

/collection/edit/:collectionid and
/collection/create

respectively, with a new react component placed in

src/client/components/form/collection.js

The form will look like

and will change post routes to add/edit collections just like the present entity-editor.

Adding entities to collection
For adding entities to collections, we will require a popover component placed in

./src/client/components/pages/parts/collection-popover.js

which will be passed on as a child component in the search page and the entity pages.
The search page with the child component will look like

And entity page will look like

The popover component will filter the collections corresponding to the entity type.
It will also include a way to create collections on the go by pressing the Create collection button, which will change the content in popover to show a creation form

A collection created in this way will add the entity where it was created from and get the entity type from that entity.

Browse Public collections
Users will also be able to browse various public collections made by other users with ui similar to the current search ui,

/collection/search

By pressing the + button (which can be changed to show things like Copy collection etc. instead of + after feedback), user will be shown a popup where he/she will either be able to create a copy of the collection, i.e he/she will create in a new collection with a new id and copying all the items inside the original collections, from which he/she can easily remove/add new items, without causing any change in the original collection, or create a reference to the original collection, i.e. if the original owner changes items in the collection, the same changes will be seen the referenced collection.

To explain this better, in the db, if the user chooses the first option, in bookbrainz.collection table, a new row with new collection_id and parent_collection_id which is equal to this new collection_id will be created.
In bookbrainz.collection_item table, all the entries corresponding to the old collection will be duplicated but with the new collection_id.

In the second option, in bookbrainz.collection table, a new row with new collection_id, but the parent_collection_id equal to the old collection will be created. As the items in bookbrainz.collection_item table are referenced using parent_collection_id, no new entries in this table will be required.

A new component will need to be created for this which would be passed as child component to the SearchPage component in place of the SearchResults component

src/client/components/pages/parts/component-search-results.js

Schema / bookbrainz-data-js changes

In the database side of things, there will be two tables required as shown below

dbeditfinal

CREATE TABLE bookbrainz.collection(
    collection_id SERIAL PRIMARY KEY,
    parent_collection_id INT,
    collection_name VARCHAR NOT NULL,
    collection_desc TEXT,
    entity_type bookbrainz.entity_type NOT NULL, 
    editor_id INT NOT NULL,
    created_at DATE NOT NULL,
    private_status BOOLEAN NOT NULL
);
ALTER TABLE bookbrainz.collections ADD FOREIGN KEY (editor_id) REFERENCES bookbrainz.editor (id);


CREATE TABLE bookbrainz.collection_item(
    parent_collection_id INT NOT NULL,
    bbid UUID NOT NULL
);
ALTER TABLE bookbrainz.collection_item ADD FOREIGN KEY (parent_collection_id) REFERENCES bookbrainz.collection (parent_collection_id) ON DELETE CASCADE;
ALTER TABLE bookbrainz.collection_item ADD FOREIGN KEY ADD FOREIGN KEY (bbid) REFERENCES bookbrainz.entity (bbid) ON DELETE CASCADE;

bookbrainz.collection table will contain all the collections created and will contain the userid to identify the user who created the collection. It will also create information like name, entity type, description, Private/Public, date of creation etc.

For getting the entities present in a collection, another table bookbrainz.collection_item will be made which will contain parent_collection_id and the bbid of the item we want to add in the collection. For example in the collection 1 we want to add books with bbid 1,2 and 3, there would be 3 rows entered with parent_collection_id 1 and bbid 1,2 and 3. We can then refer the items in a collection where parent_collection_id are equal in both the tables.

In bookbrainz-data-js, will we require two new models,

const CollectionEntity = bookshelf.Model.extend({
    tableName: 'bookbrainz.collection_item',
    collection:function(){
        return this.belongsTo('Collection','parent_collection_id')
    }
})

and

const Collection = bookshelf.Model.extend({
    tableName:'bookbrainz.collections',
    entities : function(){
        return this.hasMany('CollectionEntity','parent_collection_id')
    }
})

Which can be exported with same name.

A change in Editor model will also be required, an addition of

collection(){
    this.hasMany("Collection","editor_id");
}

To reflect the changes in schema.

Changes to Routes / Fetching/modifying Data

Handling all the collections will require a new /collection route in
src/server/routes.js
and a new file in
src/server/routes/entity/collection

It will contain a collection edit handler

fetchInfo={
   // collection_id if the collection
};
newData={
    // Data from the edit form
};
Collection.forge(fetchInfo).fetch().then((collection)=>{collection.save(newData)});

Collection create Handler

newData={
    // Data from the collection creation form
};
Collection.forge(newData).save();

Adding entity to a collection Handler

newData={
    // collection_id and bbid of the entity to be added
};
CollectionEntity.forge(newData).save():

Deleting item from collection Handler

fetchInfo={
    // collection_id of the collection, bbid of the item to be deleted
};
CollectionEntity.forge(fetchInfo).fetch().then((item)=>{item.destroy()});

Deleting Collection Handler

fetchInfo={
    // collection_id of the collection, bbid of the item to be deleted
};
Collection.forge(fetchInfo).fetch().then((collection)=>{collection.destroy()});

Fetching of data will be done within the different get routes and will be passed as props, so no new post routes will be required.

Proposed Timeline

A broad timeline of the work to be done is as follows:

Community Bonding (May 06 - May 27):

In the community bonding period the main focus will be to learn the coding style used in the project,identifying all the technologies required for the project,set up a working environment for the project, and discuss what more needs to be changed or added to the project. I will be having my university examinations during this period, but I will still try to be active on the IRC and do the above mentioned tasks.

Here is the week-by-week timeline of the 13 weeks GSoC coding period (27 May - 19 August):

  • Week 1 (May 27 - June 2):

    • Begin with setting up the environment.
    • Understand how the bookshelf.js and bookbrainz-data orm works and interacts with the application (How the props are fetched from database and passed to the components, how more entities are created and edited)
  • Week 2 (June 3 - June 9):

    • Start working on writing SQL queries for adding the new tables needed.
    • Create new models in bookbrainz-data-js for the new tables added.
  • Week 3 (June 10 - June 16):

    • Create /collection routes for various different handlers like edit,add,delete collection or collection item using test driven development using mocha/chai.
  • Week 4 (June 17 - June 23):

    • Complete if any work left.
    • Document the changes made up until now.
    • Fix bugs if found any.

First evaluations here

  • Week 5 (June 24 - June 30):

    • Build editor-collection, add/edit collection form.
  • Week 6 (July 01 - July 07):

    • Test the components manually and alter/improve them if any changes are needed.
  • Week 7 (July 08 - July 14):

    • Document the changes made up until now.
    • Cleanup and optimize code.
  • Week 8 (July 15 - July 21):

    • Complete if any work left.
    • Fix bugs if found any.

Second evaluations here

  • Week 9 (July 22 - July 28):

    • Build the popover component for adding items to a collection.
  • Week 10 (July 29 - August 04):

    • Identify and pass the popover component as child to all the components where needed.
    • Perform manual tests of adding items using the new component.
  • Week 11 (August 05 - August 11):

    • Build the Browse Public Collection, making sure it works as expected using manual testing.
  • Week 12 (August 12 - August 18):

    • Catch up on work if any left.
    • Cleanup up and optimize code.
  • Week 13 (August 19 - August 26):

    • Document the changes made during this phase.
    • Work for the final submission.
  • After Summer of Code

Continue working on BookBrainz. Fixing any bugs if found later.
Also will try to work on picard, as I personally use it and would like to see it improve even more.

Detailed information about yourself

I’m a second year student studying Computer Engineering at Thapar Institute of Engineering and Technology, Patiala. I first came across MetaBrainz by picard, which I have been using for a long time. I was intersted in BookBrainz project as I wanted to contribute to MetaBrainz and BB used the technologies that I knew and was interested in. I have already learned things like server side react rendering by looking at the codebase and hope to learn more things too.

  • Tell us about the computer(s) you have available for working on your SoC project!
    I have a lenovo ideapad with i5 7th gen, 8gigs RAM. Currently I’m running Windows/Ubuntu/Mint/Manjaro on quad boot. Will dedicate one of the slot to the project if selected.
  • When did you first start programming?
    I first started programming in my first year at college, when I got introduced with C and C++. Later I learned various web technologies like the MERN stack etc. from various sources.
  • What type of music do you listen to? (Please list a series of MBIDs as examples.)
    I listen a lot of electronic music ranging from melodic future bass to melodic dubstep. Some of the artists I have on repeat are Seven Lions,Illenium and many many more.
  • What aspects of the project you’re applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?
    The idea of having a place where you can look up for all the books in existence, browse through their various editions all over the years so conveniently interests me the most.
  • Have you ever used MusicBrainz to tag your files?
    Yes, my first introduction to MetaBrainz was through picard, which I use for tagging my music since more than 3 years ago.
  • Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
    No.
    • If you have not contributed to open source projects, do you have other code we can look at?
      I have done little hobby projects, hackathon projects, tutorial walk along projects. Code for all is available on my github.
  • How much time do you have available, and how would you plan to use it?
    I will have 30-35 hours available during a week.
    I plan on utilizing 4-5 hours per day on the project.
  • Do you plan to have a job or study during the summer in conjunction with Summer of Code?
    I plan to study DS and algo during the summer, learn python and get some kind of practical introduction to ML.
1 Like

Hi Aniket,
Thanks for your proposal!
I’ve been looking forward to this functionality I must say.

Overall, my main concern at the moment is that you haven’t yet been very deep in the BookBrainz codebase (as a matter of fact, I don’t recall seeing any PR from you but please correct me if I’m wrong!).
As noted later below the codebase is fairly complex when it comes to creating, editing and storing entities, and it seems like it would be a deep jump.

I would suggest having a closer look at some of the meat of the code (for example try to follow from beginning to end how to create one of the entities, including its model in the ORM bookbrainz-data-js) and starting to contribute some PRs to the project to familiarise yourself and get a better understanding of it.

Now for the details:

  • These pages will require 2 additional react components, /collectionedit /collectionadd

    • Have a look at the entity-editor` in BB: the same component is used for creating and editing an entity. The same should apply here.
    • It should also take optional query parameters, for example if you want to create a new collection from the popup with a name and an entity to put in it. The mechanism here needs clarification.
    • The visual aspect of adding a new collection from the popup also needs another mockup: where do I enter description, type, etc. ? Does that appear in the popup? Do I get redirected to a page?
  • Browse collections: what would be your suggestion for where in the project to put the collection search page?

  • Database:

    • I don’t think this is right. I would expect the collection entity to contain information such as name, description, etc. You haven’t described how entities would be in a collection, nor how a collection would be associated with a user. You will need a better understanding of how the database is organized.
    • Have a closer look at how relationships for example (or identifiers) are done in the database: there is an item (relationship table), a group (relationship_set) and a table to say which relationships are in which groups (relationship_set__relationship). The relationship_set_id then appears in the entity data tables. In this case, a collection_set_id would appear in the editor table
  • Timeline

    • You are most likely going to need more than a week to familiarise yourself with the various parts of the codebase. It is a rather complex system to understand; you might be surprised by the complexity.
    • I don’t understand week 5

      Begin identifying and what all components are need to be edited for adding functionality to add entities to collections.

    • week 6 this is unclear to me

      add collections corresponding to the bbid of the collection

    • week 9 What do you have in mind for “the public collection showcase” Could you elaborate?
  • Design:

    • I don’t think a tickbox is not the best choice in the popups.
    • What do the “+” buttons on the "Browse collections” search results rows do?
    • There should really be an icon for collections, that “+” sign alone isn’t clear enough I think
    • Two different good examples of popups: Youtube and Goodreads
    • I would expect to see a “Collections” tab in the user’s profile. And I don’t know if tabs is the best way to present collections in the collection display page. What if I have 45 collections?
3 Likes

@mr_monkey Thanks for the Feedback!! I have made the suggested changes and would love to hear some more feedback!

Thanks for the modifications @projectescape

Here are a few more comments and questions:

  • user will be able to create a replica of that collection

    How will that work database-wise?

  • column and tables naming in the SQL parts could be improved. We use underscore_notation. Have a look at the current schema

  • The naming of “collection entity” is not very clear. especially if the other table represents Collection entities… do you see the issue?

  • For getting the entities present in a collection, another table will bookbrainz.collectionentity will be made which will contain collectionid and corresponding bbid.

    Can you rephrase this? What corresponding bbid? Corresponding to what?

  • Adding entity to a collection Handler CollectionEntity.forge({data}).save():

    I don’t think this is going to work as you described, or maybe I don’t understanding it. What is {data} in this case? See Bookshelf.js | API Reference . I invite you again to look at how bookbrainz currently handles sets of data (relationships, identifiers or aliases, for example.

  • also they will be easily able to fetch information using Model.forge({data}).fetch();

    Could you be more specific? Which model are you talking about, and what is {data}? What information are you expecting to get returned? Can you write an expected output?

  • Regarding the timeline, I’d rather you concentrate your efforts on the database and ORM side before touching any front-end component.

  • You only briefly mention tests, and leave documentation for week 11. I would want you to use test-driven development and document as you go. Both are always very important and unfortunately often left until the end.

3 Likes

@mr_monkey Thanks again for your recommendations :slight_smile: . I have made some more changes to address the issues you talked about.