GSoC 2020: Unified form

anirudhjain75 · March 17, 2020, 6:25am

Google Summer of Code 2020 : Unified Form

Personal information

Nickname: Cyna

IRC nick: Cyna

GitHub: anirudhjain75

Twitter: Cybercynide

Proposal

Problem:

Overwhelming terms like Work, Edition, Edition Group might not be the best choice for new editors. Editor have to create works individually, and link the books though relationships section. The Author as well as the publisher have to be created before linking them with their respective books. The ease to add new books relevent details help improve editor exprience by reducing the time consumed in creating an edit.

Plan:

I plan on merging possible entites into a single Unified form to go alongside the current forms. Creating a tab based approach for data related to book. Tabs could be content, basic info, and additional properties. The content tab will focus on content of the book like the works included in the book, description of the book, genre of the book, etc. The basic information will focus on the title of the book, orignality as in if its translated or original edition. the author and publisher of the title. and if its translated then the translators details. The additional details tab would be focused on the external links, identifiers and other resources identifiers of the book. The UI components and other resources will be referred from https://github.com/metabrainz/design-system# to keep it Uniform with other MetaBrainz Projects. It will also use MusicBrainz pattern of making the property name bold if its required.

Project Details

Basic Info Tab

Basic information tab focuses on basic attributes of a book like its title, author, release date, language, etc.

Below are the property list and the most probable type that will be used for it

Property	Type
Title	InputField
Author	EntitySearchField with add new option
Format	SelectField
Publisher	EntitySearchField with add new option
Release Date	PartialDate
Is Translated	Checkbox
Translator	EntitySearchField with add new option of type Author
Language	SelectField
Original Book	EntitySearchField
Original Language	Select Field

Original Language, Original Book and Translator fields would only come up incase the isTranslated field is selected as that would mean that the book is translated. Incase the user adds the original book a relationship will be created as a translated edition else the user can simply choose the original language of the book and the same details would create another edition with different language

Content Tab

The Content tab is focused on the content of the Book as in the works it contains like poems, pieces of literature, articles, short story, etc. This is basically the area where the data of work is embedded or related to the current book. It will be the portion of book that related it to Work Entity in the current setup.

Below are the property list and the most probable type that will be used for it

Property	Type
Description	InputField
Genre	SelectField
Content	Dynamically created table of ContentType(Work)

The Content Type could be considered as

Property	Type
Title	InputField
Content Type	SelectField
Addition Properties	Modal

Additional properties modal containers identifiers, external links, and also option to choose an author incase its not the same as book. An idea for additional properties modal can be seen below

Relationships Tab

Relations Tab improved upon the current relationships section which previews the whole relationship in form of connected blocks. The relationships are created from the relationship-editor modal. There will be a button to add relationships. The added entities would have blocks created for each of them and linked using lines and with relationship title below the related component. This currently only supports linking existing entites to avoid complexities. The line for the previews would be made relying on LineTo library of react.

Physical Properties and external links

This tab consist of properties such as dimensions, weight and any other physical property of a book and also includes support to add external links such as Wikipedia link. and other global identifiers.

Below are the property list and the most probable type that will be used for it

Property	Type
Dimenstions	Width and height InputFields
Weight (incase physical book)	InputField
Identifiers	Dynammic TableOfIdentifiers
External Links	Dynammic TableOfExternalLinks

Table of Identifiers

Property	Type
Type	Select Field
Identifier	FormatedInputField

Table of External Links

Property	Type
Type	Select Field
Link	FormatedInputField

The Identifiers in here are linked to the book as an Edition.

These tabs are the basic tabs that exist at all times, apart from these there are two mode tabs that will be invoked incase author / publishers EntitySeachField's create new option is invoked. These two tabs are Author and Publisher. They will be used to get Author / Publisher related information from editors incase they want to create new author or publisher. It improves the current way of having to create everything in a new tab, by using internal tab based approach.

Author Tab

This tab is focused on getting obtaining information about the new author to be created. The name property of this tab would be linked with the EntitySearchField so changes would reflect on both areas. This tab takes in information like birth and death information incase the author is dead, Place / Area where the author is from and some other details.

Below is a table that lists the properties of the author tab and its probable type.

Property	Type
Name	InputField
Alias	AliasModal
Gender	SelectField
Birth Date	PartialDate
IsDead	Checkbox
DeathDate	PartialDate
Language	SelectField

Publisher Tab

This tab is focused on information related to Publishers. It becomes visible when editor chooses create new option in EntitySearchField.

Below is a table that lists the properties of the Publishers tab and its probable type.

Property	Type
Name	InputField
Alias	AliasModal
Languages	SelectField
Date Founded	PartialDate
Area	EntitySearchField of type place

These are all the tabs that are supposed to be a part of the Unified Form.

Editor Preview

Editor Preview is one of the major feature of this form which allows editor to ensure the accuracy of edits. The editor can check to see if the changes are exactly what was required before submitting the edit. The EditNote section will be shifted to Editor Preview instead to ensure editor know what they are making changes to or creating.

It will give a preview on which entity will be created and the data that will be added or changed. This includes the new artist and / or publisher that might be created with the edit.

This will be using EditDiff component from the Musicbrainz project or something similar.

Above is a layout that is would be similar to the implementation of relationships preview... This would be a big component that uses EditDiff to display differences and show them linked using the graph / diagram view. The above preview layout is a concept art and will be worked to improve user experience.

Developement Part

As far from what I've seen is that Bookbrainz uses redux in its forms for dispatching actions and follows FlowTypes. The current forms lie at https://github.com/bookbrainz/bookbrainz-site/tree/master/src/client/entity-editor. All the forms are built from components that lie here.

The unified form will be insame place as the other forms and will be built from components from the Metabrainz design system to be native with other MetaBrainz projects. These components will be selective stored at /src/client/components based on whichever is used.
The unified form will also follow the setup of making the property name bold incase the property is required.
Test cases for each component will be added soon after creating the components in MochaJs (I see bookbrainz uses it for testing)

Addons for the Project ( Incase time permits )

Guided Demo

Guided Demo are temporary overlays created to guide new editor thoughtout the process of creating a book to help them understand how to proceed with the form. Here is an example. Click on Click me I'm a Demo button for futher idea into what a guided demo is.

Timeline

Community bonding period

Get more familiarized with bookbrainz project and test suits used by bookbrainz getting more clearer on the objectives of the project.

Phase 1 ( June 1st - 29th ) : Design and Implement basic version of the Unified Form

Week 1 and 2 ( June 1st - 14th ) : Create a basic wireframe of UI / UX by continuosly improving it to be most user friendly. Also sort out the design elements that will be used in the Project

Week 3 and 4 ( June 15th - 28th ): : Create a basic working form in bookbrainz project using the design elements from Metabrainz design system and bookbrainz components with all the tabs functioning ( without data but only UI elements ).

June 29th - July 3rd: Working on Redux with the previous week goal and making sure to complete it.

Phase 2 ( July 4th - 27 ) : Relationship Editor Revamp

Week 5 ( July 4th - 10th): Design and Create a Relationship Editor Preview template for implmenting further

Week 6 (July 10th - 17th): Inprove the relationship editor to work in consistently updating on each state change using Redux and Integrate it into the Unified Form successfully.

Week 7 and 8 (July 17th - July 27th): Work with mentor to wire up the submit of the form to make changes to the database i.e. create all the required revisions for the entity.

July 27th - July 31st: Make sure Phase 2 of the project is completed i.e. The form should work with data. It should be submittable and all the test cases have been written. All the revisions should be created as expected.

Phase 3 ( August 1st - 24th ) : Add Edit Preview for Unified Form

Week 9 ( August 1st - 8th ) : Design a preview layout and implement the EditDiff for bookbrainz fields.

Week 10 ( August 9th - 16th ) : Create the preview page and add test cases for it

Week 11 ( August 17th - 24th ) : Merge the preview page and Unified form. Ideally the form should work as it is proposed.

August 24th - 31st : Ensure that every task has been completed and the form is working. This section of time is for adjusting the backlog incase any occur.

Detailed information about yourself

Tell us about the computer(s) you have available for working on your SoC project!

I have a MacBook Pro 2015 for working on during the summer of code as well as a custom build pc with 4th Gen i5 with 16gb ram and GTX 970.

When did you first start programming?

I started programming in high school as a competitive programmer

What aspects of the project you're applying for (e.g., MusicBrainz, AcousticBrainz, etc.) interest you the most?

Bookbrainz was created initially in React and has a lot of potential to become a good website. It interests me as there is a high scope of improvement in the project and a good playground for me to learn and improve myself just by improving the experience of BookBrainz users.

Have you ever used MusicBrainz to tag your files?

I haven't as of now as I don't keep music locally and prefer steaming it.

Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?

For the past year I've contributed to musicbrainz project and all of the code can be found at https://github.com/pulls?q=is%3Apr+author%3Aanirudhjain75+musicbrainz-server

What sorts of programming projects have you done on your own time?

I've created various type of projects from mobile app to websites and also games.

How much time do you have available, and how would you plan to use it?

I have around 5-7 hours daily everyday which I can put on working during Summer of Code. That should be around 35 - 49 hours Weekly.

Do you plan to have a job or study during the summer in conjunction with Summer of Code?

No I dont have any such plans as of now, other than interviewing at companies for internships and placement offers.

anirudhjain75 · March 29, 2020, 8:19am

@mr_monkey It would be great if you could have a look at this proposal.

mr_monkey · March 30, 2020, 12:24pm

Cyna — Unified form

Hi Cyna, thanks for the proposal !

First some general comments:

The proposal is quite short, and direly lacks concrete details. The technical aspects in particular are much too vague.
Your lack of experience with editing on BookBrainz shows, and I think you would benefit from entering books of various kinds (novel, translation, short stories, etc.) to really understand how everything fits together and what the issues are.
I am worried that your level of knowledge of the codebase is going to make it very difficult to achieve your goals in the time we have. To be clear I don’t doubt your ability but the time you have compared to the complexity of the codebase. I haven’t seen anything that suggests you’re familiar with it, and I know from experience it takes a while to get used to the entity editing/creation code, amongst other things.
On the user experience side, I think there’s room for improvement. It’s currently quite dry, not much improved compared to the existing forms. This is an opportunity to improve and clarify the nomenclature and make it more easily accessible and more user friendly rather than look like a database

Regarding the UI mockup, here’s a list of questions I couldn’t answer:

Basic information
- What does the Edition field correspond to in the schema?
- How do I enter the publication date?
- How do I enter work details?
- How do I enter multiple works? I see a tab for multiple works (I think), but how do I get there?
Physical details & Work
- First of all, the tab title is not clear
- There’s some physical details missing compared to the existing Edition form
- What are aliases? Why in the physical details tab? and What entity do they refer to?
- “Alias modal” is not clear at all for a beginner user
- Same for identifiers: what entity will they be applied to?
Work tab
- “works included” is not clear at all
- How do I say a work is a translation of another work?
Author and Publisher tabs are pretty much the same as the existing ones; not much to say about that.
I’d like to see the preview tab, how you’ll display relationships between entities in particular I’m curious about

realtime rendering on type from the backend

I don’t understand what that means. Can you please rephrase?

The relationship logic will be redefined to automatically link works created with the book as part of the book

More details please. What parts of the code are we talking about? What about the other types of relationships?

the edition name will also be considered for adding versioning of book to the book data

I don’t understand this part

he can choose

Gender neutral, please “they”, for example.

Similar workflow is allow proposed for the work entity.

“also”?

Week 4: Assemble all the components and make sure the UI / UX is working fine without any issues

This hides a lot of complexity, I think a week is much too short. There’s more moving parts like Redux to handle state on the front-end, form validation, etc.

Week 5 : Working on fetching data from the database to pollute the UI elements like Select tag, Search elements like Area and other components

“populate", I assume

There is an existing entity search component that is used on the website. Would that be a separate component? If so, why not reuse the existing one?

Week 6 : Testing all the elements and adding test cases for all of them

For your sanity, you’ll want to add tests as you go, rather than doing it all in one block.

Week 7 and 8 : Finding a way to post the changes to the database on submit and working on the relationship logic for the created entities.

This is the other crux of the project, especially without much prior knowledge of the codebase.

Two weeks is definitely way too short for that, and I’m worried you won’t have time to get familiar enough with the codebase on time for the project.

I would suggest putting the guidance system as a stretch goal and revising the timeline to accommodate the comments above.

Good luck with the continuation !

anirudhjain75 · March 31, 2020, 6:27am

hey @mr_monkey. I’ve remade the whole proposal. Would love feedback of it

anirudhjain75 · March 31, 2020, 9:01am

Incase of my experience with bookbrainz… I’m confident that by the time its community bonding period I will be able to make changes to bookbrainz easily. I’ll be working on the bookbrainz codebase atleast for another month to gain experience on it

mr_monkey · March 31, 2020, 9:52am

The proposal is taking shape nicely !

Generally, from our exchanges so far, it seems to me you’re not as comfortable with the user experience and design side of things compared to your technical knowledge, which I do not doubt.
At this point I’m a little worried that you’ll be relying on me too much to move the UX and UI forwards.

Below are the property list and the most probable type

Honestly, I’m not worried about the type. I am more interested in your understanding of what a field corresponds to in the underlying schema.
This goes for all the tables where you have the type (you can leave it there, it won’t hurt, but I find the types fairly obvious looking at the mockups)

Content Tab

We currently don’t have description or genre, but if we did, they would most likely be part of the Work entity. The question that arises then is which work do the fields in your mockup correspond to?
Regarding the additional properties button, I think you can do better and have all the necessary information for each Work presented in a user-friendly manner, without being too overwhelming.
There’s a middle ground in there somewhere. Maybe expandable sections is the would be a way to solve this?

Additional properties focuses on less interesting stuff

I think you can phrase that better

Editor Preview

This can be improved. I think it would be helpful and didactic to show the user which entities are being created (like Author and Publisher in this case, whereas the Work(s), Edition(s) and Edition Group are not clearly shown).
There’s also no mention of relationships between entities, which would also be nice to visualize.

Week 3 and 4…

(Week 3 starts on the 15th and week 4 ends on 28th)

There’s going to be a hidden block of complexity here when it comes to Redux. Currently the redux code is mostly shared across entities, with an $entityType$Section (authorSection, etc.) that is different for each entity type.
There’s going to be some complex refactoring needed to keep using the existing code while allowing for new cases (like multiple entities of the same type).
I think that 29th to 3rd period should be added to the previous two weeks with that goal in mind.

Week 5

Considering the code for the entity search fields already exists and is working, I’m not sure what you’ll need to be doing here.
“The select field should also get all the data required by it.” Not sure I understand this fully. Could you clarify what data do you need, and for what use?

Make note of all the revisions that have to be created

What do you mean by making note?

Keep up the good work, the proposal is getting better each time

anirudhjain75 · March 31, 2020, 10:09am

I’m comparing a book to an edition which has a set of work entity. I’ve not mentioned this part anywhere in my mockup to reduce the terminology.

I’ll see if I can divide it further to prevent using addition properties section

Again I’ve not mentioned Edition / Edition Group in the mockup as its sort of hidden. Each book created will be an edition with work connected to it.

As for this part, I’ve only though of relating work to edition by adding work to a book.

Sorry I assumed that the search fields dont have the data in them initially.

I am trying to improve my UX and design side but the current layout is mainly to keep the look of the site professional-ish, Again I’m still improving the UI/UX side and hope to get better at it

Thanks for the review

anirudhjain75 · March 31, 2020, 10:11am

As for this, I mean the SQL statements that has to be created to make the revisions work. Are you looking to get this done though existing components instead ? I’ll look more into this and let you know. Thanks

mr_monkey · March 31, 2020, 2:14pm

We use an ORM most of the time to interface with the database.
The code can be a bit complicated, but you’ll find good examples of how it’s used in this file (particularly handleCreateOrEditEntity): A fair warning: It’s dense code and spagetti-like.

github.com

bookbrainz/bookbrainz-site/blob/master/src/server/routes/entity/entity.js

/*
 * Copyright (C) 2016  Ben Ockmore
 *               2016  Sean Burke
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License along
 * with this program; if not, write to the Free Software Foundation, Inc.,
 * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 */

// @flow

This file has been truncated. show original

mr_monkey · March 31, 2020, 3:00pm

I’m comparing a book to an edition which has a set of work entity. I’ve not mentioned this part anywhere in my mockup to reduce the terminology.

I want to see more time permitting. There’s too little details in the content section for me to have a good idea of how it will work.

Again I’ve not mentioned Edition / Edition Group in the mockup as its sort of hidden. Each book created will be an edition with work connected to it.
As for this part, I’ve only though of relating work to edition by adding work to a book.

I think this is at the center of the problem; lots of what this unified form does is connect the entities in the right way for the user. With lots of possible combinations of all kinds, it makes it difficult to make a form that is both easy to understand (with some hidden complexity for the most common use cases) and gives users a glimpse into what’s going on in the background. Tricky.

In any case, don’t forget to submit the proposal in time

mr_monkey · March 31, 2020, 3:17pm

Sorry I missed the relationship previews !
I think showing them as a graph like that has some potential, it’s a nice solution ans clear for beginner users.

On other thing I missed:
In the basic info tab, instead of original language I think what you would really need is a way to search for or create the Work entity for the original work.

anirudhjain75 · March 31, 2020, 3:19pm

Yea two books would be created if the Book is missing and is translated