GSOC-2020: Bookbrainz Documentation

Tags: #<Tag:0x00007f8211123848>

GSOC-2020: Bookbrainz Documentation

Personal Information

Name: Chinmay Kothari
IRC Nick: Chinmay3199
MB username: chinmay3199
Email: chinm170107016@iitg.ac.in
Github: chinmay-kothari
Time Zone. UTC+05:30

Project Overview

The current documentation of Bookbrainz is very poor and scattered. Both user and developer documentation hasn’t been updated and needs to be revamped. A good documentation would be very useful for new users coming to the site. Automated deployment of the JSDoc codes from the source code needs to be done as the existing service doclets.io is defunct now. The code coverage presently is at 44% which could be improved seriously. The source code is not very well documented with some necessary functions not documented. The user guide lacks several features like style guideline, glossary etc which needs to be added.

New features and developments -

  • Replacing the automated documentation deployment service for JSDoc with a Github Action that would deploy the docs on Github Pages.
  • Improving the code coverage by writing unit tests.
  • Identifying the need and improving the documentation in the source code.
  • Revamping the user guide with adding more features like tutorials, style guides, to improve the accessibility for a new user.

Automated Documentation Deployment

I plan to use the JSDoc Action to first arrange all JSDoc comments across the whole source code in a single directory. Then with using GitHub Pages Action, the arranged comments will be deployed on a github page that could easily be hosted.

To establish our workflow make .github directory and workflows subdirectory in to our main codebase here.

An out directory would be made to store all the generated files containing JSDoc comments. This will always be an empty directory as the process will be run on a virtual machine, preferably ubuntu, minimising the need to store files.

on:
  push:
    branches:
      - master

Actions would be executed at every push into the master file. This will ensure that a fresh webpage is generated at every change with the master.

Minami template would be used for the webpage with README file as the home.

- name: Build
        uses: andstor/jsdoc-action@v1
        with:
          source_dir: ./src
          recurse: true
          output_dir: ./out
          config_file: conf.json
          template_name: minami
          front_page: README.md

./src directory here will be addresses of all the files having JSDoc comments in them.

Above workflow generates a common file having all JSDoc comments and stores in ./out directory.

To deploy all the JSDoc comments, Github Pages Action would be used to give a github page that would be hosted.

- name: Deploy
        uses: peaceiris/actions-gh-pages@v3
        with:
          deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
          publish_dir: ./out

The files stored in ./out directory will be published and hosted at a Github page.

Attached below is mockup of the template used and the design of the webpage:

Code Coverage and JSDoc Comments

Presently the code coverage of the Bookbrainz is 44% and that could be increased significantly over the summer.

For writing unit tests Mocha framework with Chai assertion library will be used.

Using the link, files that are poorly tested will be addressed first, mostly the source directory.

Some of the important files to be tested and documented:

/src/server/routes/editor.js
/src/server/routes/entity/entity.js
/src/server/helpers/entityRouteUtils.js
src/server/helpers/handlers.js
/src/server/helpers/utils.js
/src/server/helpers/middleware.js

For example, here’s a list of some few functions that I think should be tested at priority:

In entity.js:

  • handleCreateOrEditEntity
  • handleDelete
  • addNoteToRevision
  • processWorkSets, processEditionSets, processEntitySets, etc.

In entityRouteUtils.js:

  • makeEntityCreateOrEditHandler
  • entityEditorMarkup
  • generateEntityProps

In middleware.js:

  • makeLoader
  • makeEntityLoader

In test/test-site.js, only /about, /contribute, /develop are getting test. Improving the file to cover all the routes:

  • GET /revisions
  • GET /register
  • GET /search

Similarly several different funtions all over the codebase will be covered for testing and documentation. To get a look at my work on writing testcases my PRs are available here.

User Guide

Bookbrainz beginner’s user guide needs to be improved to make the website more accessible to the new users. A whole revamp of the user guide will be done with some major changes in the sections like Glossary, Web interface sections. Likewise to Musicbrainz a how-to section would be added to make the initial process easy for the user.

The current user guide will be completely changed using Sphinx and readthedocs. Presently the user guide uses .md file format which difficult to maintain, this would be replaced by .rst file format.

All the changes will be made in the user guide’s repo, avoiding the need to change to address that guide presently uses here.

I was thinking about changing the whole user guide with docz and deploying on Netlify. Though discussion with mentor is required to get the feasibility of the process and whether to do it or not.

Sitemap -

This is the basic template of the user guide.

General -

Explains the user what Bookbrainz is all about, what we aim to do here. Briefs about the technologies used by Bookbrainz and how user can become an editor or a contributor. Introduction about the Bookbrainz community will be given along with IRC channel and metabrainz community. Information about the database and how the webservice works would be added mostly like Musicbrainz’s About.

Glossary -

Glossary is the section in which detailed description of terminologies used over widely over Bookbrainz will be written. A glossary would help to make new user get familier with the enviornment with ease. Definations will be coupled with common examples to improve the understanding. Some of the common terms that will be covered here will be:

  • Entity and its different types,i.e, Work, Edition, Edition Group, Author, Publisher
  • Features like Relationship, Revisions, Collections, Achievements and more.
  • Terms used while filling up the form like Identifier, Disamiguation, Alias.
  • Some general knowledge of world of books like Poem, Anthrology, Novels, Plays, Article, Scientific papers etc.

Web Interface -

Explaining different parts of the website to the user is an important part of the documentation. This aims to give the user a trip around the whole website giving brief knowledge about features of Bookbrainz. With all the terminologies already explained in the Glossary section, links to them will be placed while explaining the interface. This would be useful Some of the features to be explained are -

  • Statistics
  • Help
  • Sign in / Register
  • Add Entity
  • Search
  • Achivements

With this being explained new users would know better how the website behaves.

How-To’s -

These will be light-weighted documentation with a major focus on tutorials rather than the documentation itself. Tutorials on different aspects will be there along with the images to guide a new user properly.

Some of the basic how-to’s that are planned are -

  • Creating a BookBrainz user account.
  • AddingEntity tutorial for every entity, i.e Work, Edition, Edition Group, Author, Publisher.
  • Adding relationship with the entities
  • Merging two entities if the same.
  • Editing already created entities and guidelines on writing editing notes or revision notes.

To get a reference on how the tutorials will be made can be looked here.

Frequently Asked Questions -

Taking the reference from Musicbrainz FAQs.
This could be divided into 2 sections, namely, General FAQs and developer FAQs. General FAQs will cover the questions regarding working of the site, while developer FAQ will deal with some common development questions. An example-

  • Q. What is an entity?
    These are the conceptual items representing literature along with its associates like author and publication.

This section would be needing help from others involved with the Bookbrainz to get the common questions.

Bookbrainz’s Style Guide-

Bookbrainz presently uses Musicbrainz’ style guide which here will serve as the reference.

Style guidelines will be done for -

  • All the entities name, specifying how one should write the entity name.
  • Relationship styling between two entities.
  • Sort Name, specifying different sort name approaches for every entity.
  • Writing aliases for entities.
  • Language specific guidelines for multiple languages, same as here.

Style guideliens could be further expanded to cover development guidelines as well.

Contribute -

This section will address users who are excited to be a contributor to Bookbrainz. With some details about the softwares used by bookbrainz, contributing guidelines will be mentioned.

  • Instructions on setting up a local bookbrainz server will be given with reference to the developer guide.
  • Instructions on contribution to the user guide will be given.
  • Contribution guidelines covering aspects to take care while contributing to Bookbrainz.

Timeline

Here is a more detailed week-by-week timeline of the 12 weeks GSoC coding period to keep me on track.

Pre-Bonding Period (April)

I will start with /src/server/helpers/utils.js file to start writing the test cases. This would help me get a better grip on using Mocha framework and will give the mentor a better judgement on my abilities.
I already had a merged PR on writing testcase to Bookbrainz-data-js and for that I studied Mocha framework and got familiar with it.

Community Bonding Period

I will learn more on End-to-End testing and get familiar with Cypress so that we can shift to writing End-to-End test cases as well.

I was already thinking about replacing the readthedocs template with something more appealing to the user and that got me to docz. I will discuss with the mentor the possibility making this change and working towards it.


I will keep switching between the different projects, this will ensure I don’t get exhausted on any single one, also giving time for review process.

Phase-1

Week-1: We’ll start with adding General and Glossary sections.

Week-2: Will start this test-case first focusing on src/server/routes

Week-3: Move back to writing Web Interface section for user guide.

Week-4: Going back to writing test case.

Phase-1 Evaluation here

Phase-2

Week-5: Enabling and deploying the automated documentation from the JSDoc comments will be done here.

Week-6: Will work on Tutorial section for user guide.

Week-7 & Week-8: Writing test case for src/server/helper directory.

Phase-2 Evaluation here

Phase-3

Week-9: Documenting Style guide for Bookbrainz

Week-10: Moving back to completing the test cases for src/server

Week-11: Completing user guide with FAQs and Contribution sections.

Week-12: Writing test cases for remaining important directories. I plan to increase the code coverage to 60%.

Stretch Goal

With everything going as per the plan I would try hard to achieve a Code Coverage of industry average 70%.


About Me

I am a 3rd-year student of IIT Guwahati. I got to know about Bookbrainz from my friend, Prabal also working with Bookbrainz. Also, I had a brief idea about Metabrainz with our senior Sambhav Kothari’s Blog. The kind of work he was doing really intrigued me. I started using Bookbrainz in around mid-February but it took me some time to get used to the website.

Tell us about the computer(s) you have available for working on your SoC project!
I have an asus laptop with i7 processor with 8gb of ram.

When did you first start programming?
I started programming in my first year about 3 years ago.

What type of books do you read?
In my school, I had a great time reading Agatha Christie with The Big Four one of my favorites. Over the year I have read Mein Kampf, Harry Potter, and recently John Green’s books.

Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
I’ve contributed to Bookbrainz only.

How much time do you have available, and how would you plan to use it?
I have nothing to do this summer. I will be able to give 3-4 hours fairly easily every day. Amounting it to at least 30-35 hours a week in the project.

Hi @mr_monkey,

I am confused on what to write in the Code coverage section about increasing the number of test cases. It would be great help you can guide me on that.

Also kindly review the draft. Thanks!

Hi @ chinmay3199!

Thanks for the proposal, this looks a lot more solid.

Generally, I would like to see more examples of testing and writing from you to be able to evaluate the scope of the project.
I have seen one test on bookbrainz-data-js, but I’d like to see more, it wasn’t much to judge by :slight_smile:

Presently the user guide uses .md file format which difficult to maintain, this would be replaced by .rst file format.

I find markdown pretty accessible, and it also allows users to submit PRs straight from the GitHub editor. Could you talk me through how rst files would make it easier to maintain?

I was thinking about changing the whole user guide with docz and deploying on Netlify.

I don’t know either of those; could you give me a short summary of their respective benefits over the existing solutions?

Poem, Anthropology, Novels,…

I think you mean Anthology here :wink:

Sitemap

That looks pretty good for now.

Web Interface

One addition: Revisions (this page, currently in beta only: https://beta.bookbrainz.org/revisions)

Q. What is an entity? These are the conceptual things representing parts of literature along with the associations like author and publications

I think this could be improved upon. Some terms are too vague, like ‘things’, ‘parts of literature’ and ‘publications’.

I do realize it’s difficult to describe these immaterial terms, but it will be necessary to be able to do so to ensure the quality of the documentation.

Week-5 & Week-6: Enabling and deploying the automated documentation from the JSDoc comments will be done here.

Why do you think this will take two weeks? From what i’ve seen on your test repo, this shouldn’t take that long, unless i’m missing something.

Generally regarding the timeline, I would anticipate a lot of back-and-forth for the duration of the project.

I think there would be some advantage in mixing writing for the user guide and writing tests.

That will 1. give me enough time to review the text parts while having something else to work on, and 2. allow you to stay mentally sane :slight_smile: It can be overwhelming writing tests for days on end (from experience).

Regarding what to write in the code coverage section, it’s a tricky one.
You could create a list of functions you think should be tested, with some idea of priority, and link it in the proposal.
That in itself would serve to demonstrate your understanding of the codebase.
To take the examples from your proposal, getAdditionalRelations isn’t the most pressing to test, nor the most complex code (added complexity in code always comes with the possibility for it to break; a simple function like this one is unlikely to break from someone’s code contribution)

Hi @mr_monkey,

I have made the changes to the code coverage section to specify some important functions. Changes the timeline to keep shifting between the things. Also did changes you recommended.

I don’t know either of those; could you give me a short summary of their respective benefits over the existing solutions?

The only benefit that I saw was improved design basically :slightly_smiling_face:. Also, it provides a playground text editor to help to see how code will look. Though I need to look more into this before suggesting anything. I was planning to work more with it in the next week to get clearer image.

Week-5 & Week-6: Enabling and deploying the automated documentation from the JSDoc comments will be done here.

The one problem that I saw with my test repo was, the Action doesn’t show JSDoc comments in their respective directories. I was thinking to work on that with Actions developer to check if we can get them in their seperate directories. Though I have changed its timeline to one week, and it now makes the timeline better for me :slightly_smiling_face:

I find markdown pretty accessible, and it also allows users to submit PRs straight from the GitHub editor. Could you talk me through how rst files would make it easier to maintain?

The reason I started with rst format is that sphinx supported it better. For rst, sphinx has roles and directives that make making writing code easy in different languages. But true that here in our case, we don’t actually need to write code that much. Rst format also allows us to submit a PR directly from Github editor. Also true that md format is much more common with the users. If we want to go ahead with the markdown format, I am very open to that idea. Please let me know.

I think you mean Anthology here

Thanks for correcting it :sweat_smile:

OK, thanks for the clear answers, those are valid points.
I think this is at a good level of detail now.

Don’t forget to send the proposal on the GSOC website !

Sure I will send it on the GSOC website.

Thanks for reviewing!

1 Like