Here is my Draft Proposal for GSOC 21 @BookBrainz on the topic : Add subscriptions and email notifications
The usecase of BookBrainz has been increasing and the graph of number of users is going up day by day, For a user to manage and catchup the edits they make, the entity they create or to follow another artist, user, group, publisher , becomes so much tough and one would feel the need of an assistant to manage those things , though keeping an assistant is not affordable at all 
, nevertheless this project aims to provide the ease by contantly reminding the users through email about things they want to follow and manage.
Main Features included :
- A user wil be notified by default when their is new note on revisons they make or they subscribe
- A user can also choose to subsribe other user, work, edition, edition-groups, publisher, collection( public ) explicitly
- Extended goal includes establishing user preference over this default controls.
Frontend changes
Easy access to subscriptions listing
- This would facilate easy and fast access to users subscriptions
- proper icon will be placed instead of that same as of sigout icon
subscriptions listing
- can use the idea of collection table to list the subscriptions
- a checklist and a button to unscribe the multiple subscriptions would be there
- keeping and removing of table fields are subjected to discussions ahead.
- pager component will be reused
subscriber listing
- will list all the editors( a BB user ) who have subscribed to the edits that we are making
- listing may also include some specific works of ours that has been been suscribed.
- For the intial goal only users who are suscribers of us will be displayed in a table
- pager component will be reused
Subscribing/UnSuscribing a user
-
A BB user (editor ) can subscribe/Unsuscribe to the other editor by a simple buton interface
-
Likewise a simple button would placed for each of other entity ( TO DO )
- Work
- Edition
- Edition Group
- Author
- Publisher
- Collection
DATABASE Changes
There will be total 8 new table , because we will have 7 relation with editor(user) → entity, where entities are ( revisions,author, work, editon, editon_group, publisher, collection ) + 1 for user–>user subscription
They will be like :-
user_subscribed_<entity_type>
1 ) user_subscribed_revision
ORM:
CREATE TABLE bookbrainz.user_subscribed_revision(
user_id serial NOT NULL,
revision_id uuid NOT NULL,
CONSTRAINT user_subscribed_author_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_author_fk_1 FOREIGN KEY (author_id) REFERENCES bookbrainz.revision(id)
);
2 ) user_subscribed_author
ORM :
CREATE TABLE bookbrainz.user_subscribed_author (
user_id serial NOT NULL,
author_id uuid NOT NULL,
CONSTRAINT user_subscribed_author_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_author_fk_1 FOREIGN KEY (author_id) REFERENCES bookbrainz.author_header(bbid)
);
3 ) user_subscribed_work
ORM:
CREATE TABLE bookbrainz.user_subscribed_work (
user_id serial NOT NULL,
work_id uuid NOT NULL,
CONSTRAINT user_subscribed_work_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_work_fk_1 FOREIGN KEY (work_id) REFERENCES bookbrainz.work_header(bbid)
);
4) user_subscribed_edition
ORM:
CREATE TABLE bookbrainz.user_subscribed_edition (
user_id serial NOT NULL,
edition_id uuid NOT NULL,
CONSTRAINT user_subscribed_edition_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_edition_fk_1 FOREIGN KEY (edition_id) REFERENCES bookbrainz.edition_header(bbid)
);
5 ) user_subscribed_editon_group
ORM:
`CREATE TABLE bookbrainz.user_subscribed_editiongroup (
user_id serial NOT NULL,
edition_group_id uuid NULL,
CONSTRAINT user_subscribed_editiongroup_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_editiongroup_fk_1 FOREIGN KEY (edition_group_id) REFERENCES bookbrainz.edition_group_header(bbid)
);
6 ) user_subscribed_publisher
ORM:
CREATE TABLE bookbrainz.user_subscribed_publisher (
user_id serial NOT NULL,
publisher_id uuid NOT NULL,
CONSTRAINT user_subscribed_publisher_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_publisher_fk_1 FOREIGN KEY (publisher_id) REFERENCES bookbrainz.publisher_header(bbid)
);
7 ) user_subscribed_collection
CREATE TABLE bookbrainz.user_subscribed_collection (
user_id serial NOT NULL,
collection_id uuid NOT NULL,
CONSTRAINT user_subscribed_collection_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_collection_fk_1 FOREIGN KEY (collection_id) REFERENCES bookbrainz.user_collection(id)
);
8 ) user_subscribed_user
ORM:
CREATE TABLE bookbrainz.user_subscribed_user (
user_id serial NOT NULL,
subscriber_id serial NOT NULL,
CONSTRAINT user_subscribed_user_fk FOREIGN KEY (user_id) REFERENCES bookbrainz.editor(id),
CONSTRAINT user_subscribed_user_fk_1 FOREIGN KEY (subscriber_id) REFERENCES bookbrainz.editor(id)
);
Setting Up a MAILER
Establishment of A BB dedicated mailer to deliver all our mails to the subscriber is already under progess , almost a step away 
Here i have raised a PR for the same
It uses :-
- Nodemailer ( our mailer agent )
- SMTP server
APIs :
subscribe
- app.get(‘/subscribe/revision/:revisionId’, controller)
- app.get(‘/subscribe/user/:userId’,controller)
- app.get(‘/subscribe/entity_type/:id’,controller) ( will handle user—>entity subscription, entity can be any of 6)
Unsubscribe
- app.get(‘/unsubscribe/revision/:revisionId’, controller)
- app.get(‘/unsubscribe/user/:userId’,controller)
- app.get(‘/unsubscribe/entity_type/:id’,controller) ( will handle user—>entity subscription, entity can be any of 6)
MECHANISM OF MAILING
1) user_subscribed_revision
Suggested by @CatQuest , whenever a user make revision to any entity then they should automatically be added to user_subscribed_revision table and they should recieve email if anyone comments ( add note) on that revision .
- whenever a user hits the endpoint “/revisions/:id/note” the controller will find the subscribers of that revision from user_subscribed_revision table using revision_id and then send them mail.
2) user_subscribed_entity
Suppose a user subscribes a work entity ,I have assumed following mechanism in order to process subscription :-
- Any changes made to that entity will count as its revision , so whenever a new revision happens we will seek to the database user_subscribed_work database to find all its subscribers (BB user)
- We will find what we need to send in email by followig method :-
- work_revision table contains data_id column
- using that data_id we will find the final state of work and then
- Either we can list the changes made to the email body or simply we can use edit-note entered by user while editing the work entity + link of that revision
- And finally Use the mailer to send the mail to subcribers.
Likewise we will do the same for rest of entity( author, edition, edition_group, publisher, collection).
3) user_subscribed_user
From the discussio so far i have come to conclusion that , any entity modification ( create , delete , revision ) by the user will ultimately be notified to the its subscribers .
Scaling
The best way to scale our system is to scale it horizontaly and establish task-ques to store the jobs to be in bulk and to be processed by separate mailing service.
- I have a flowchart for the same :-
Setting Up RabbitMQ
- RabbitMQ will be setup using its docker image
- I have decided to setup this during community bonding period with some guidance of mentors
connection with RabbitMQ
- Will use
ampqlibinterface to do so
connect.js
import Interface from 'amqplib'
let channel = Interface.connect(URI)
.then( (connection)=>{ return connection.createChannel() })
.catch((error)
=>{ throw error})
export default channel
// this connection will be utillized by publisher and consumer to que and deque the jobs respectively
Publisher
publisher.js
import channel from 'connect.js'
const queName = 'subscriptions'
let job1 = {
email_id:endrance21@gmail.com // users email will be found by looking into `users table`
message:"some message body to be sent",
}
channel.assertQueue(queName)
.then(()=>{
channel.sendToQueue(queName,Buffer.from(JSON.stringify(job1) ,{ persistent: true } ); // will send jobs to the que
})
.catch(()=>{
//handle error !
})
Consumer
consumer.js
import channel from 'connect.js'
import mailer from "mailer.js"
const queName = 'subscriptions'
channel.assertQueue(queName)
.then(()=>{
return channel.consume(queName); // will consume jobs to the que
})
.then((job)=>{
let {content } = job;
let body = JSON.parse(content) ;
let {email,message} = body
mailer.sendMail({email,message}); //this will simply send mail to the given email
})
.catch(()=>{
//handle error !
})
Preferences :
For now :
- An user creating any entity wil automatically be subscribed to it using above apis
- If a user comments or edits any entity then they wil be automatically subscribed to that revision.
- An user can see others subscriptions and subscribers
Extended goal
Is To provide a prefernce page to user to decide if they want to subscribe to or not to their created entity or revison they make or any comment they make on any revison .
- user_preference table will be created
- columns will be decided according to prefernce type but they all will be type of bool .
- Apis to update the preference
- app.put(‘/preference/update’,controller)
- Payload will contain the final state of preference table collected through user via checkbox ui
- app.get('/preference/,controller)
- This will fetch the current state of user_preference table and used to display on users profile
- according to current state of prefernce_table controller will unscribe/subscribe entities .
Detailed information about yourself
Tell us about the computer(s) you have available for working on your SoC project!
A lenevo Legion ( i7, 10 gen + 24GB of RAM) , Separate Monitor along with laptop for multitasking
When did you first start programming?
In My freshmen year of College , i made a Audio Analyser tool using Vanilla JS.
What type of books do you read?
IT ENDS WITH US by COLLEEN HOOVER , NOVEMBER 9 by COLLEEN HOOVER AND IN Non-Fictious I have read ORIGINALS by ADAM GRANT
Have you contributed to other Open Source projects? If so, which projects and can we see some of your code?
I participated in GSOC 2020 as Student Developer , here is the work Product
How much time do you have available, and how would you plan to use it?
Official GSoC period is from June 7th to 16st August, during the vacation, I will be devoting most of my uptime to this project, 45-50 hrs a week, after re-opening of my college in July, I will be devoting 25-30 hrs a week.
TO DO’s
- Code snippets
- Gsoc timeline
- Error handling and managing enqueing system
Thank You !
