# GSoC 2018: SpamBrainz - Fighting spam with machine learning

**URL:** https://community.metabrainz.org/t/gsoc-2018-spambrainz-fighting-spam-with-machine-learning/370202
**Category:** GSoC applications
**Tags:** gsoc-2018, spambrainz
**Created:** [March 26, 2018, 7:14pm UTC](https://community.metabrainz.org/t/gsoc-2018-spambrainz-fighting-spam-with-machine-learning/370202 "2018-03-26T19:14:11Z")
**Posts on this page:** 1
**Showing post:** 4

<div class="post-metadata">

### Author: ![Leo\_Verto](https://community.metabrainz.org/user_avatar/community.metabrainz.org/leo_verto/32/42_2.png) [@Leo\_Verto](https://community.metabrainz.org/u/Leo_Verto)
#### Post date: [June 14, 2018, 6:42pm UTC](https://community.metabrainz.org/t/gsoc-2018-spambrainz-fighting-spam-with-machine-learning/370202/4 "2018-06-14T18:42:43Z")

</div>

# Notebooks, references and revised timeline

## Notebooks:

- [WIP Spam/nonspam editor comparison](https://github.com/metabrainz/spambrainz/blob/master/data-analysis/editor-comparison.ipynb)
- [Editor statistics collection](https://github.com/metabrainz/spambrainz/blob/master/data-analysis/editor-statistics-collection.ipynb)
- [Dataviz playground](https://github.com/metabrainz/spambrainz/blob/master/data-analysis/analysis-playground.ipynb)

## References

### Papers/scientific articles

- Bhowmick, Hazarika (2016) [**Machine Learning for E-mail Spam Filtering: Review,  
Techniques and Trends**](https://arxiv.org/pdf/1606.01042.pdf)
- Caruana, Li (2012) [A survey of emerging approaches to spam filtering](https://dl.acm.org/citation.cfm?doid=2089125.2089129)
- Arif, Li et al. (2017) [Sentiment analysis and spam detection in short informal text using learning classifier systems](https://link.springer.com/article/10.1007%2Fs00500-017-2729-x)
- Giyanani, Desai (2014) [Spam Detection using Natural Language Processing](http://www.iosrjournals.org/iosr-jce/papers/Vol16-issue5/Version-4/S01654116119.pdf)
- Androutsopoulos, Paliouras et al. (2000) [Learning to Filter Spam E-Mail: A Comparison of a Naive  
Bayesian and a Memory-Based Approach](https://arxiv.org/pdf/cs/0009009.pdf)
- Santosh, Maity, Mukherjee (2017) [ENWalk: Learning Network Features for Spam  
Detection in Twitter](https://arxiv.org/pdf/1704.03404.pdf)
- Miikkulainen, Liang et al. (2017) [Evolving Deep Neural Networks](https://arxiv.org/pdf/1703.00548.pdf)
- Ren, Triantafillou et al. (2018) [Meta-Learning for Semi-Supervised Few-Shot Classification](https://arxiv.org/pdf/1803.00676.pdf)
- Chen, Shah (2018) [Explaining the Success of Nearest Neighbor Methods in Prediction](https://devavrat.mit.edu/wp-content/uploads/2018/03/nn_survey.pdf)

### Blog posts

- [Building a Spam Filter Using Machine Learning](https://www.booleanworld.com/building-spam-filter-using-machine-learning/)
- [Spam detection using neural networks in Python](https://medium.com/emergent-future/spam-detection-using-neural-networks-in-python-9b2b2a062272)
- [How To Build a Simple Spam-Detecting Machine Learning Classifier](https://hackernoon.com/how-to-build-a-simple-spam-detecting-machine-learning-classifier-4471fe6b816e)
- [Naive Bayes and Spam Detection](https://opendatascience.com/naive-bayes-and-spam-detection/)
- [A TensorFlow Tutorial: Email Classification](https://jrmeyer.github.io/machinelearning/2016/02/01/TensorFlow-Tutorial.html)
- [Deceptive Spam Review Detection with CNN Using TensorFlow](https://migsena.com/deceptive_spam_part_1/)
- [Implementing a CNN for Text Classification in TensorFlow](http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/)
- [Introduction to Natural Language Processing (NLP)](https://blog.algorithmia.com/introduction-natural-language-processing-nlp/)
- [The Mobile Neural Network Lottery: How to Find Your Winning Ticket](https://heartbeat.fritz.ai/the-mobile-neural-network-lottery-how-to-find-your-winning-ticket-dcbd97ddfe20)

### Repositories

- [Tensorflow email spam detection](https://github.com/MoAbd/Spam-detection)
- [Detecting Spam Emails using CNN](https://github.com/dbsheta/spam-detection-using-deep-learning)
- [Convolutional Neural Network for Text Classification in Tensorflow](https://github.com/dennybritz/cnn-text-classification-tf)

### Books

- Ng, Andrew (unpublished): Machine Learning Yearning

### GDPR-Related

- [GDPR and Data Processing Agreements](https://www.netskope.com/blog/gdpr-data-processing-agreements/)
- [Will GDPR Make Machine Learning Illegal?](https://www.kdnuggets.com/2018/03/gdpr-machine-learning-illegal.html)
- Specifically [GDPR Article 6](https://gdpr-info.eu/art-6-gdpr/)

## Revised timeline

- Weeks 0-3: Researched existing projects, state-of-the-art of spam detection and machine learning.
- Week 1: Designed SpamBrainz’ project structure and started work on monitoring and management web interface.
- Week 3: Researched potential GDPR issues regarding SpamBrainz.
- Weeks 4-5: Received Spam data set, analyzed, compared and visualized spam/non-spam editor data.
- Week 6-7: Build the API and the SpamBrainz backend which the detection modules can plug into and a predictable dummy module to test it.
- Weeks 8-11: Create the machine learning module and tweak the hyperparameters to hopefully get it to a usable state.
- Week 12: Documentation, [designing a simple website](https://tickets.metabrainz.org/browse/SPAM-3) explaining SpamBrainz to our end users.
- Week 13: Buffer period.

---

_[View the full topic](https://community.metabrainz.org/t/gsoc-2018-spambrainz-fighting-spam-with-machine-learning/370202)._