NLP
Finding themes in social conversation
Exploring ten discussion themes in December 2017 #MeToo tweets through unsupervised topic modeling.
- 398,670 source rows before cleaning
- 10 topics
Exploring a large collection without predefined labels
This project used unsupervised topic modeling to explore recurring language in a collection of December 2017 #MeToo tweets. Instead of assigning posts to categories in advance, the LDA model learned ten statistical groups of words from the prepared text.
Text preprocessing, dictionary filtering, and a bag-of-words representation created the model inputs. The resulting term weights, word clouds, and dominant-topic distribution offer complementary views: what words characterize a topic, and how often a topic receives the strongest assignment in the processed collection. Human interpretation supplies the descriptive themes.
The work
Prepared tweets for analysis, created a filtered dictionary and bag-of-words corpus, and trained a ten-topic LDA model with Gensim.
The output
Topic word clouds and a distribution chart showed a dominant theme around the movement and shared stories, alongside political discussion, media reactions, and personal reflections.
Explore the ten topics
Select a topic to inspect its saved terms and weights. Larger weights indicate stronger representation within that topic, not the percentage of people who hold an opinion. Compare several topics before reading the word cloud and distribution chart. Topic numbers are identifiers, and these results are neither sentiment scores nor a representative summary of everyone involved in the movement.
