NLP Topic Modeling

Barbara Ma@quantiai

Company spent lots of resources collecting customer experiences surveys and also spent thousands of dollars to implement the text analytics platform.

But the key insight is missing from this platform — the key issues customers are complaining about so that the decision makers can be informed the voices of customers timely and develop plans to improve based on this crucial insight.

NLP topic modeling fits the bill.

The common approaches from topic modeling are LDA and NMF. LDA is a probabilistic approach while NMF is a deterministic approach.

I have experimented with both approaches. The outputs from NMF are more insightful with thw couple of datasets I have experimented with.

The procedure are as follows:

1. Convert to a tf-idf matrix.

2. Obtain coherence score to get idea of optimal topic numbers.

2. Apply Non-negative matrix decomposition.

3. Obtain the top relevant words and top representative sentences for each topic.

After the modeling part finished, I have also developed a web application to facilitate the usage. You may just upload the data to the portal, the app will show the important topics identified, and the key words associated with each topic, as well as three most representative comments or reviews for each topic. You may check out the app at https://doc-nlp-ai.ew.r.appspot.com.

You may leave any comments and feedbacks to quant@quantiai.one.