Text mining
text tokenizetext tokenizetext stopwordstext stemtext word_freqtext word_freqtext dtmtext dtmtext keywordstext keywordstext summarizetext summarizetext sentimenttext sentimenttext similaritytext similaritytext searchtext searchtext topicstext topicstext collocationstext collocationstext kwictext kwictext readabilitytext languagetext near_duplicatestext near_duplicatestext classifytext classifytext clustertext clustertext extracttext extract
text tokenize <text>
Description
Split a text into words: lowercase, accents removed, stop words removed (French by default), numbers removed, words shorter than 2 letters removed
Split a text into words: lowercase, accents removed, stop words removed (French by default), numbers removed, words shorter than 2 letters removed
Parameters
text: The text - String - required
text: The text - String - required
admin
text tokenize "Les chevaux de l'école courent très vite, mais les élèves n'ont pas peur !"
mentdb
{"count": 6, "tokens": ["chevaux", "ecole", "courent", "vite", "eleves", "peur"]}
text tokenize <text> <json_params>
Description
Tokens with parameters: lang (fr, en), lowercase, unaccent, stopwords, numbers, min_length, stem (Savoy for French, Porter for English)
Tokens with parameters: lang (fr, en), lowercase, unaccent, stopwords, numbers, min_length, stem (Savoy for French, Porter for English)
Parameters
text: The text - String - required
json_params: The parameters - String - required
text: The text - String - required
json_params: The parameters - String - required
admin
text tokenize "Les chevaux de l'école courent très vite" "{\"stem\": true}"
mentdb
{"count": 4, "tokens": ["cheval", "ecole", "courent", "vite"]}
text stopwords <lang>
Description
The stop words of a language (fr or en)
The stop words of a language (fr or en)
Parameters
lang: fr or en - String - required
lang: fr or en - String - required
admin
text stopwords "en"
mentdb
["a", "about", "above", "after", "again", "against", "all", "also", "am", "an", "and", "any", "are", "aren", "as", "at", "be", "because", "been", "before", "being", "below", "between", "both", "but", "by", "can", "cannot", "could", "couldn", "d", "did", "didn", "do", "does", "doesn", "doing", "don", "down", "during", "each", "few", "for", "from", "further", "had", "hadn", "has", "hasn", "have", "haven", "having", "he", "her", "here", "hers", "herself", "him", "himself", "his", "how", "i", "if", "in", "into", "is", "isn", "it", "its", "itself", "just", "let", "ll", "m", "may", "me", "might", "more", "most", "mustn", "my", "myself", "no", "nor", "not", "now", "of", "off", "on", "once", "only", "or", "other", "ought", "our", "ours", "ourselves", "out", "over", "own", "re", "s", "same", "shall", "shan", "she", "should", "shouldn", "so", "some", "such", "t", "than", "that", "the", "their", "theirs", "them", "themselves", "then", "there", "these", "they", "this", "those", "through", "to", "too", "under", "until", "up", "us", "ve", "very", "was", "wasn", "we", "were", "weren", "what", "when", "where", "which", "while", "who", "whom", "why", "will", "with", "won", "would", "wouldn", "you", "your", "yours", "yourself", "yourselves"]
text stem <word> <lang>
Description
Stem of a word: Porter (1980) for English, Savoy minimal stemmer for French (plurals, feminine)
Stem of a word: Porter (1980) for English, Savoy minimal stemmer for French (plurals, feminine)
Parameters
word: The word - String - required
lang: fr or en - String - required
word: The word - String - required
lang: fr or en - String - required
admin
text stem "generalization" "en"
mentdb
gener
admin
text stem "chevaux" "fr"
mentdb
cheval
text word_freq <json_documents>
Description
Word frequencies of documents: count, number of documents containing the word, share
Word frequencies of documents: count, number of documents containing the word, share
Parameters
json_documents: The documents - String - required
json_documents: The documents - String - required
admin
text word_freq "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]"
mentdb
{"documents": 6, "tokens": 35, "vocabulary": 30, "words": [{"word": "panne", "count": 3, "documents": 3, "share": 0.08571428571428572}, {"word": "client", "count": 2, "documents": 2, "share": 0.05714285714285714}, {"word": "facture", "count": 2, "documents": 2, "share": 0.05714285714285714}, {"word": "moteur", "count": 2, "documents": 2, "share": 0.05714285714285714}, {"word": "autoroute", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "bureau", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "courant", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "courroie", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "doit", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "envoyee", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "fevrier", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "garage", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "hier", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "livraison", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "mars", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "merci", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "nouvelle", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "paiement", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "payee", "count": 1, "documents": 1, "share": 0.02857142857142857}, {"word": "rapide", "count": 1, "documents": 1, "share": 0.02857142857142857}]}
text word_freq <json_documents> <json_params>
Description
Word frequencies with parameters: top (20) and the token parameters (lang, stem, stopwords...)
Word frequencies with parameters: top (20) and the token parameters (lang, stem, stopwords...)
Parameters
json_documents: The documents - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text word_freq "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "{\"top\": 5, \"stem\": true}"
mentdb
{"documents": 6, "tokens": 35, "vocabulary": 29, "words": [{"word": "panne", "count": 3, "documents": 3, "share": 0.08571428571428572}, {"word": "client", "count": 2, "documents": 2, "share": 0.05714285714285714}, {"word": "factur", "count": 2, "documents": 2, "share": 0.05714285714285714}, {"word": "moteu", "count": 2, "documents": 2, "share": 0.05714285714285714}, {"word": "relanc", "count": 2, "documents": 2, "share": 0.05714285714285714}]}
text dtm <json_documents>
Description
Document-term matrix (term counts): the vocabulary and one row per document, ready for ml train or ml fit
Document-term matrix (term counts): the vocabulary and one row per document, ready for ml train or ml fit
Parameters
json_documents: The documents - String - required
json_documents: The documents - String - required
admin
text dtm "[\"le chat dort\", \"le chien dort\", \"le chat mange\"]"
mentdb
{"documents": 3, "vocabulary": ["chat", "chien", "dort", "mange"], "matrix": [[1, 0, 1, 0], [0, 1, 1, 0], [1, 0, 0, 1]]}
text dtm <json_documents> <json_params>
Description
Document-term matrix with parameters: weighting (tf, tfidf, binary), min_df, max_features (1000) and the token parameters
Document-term matrix with parameters: weighting (tf, tfidf, binary), min_df, max_features (1000) and the token parameters
Parameters
json_documents: The documents - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text dtm "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "{\"weighting\": \"tfidf\", \"min_df\": 2}"
mentdb
{"documents": 6, "vocabulary": ["client", "facture", "moteur", "panne"], "matrix": [[0, 0, 0.764096101185661, 0.6451024322949592], [0.7071067811865476, 0.7071067811865476, 0, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, 0, 0.764096101185661, 0.6451024322949592], [0, 1, 0, 0]]}
text keywords <text>
Description
Keywords of a text by TextRank (graph of the co-occurrences)
Keywords of a text by TextRank (graph of the co-occurrences)
Parameters
text: The text - String - required
text: The text - String - required
admin
text keywords "Le machine learning permet de construire des modèles prédictifs à partir des données. Les entreprises utilisent ces modèles pour anticiper la demande. La qualité des données reste le facteur principal de réussite. Un modèle entraîné sur des données biaisées donne des prédictions biaisées. Il faut donc nettoyer, valider et documenter les données avant tout apprentissage."
mentdb
{"method": "textrank", "keywords": [{"keyword": "donnees", "score": 0.11000858607359185}, {"keyword": "modeles", "score": 0.06279804846602235}, {"keyword": "biaisees", "score": 0.05925444755022994}, {"keyword": "permet", "score": 0.038165131467265676}, {"keyword": "construire", "score": 0.03609360332659851}, {"keyword": "documenter", "score": 0.033856064589681434}, {"keyword": "principal", "score": 0.0337282945801648}, {"keyword": "reussite", "score": 0.033604927601340005}, {"keyword": "facteur", "score": 0.03309555321354174}, {"keyword": "predictifs", "score": 0.03299840385551386}]}
text keywords <text> <json_params>
Description
Keywords with parameters: method (textrank, rake: key phrases, tfidf: against a corpus), top, window, corpus (other documents for tfidf), lang
Keywords with parameters: method (textrank, rake: key phrases, tfidf: against a corpus), top, window, corpus (other documents for tfidf), lang
Parameters
text: The text - String - required
json_params: The parameters - String - required
text: The text - String - required
json_params: The parameters - String - required
admin
text keywords "Le machine learning permet de construire des modèles prédictifs à partir des données. Les entreprises utilisent ces modèles pour anticiper la demande. La qualité des données reste le facteur principal de réussite. Un modèle entraîné sur des données biaisées donne des prédictions biaisées. Il faut donc nettoyer, valider et documenter les données avant tout apprentissage." "{\"method\": \"rake\", \"top\": 5}"
mentdb
{"method": "rake", "keywords": [{"keyword": "machine learning permet", "score": 9}, {"keyword": "donnees biaisees donne", "score": 7.25}, {"keyword": "predictions biaisees", "score": 4.5}, {"keyword": "entreprises utilisent", "score": 4}, {"keyword": "facteur principal", "score": 4}]}
Used in use cases (1 recipe)
Data scientist › Customer messages: routing, sentiment and recurring topics
Data scientist › Customer messages: routing, sentiment and recurring topics
text summarize <text>
Description
Extractive summary by TextRank: the most central sentences, in their original order
Extractive summary by TextRank: the most central sentences, in their original order
Parameters
text: The text - String - required
text: The text - String - required
admin
text summarize "Le machine learning permet de construire des modèles prédictifs à partir des données. Les entreprises utilisent ces modèles pour anticiper la demande. La qualité des données reste le facteur principal de réussite. Un modèle entraîné sur des données biaisées donne des prédictions biaisées. Il faut donc nettoyer, valider et documenter les données avant tout apprentissage."
mentdb
{"sentences": 5, "summary": "Le machine learning permet de construire des modèles prédictifs à partir des données. La qualité des données reste le facteur principal de réussite. Un modèle entraîné sur des données biaisées donne des prédictions biaisées.", "selected": [0, 2, 3], "scores": [0.26038240497222953, 0.12474642317978736, 0.16930140985605013, 0.27626835213588274, 0.16930140985605013]}
text summarize <text> <json_params>
Description
Summary with parameters: sentences (3), lang
Summary with parameters: sentences (3), lang
Parameters
text: The text - String - required
json_params: The parameters - String - required
text: The text - String - required
json_params: The parameters - String - required
admin
text summarize "Le machine learning permet de construire des modèles prédictifs à partir des données. Les entreprises utilisent ces modèles pour anticiper la demande. La qualité des données reste le facteur principal de réussite. Un modèle entraîné sur des données biaisées donne des prédictions biaisées. Il faut donc nettoyer, valider et documenter les données avant tout apprentissage." "{\"sentences\": 2}"
mentdb
{"sentences": 5, "summary": "Le machine learning permet de construire des modèles prédictifs à partir des données. Un modèle entraîné sur des données biaisées donne des prédictions biaisées.", "selected": [0, 3], "scores": [0.26038240497222953, 0.12474642317978736, 0.16930140985605013, 0.27626835213588274, 0.16930140985605013]}
text sentiment <text>
Description
Sentiment of a text from a French lexicon (weights -3 to 3) with negations and intensifiers: score, compound in [-1, 1], label, the words found
Sentiment of a text from a French lexicon (weights -3 to 3) with negations and intensifiers: score, compound in [-1, 1], label, the words found
Parameters
text: The text - String - required
text: The text - String - required
admin
text sentiment "Le service était vraiment excellent, mais la livraison n'était pas rapide."
mentdb
{"score": 3.5, "compound": 0.6704783996548059, "label": "positive", "positive_words": 1, "negative_words": 1, "words": [{"word": "excellent", "weight": 4.5, "negated": false}, {"word": "rapide", "weight": -1, "negated": true}]}
text sentiment <text> <json_params>
Description
Sentiment with parameters: lang (fr, en), lexicon (your own words and weights, added to the default lexicon)
Sentiment with parameters: lang (fr, en), lexicon (your own words and weights, added to the default lexicon)
Parameters
text: The text - String - required
json_params: The parameters - String - required
text: The text - String - required
json_params: The parameters - String - required
admin
text sentiment "I did not like this product, it is really bad." "{\"lang\": \"en\"}"
mentdb
{"score": -4, "compound": -0.7184212081070996, "label": "negative", "positive_words": 0, "negative_words": 2, "words": [{"word": "like", "weight": -1, "negated": true}, {"word": "bad", "weight": -3, "negated": false}]}
Domain words
admin
text sentiment "Le logiciel plante encore" "{\"lexicon\": {\"plante\": -2}}"
mentdb
{"score": -2, "compound": -0.4588314677411235, "label": "negative", "positive_words": 0, "negative_words": 1, "words": [{"word": "plante", "weight": -2, "negated": false}]}
Used in use cases (1 recipe)
Data scientist › Customer messages: routing, sentiment and recurring topics
Data scientist › Customer messages: routing, sentiment and recurring topics
text similarity <text_a> <text_b>
Description
Similarity of two texts: cosine of the word counts
Similarity of two texts: cosine of the word counts
Parameters
text_a: The first text - String - required
text_b: The second text - String - required
text_a: The first text - String - required
text_b: The second text - String - required
admin
text similarity "le moteur est en panne" "panne du moteur ce matin"
mentdb
{"method": "cosine", "similarity": 0.8164965809277261}
text similarity <text_a> <text_b> <json_params>
Description
Similarity with parameters: method (cosine, jaccard), lang, stem
Similarity with parameters: method (cosine, jaccard), lang, stem
Parameters
text_a: The first text - String - required
text_b: The second text - String - required
json_params: The parameters - String - required
text_a: The first text - String - required
text_b: The second text - String - required
json_params: The parameters - String - required
admin
text similarity "les moteurs sont en panne" "panne du moteur" "{\"method\": \"jaccard\", \"stem\": true}"
mentdb
{"method": "jaccard", "similarity": 1}
text search <query> <json_documents>
Description
Search engine: BM25 ranking of the documents for a query (stemmed words), with the score of each result
Search engine: BM25 ranking of the documents for a query (stemmed words), with the score of each result
Parameters
query: The query - String - required
json_documents: The documents - String - required
query: The query - String - required
json_documents: The documents - String - required
admin
text search "panne moteur" "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]"
mentdb
{"query_terms": ["panne", "moteu"], "results": [{"index": 0, "score": 1.7008979103226691, "text": "Le moteur est tombé en panne hier soir sur l'autoroute."}, {"index": 4, "score": 1.5805198144413795, "text": "Nouvelle panne du moteur : le garage doit remplacer la courroie."}, {"index": 2, "score": 0.740767979224369, "text": "Panne de courant au bureau, le serveur a redémarré."}]}
text search <query> <json_documents> <json_params>
Description
Search with parameters: top (5), k1 (1.5), b (0.75), lang, stem
Search with parameters: top (5), k1 (1.5), b (0.75), lang, stem
Parameters
query: The query - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
query: The query - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text search "facture paiement" "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "{\"top\": 2}"
mentdb
{"query_terms": ["factur", "paiement"], "results": [{"index": 5, "score": 2.746633772045519, "text": "Relance de paiement envoyée pour la facture de février."}, {"index": 1, "score": 0.9446049698909709, "text": "La facture de mars n'est toujours pas payée, merci de relancer le client."}]}
text topics <json_documents>
Description
Topics of documents by LDA (latent Dirichlet allocation, collapsed Gibbs sampling): the words of each topic and the topics of each document
Topics of documents by LDA (latent Dirichlet allocation, collapsed Gibbs sampling): the words of each topic and the topics of each document
Parameters
json_documents: The documents - String - required
json_documents: The documents - String - required
admin
text topics "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]"
mentdb
{"k": 3, "vocabulary": 4, "topics": [{"topic": 0, "share": 0.4444444444444444, "words": [{"word": "client", "weight": 0.49752475247524747}, {"word": "facture", "weight": 0.49752475247524747}, {"word": "moteur", "weight": 0.0024752475247524753}, {"word": "panne", "weight": 0.0024752475247524753}]}, {"topic": 1, "share": 0.3333333333333333, "words": [{"word": "panne", "weight": 0.9901315789473684}, {"word": "client", "weight": 0.003289473684210526}, {"word": "facture", "weight": 0.003289473684210526}, {"word": "moteur", "weight": 0.003289473684210526}]}, {"topic": 2, "share": 0.2222222222222222, "words": [{"word": "moteur", "weight": 0.9852941176470587}, {"word": "client", "weight": 0.004901960784313725}, {"word": "facture", "weight": 0.004901960784313725}, {"word": "panne", "weight": 0.004901960784313725}]}], "document_topics": [[0.2380952380952381, 0.380952380952381, 0.380952380952381], [0.5238095238095238, 0.2380952380952381, 0.2380952380952381], [0.2777777777777778, 0.4444444444444445, 0.2777777777777778], [0.4444444444444445, 0.2777777777777778, 0.2777777777777778], [0.2380952380952381, 0.380952380952381, 0.380952380952381], [0.4444444444444445, 0.2777777777777778, 0.2777777777777778]]}
text topics <json_documents> <json_params>
Description
Topics with parameters: k (3), iterations (500), alpha, beta, top_words (8), min_df (2), max_features, seed, lang, stem
Topics with parameters: k (3), iterations (500), alpha, beta, top_words (8), min_df (2), max_features, seed, lang, stem
Parameters
json_documents: The documents - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text topics "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "{\"k\": 2, \"min_df\": 1, \"stem\": true, \"top_words\": 4}"
mentdb
{"k": 2, "vocabulary": 29, "topics": [{"topic": 0, "share": 0.42857142857142855, "words": [{"word": "relanc", "weight": 0.13145846958796598}, {"word": "bureau", "weight": 0.06605624591236102}, {"word": "courant", "weight": 0.06605624591236102}, {"word": "courroi", "weight": 0.06605624591236102}]}, {"topic": 1, "share": 0.5714285714285714, "words": [{"word": "panne", "weight": 0.1483489403647117}, {"word": "client", "weight": 0.09906357811729916}, {"word": "factur", "weight": 0.09906357811729916}, {"word": "moteu", "weight": 0.09906357811729916}]}], "document_topics": [[0.22727272727272727, 0.7727272727272727], [0.5416666666666666, 0.4583333333333333], [0.55, 0.45], [0.55, 0.45], [0.375, 0.625], [0.55, 0.45]]}
text collocations <json_documents>
Description
Collocations: the pairs of words that appear together more than by chance (Dunning log-likelihood G2 and PMI)
Collocations: the pairs of words that appear together more than by chance (Dunning log-likelihood G2 and PMI)
Parameters
json_documents: The documents - String - required
json_documents: The documents - String - required
admin
text collocations "[\"machine learning et deep learning\", \"le machine learning en production\", \"machine learning appliqué\", \"apprentissage du machine learning\"]"
mentdb
{"tokens": 13, "collocations": [{"bigram": "machine learning", "count": 4, "pmi": 1.37851162325373, "g2": 11.044261777607005}]}
text collocations <json_documents> <json_params>
Description
Collocations with parameters: min_count (2), top (20), lang, stopwords
Collocations with parameters: min_count (2), top (20), lang, stopwords
Parameters
json_documents: The documents - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text collocations "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "{\"min_count\": 2}"
mentdb
{"tokens": 35, "collocations": []}
text kwic <json_documents> <word>
Description
Keyword in context: each occurrence of a word with the words before and after (concordance)
Keyword in context: each occurrence of a word with the words before and after (concordance)
Parameters
json_documents: The documents - String - required
word: The word - String - required
json_documents: The documents - String - required
word: The word - String - required
admin
text kwic "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "panne"
mentdb
{"matches": 3, "lines": [{"document": 0, "left": "Le moteur est tombé en", "word": "panne", "right": "hier soir sur l'autoroute"}, {"document": 2, "left": "", "word": "Panne", "right": "de courant au bureau le"}, {"document": 4, "left": "Nouvelle", "word": "panne", "right": "du moteur le garage doit"}]}
text kwic <json_documents> <word> <json_params>
Description
Keyword in context with parameters: window (5 words)
Keyword in context with parameters: window (5 words)
Parameters
json_documents: The documents - String - required
word: The word - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
word: The word - String - required
json_params: The parameters - String - required
admin
text kwic "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "facture" "{\"window\": 3}"
mentdb
{"matches": 2, "lines": [{"document": 1, "left": "La", "word": "facture", "right": "de mars n'est"}, {"document": 5, "left": "envoyée pour la", "word": "facture", "right": "de février"}]}
text readability <text> <lang>
Description
Readability: Flesch reading ease (English) or Kandel and Moles (French), words per sentence, syllables per word
Readability: Flesch reading ease (English) or Kandel and Moles (French), words per sentence, syllables per word
Parameters
text: The text - String - required
lang: fr or en - String - required
text: The text - String - required
lang: fr or en - String - required
admin
text readability "Le machine learning permet de construire des modèles prédictifs à partir des données. Les entreprises utilisent ces modèles pour anticiper la demande. La qualité des données reste le facteur principal de réussite. Un modèle entraîné sur des données biaisées donne des prédictions biaisées. Il faut donc nettoyer, valider et documenter les données avant tout apprentissage." "fr"
mentdb
{"sentences": 5, "words": 55, "syllables": 101, "words_per_sentence": 11, "syllables_per_word": 1.8363636363636364, "long_words_percent": 47.27272727272727, "score": 60.67863636363637, "formula": "Kandel and Moles (Flesch adapted to French)", "level": "easy to standard"}
text language <text>
Description
Language of a text (French, English, Spanish, German, Italian, Portuguese) from its function words, with a confidence
Language of a text (French, English, Spanish, German, Italian, Portuguese) from its function words, with a confidence
Parameters
text: The text - String - required
text: The text - String - required
admin
text language "This is a short text written in English to test the language detection."
mentdb
{"language": "en", "confidence": 0.8, "scores": {"de": 0, "en": 0.38461538461538464, "es": 0, "fr": 0, "it": 0.07692307692307693, "pt": 0.07692307692307693}}
text near_duplicates <json_documents>
Description
Near duplicate documents: Jaccard similarity of the word shingles (3 words), the pairs above 0.8
Near duplicate documents: Jaccard similarity of the word shingles (3 words), the pairs above 0.8
Parameters
json_documents: The documents - String - required
json_documents: The documents - String - required
admin
text near_duplicates "[\"le chat dort sur le tapis rouge du salon\", \"le chat dort sur le tapis rouge du salon ce soir\", \"une phrase différente\"]"
mentdb
{"documents": 3, "pairs": []}
text near_duplicates <json_documents> <json_params>
Description
Near duplicates with parameters: shingle (words per shingle, 3), threshold (0.8)
Near duplicates with parameters: shingle (words per shingle, 3), threshold (0.8)
Parameters
json_documents: The documents - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text near_duplicates "[\"le chat dort sur le tapis rouge du salon\", \"le chat dort sur le tapis rouge du salon ce soir\"]" "{\"threshold\": 0.6, \"shingle\": 2}"
mentdb
{"documents": 2, "pairs": [{"a": 0, "b": 1, "jaccard": 0.8}]}
text classify <json_training> <json_labels> <json_new>
Description
Classification of texts by multinomial naive Bayes trained on labeled documents: label and probabilities of each new document
Classification of texts by multinomial naive Bayes trained on labeled documents: label and probabilities of each new document
Parameters
json_training: The training documents - String - required
json_labels: Their labels - String - required
json_new: The documents to classify - String - required
json_training: The training documents - String - required
json_labels: Their labels - String - required
json_new: The documents to classify - String - required
admin
text classify "[\"facture impayée relance paiement\", \"remboursement paiement carte\", \"panne écran ordinateur\", \"bug logiciel plantage\"]" "[\"compta\", \"compta\", \"support\", \"support\"]" "[\"le logiciel plante au démarrage\", \"relance pour la facture de mars\"]"
mentdb
{"classes": 2, "vocabulary": 12, "predictions": [{"label": "support", "probability": 0.6785714285714286, "probabilities": {"compta": 0.32142857142857145, "support": 0.6785714285714286}}, {"label": "compta", "probability": 0.7821363910681954, "probabilities": {"compta": 0.7821363910681954, "support": 0.21786360893180443}}]}
Complete: a ticket table -> texts and teams -> routing of the new tickets
admin
-> "[rows]" "[{\"text\": \"la facture est fausse\", \"team\": \"billing\"}, {\"text\": \"remboursement non recu\", \"team\": \"billing\"}, {\"text\": \"double prelevement carte\", \"team\": \"billing\"}, {\"text\": \"colis arrive casse\", \"team\": \"delivery\"}, {\"text\": \"livraison en retard\", \"team\": \"delivery\"}, {\"text\": \"colis jamais recu\", \"team\": \"delivery\"}, {\"text\": \"application plante\", \"team\": \"technical\"}, {\"text\": \"impossible de se connecter\", \"team\": \"technical\"}, {\"text\": \"erreur au paiement sur le site\", \"team\": \"technical\"}]"; json load "tuto_c" (text classify (ml prep vector [rows] "text") (ml prep vector [rows] "team") "[\"mon colis est en retard\", \"la facture de mars est fausse\"]"); -> "[res]" (concat (json select "tuto_c" "/predictions/[0]/label") ", " (json select "tuto_c" "/predictions/[1]/label")); json unload "tuto_c"; [res]
mentdb
delivery, billing
text classify <json_training> <json_labels> <json_new> <json_params>
Description
Classification with parameters: alpha (Laplace smoothing, 1), lang, stem, stopwords
Classification with parameters: alpha (Laplace smoothing, 1), lang, stem, stopwords
Parameters
json_training: The training documents - String - required
json_labels: The labels - String - required
json_new: The new documents - String - required
json_params: The parameters - String - required
json_training: The training documents - String - required
json_labels: The labels - String - required
json_new: The new documents - String - required
json_params: The parameters - String - required
admin
text classify "[\"great product\", \"bad service\", \"excellent quality\", \"terrible delay\"]" "[\"pos\", \"neg\", \"pos\", \"neg\"]" "[\"great quality\"]" "{\"lang\": \"en\"}"
mentdb
{"classes": 2, "vocabulary": 8, "predictions": [{"label": "pos", "probability": 0.8, "probabilities": {"neg": 0.19999999999999996, "pos": 0.8}}]}
Used in use cases (1 recipe)
Data scientist › Customer messages: routing, sentiment and recurring topics
Data scientist › Customer messages: routing, sentiment and recurring topics
text cluster <json_documents>
Description
Clustering of documents (spherical k-means on TF-IDF, cosine): the cluster of each document and its top words
Clustering of documents (spherical k-means on TF-IDF, cosine): the cluster of each document and its top words
Parameters
json_documents: The documents - String - required
json_documents: The documents - String - required
admin
text cluster "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]"
mentdb
{"k": 3, "clusters": [{"cluster": 0, "size": 2, "top_words": ["bureau", "courant", "redemarre", "serveur", "envoyee", "fevrier"]}, {"cluster": 1, "size": 2, "top_words": ["moteur", "panne", "autoroute", "hier", "soir", "tombe"]}, {"cluster": 2, "size": 2, "top_words": ["client", "livraison", "rapide", "satisfait", "service", "mars"]}], "labels": [1, 2, 0, 2, 1, 0], "cohesion": 0.7442269768092332}
text cluster <json_documents> <json_params>
Description
Clustering with parameters: k (3), min_df, top_words, seed, lang, stem
Clustering with parameters: k (3), min_df, top_words, seed, lang, stem
Parameters
json_documents: The documents - String - required
json_params: The parameters - String - required
json_documents: The documents - String - required
json_params: The parameters - String - required
admin
text cluster "[\"Le moteur est tomb\u00e9 en panne hier soir sur l'autoroute.\", \"La facture de mars n'est toujours pas pay\u00e9e, merci de relancer le client.\", \"Panne de courant au bureau, le serveur a red\u00e9marr\u00e9.\", \"Le client est tr\u00e8s satisfait du service et de la livraison rapide.\", \"Nouvelle panne du moteur : le garage doit remplacer la courroie.\", \"Relance de paiement envoy\u00e9e pour la facture de f\u00e9vrier.\"]" "{\"k\": 2, \"stem\": true}"
mentdb
{"k": 2, "clusters": [{"cluster": 0, "size": 3, "top_words": ["panne", "moteu", "bureau", "courant", "redemar", "serveu"]}, {"cluster": 1, "size": 3, "top_words": ["factur", "relanc", "client", "envoye", "fevri", "paiement"]}], "labels": [0, 1, 0, 1, 0, 1], "cohesion": 0.6485451649729835}
text extract <text>
Description
Extract the entities of a text: emails, urls, IBAN, phones, dates, amounts, percentages, IP addresses, hashtags, mentions
Extract the entities of a text: emails, urls, IBAN, phones, dates, amounts, percentages, IP addresses, hashtags, mentions
Parameters
text: The text - String - required
text: The text - String - required
admin
text extract "Contact : jean.dupont@exemple.fr ou +33 6 12 34 56 78, site https://www.mentdb.org, facture 1 250,50 € du 12/10/2026, remise 15 % #promo"
mentdb
{"email": ["jean.dupont@exemple.fr"], "url": ["https:\/\/www.mentdb.org"], "iban": [], "phone": ["+33 6 12 34 56 78"], "date": ["12\/10\/2026"], "amount": ["1 250,50 \u20AC"], "percent": ["15 %"], "ip": [], "hashtag": ["#promo"], "mention": []}
text extract <text> <json_params>
Description
Extraction with parameters: types (the list of the types to keep)
Extraction with parameters: types (the list of the types to keep)
Parameters
text: The text - String - required
json_params: The parameters - String - required
text: The text - String - required
json_params: The parameters - String - required
admin
text extract "Appelez le 02 62 12 34 56 ou écrivez à support@exemple.re" "{\"types\": [\"phone\", \"email\"]}"
mentdb
{"email": ["support@exemple.re"], "phone": ["02 62 12 34 56"]}
