Case 02 / Recommendation AI · RAG · Sentiment classification
Skincare AI: recommendations, RAG & sentiment classification.
A Turkish assistant that recommends products from customer review evidence, answers questions with retrieved sources, and classifies review sentiment and topics. Original quotations stay visible alongside the model's interpretation.
- mDeBERTa-v3
- LoRA
- Sentiment Classification
- Multilingual E5
- BM25
- RAG
- Qwen 4B
- PyTorch
- Streamlit
- 01 / UNDERSTANDRead the requestProduct type, skincare concerns, and explicit skin type
- 02 / RETRIEVEFind review evidenceBM25 and semantic E5 search with product filters
- 03 / ASSESSCheck the experiencesQwen evaluates request-specific support and cautions
- 04 / EXPLAINShow products and sourcesShortlists, cited summaries, and original quotations
- Sentiment test agreement
- 88.18%against existing classifier annotations
- Sentiment macro F1
- 0.7935selected main model · fixed test predictions
- Classifier outputs
- 3 + 10sentiment labels and overlapping topic groups
Choose products through customer experience.
A person looking for a cleanser for acne-prone skin needs more than a product name or a broadly positive review. The relevant question is whether customers describe experiences that support that particular need, and whether other reviews give reasons for caution.
This project connects retrieval, language-model assessment, and a separately trained text classifier in a Turkish Streamlit assistant. Product suggestions come from available listings and their review evidence; search and chat expose the underlying records so the answer can be inspected.
Classify text, retrieve evidence, then explain.
| Task | Model or method | What it produces |
|---|---|---|
| Sentiment and topics | Fine-tuned mDeBERTa-v3-base with LoRA | Negative, neutral, or positive sentiment for reviews; overlapping topics for reviews and questions |
| Evidence retrieval | BM25 + frozen multilingual E5-small | Original records found by words, meaning, or fused rankings |
| RAG answers | Local Qwen 4B | Turkish summaries with citations and matching source quotations |
| Product recommendations | Retrieval + Qwen review assessment | Ranked listings supported by relevant experiences, with cautions |
The classifier is an encoder with two prediction heads, not a text-generation model. LoRA adapts query and value projections; pooled representations feed the sentiment and multilabel topic heads. Missing task labels are masked during training. Questions receive topic predictions only, because they lack sentiment supervision.
E5 and Qwen are pretrained multilingual models used for inference. Only the sentiment/topic classifier was fine-tuned here. Its ten manually designed topic groups include effectiveness, irritation, acne, texture, skin suitability, routine compatibility, ingredients and authenticity, expiry and storage, packaging and delivery, and value and volume.
A positive review is not always relevant support.
The request parser identifies product type and needs from the message and follow-up context. Retrieval discovers candidate products and collects distinct reviews for each listing. Qwen assesses relevance, favorable or adverse experience, and explicitly supported skin-type matches for the current request.
The default shortlist shows up to three products, each requiring at least two distinct, directly relevant favorable review texts. Relevant negative and mixed experiences remain visible. Reviews from different listings are not pooled, and customer questions are not treated as evidence that an outcome occurred.
Evidence search and chat use one flow: Qwen summary plus direct quotations. Generated claims must cite supplied sources and include matching quotations. If generation fails, the original records remain accessible. These checks verify where the text came from; they cannot prove that the model interpreted it correctly.
Compare learned representations with the baseline.
Normalized duplicate-text groups stay in the same train, development, or test partition, and training deduplicates consistent labelled groups. Development selects checkpoints and topic thresholds. The following fixed-model test comparisons use the original ten-topic target definition.
| Model | Sentiment agreement | Sentiment macro F1 | Topic macro F1 |
|---|---|---|---|
| Original TF-IDF baseline | 83.01% | 0.7437 | 0.7406 |
| Frozen encoder + trained heads | 77.43% | 0.6785 | 0.5726 |
| LoRA reference, epoch 20 | 86.42% | 0.7893 | 0.7766 |
| Selected main: softened class weights | 88.18% | 0.7935 | 0.7798 |
The main model improves sentiment agreement by 5.17 percentage points over the original TF-IDF baseline. In a matched five-epoch development experiment, square-root class weighting raised agreement from 85.46% to 87.72%. Neutral precision improved from 0.4550 to 0.5342, while neutral recall decreased from 0.6303 to 0.5140.
A separate 24-topic candidate broadened the task but did not replace the main model: sentiment test agreement was 87.99% and macro F1 was 0.7854. Topic F1 across different taxonomies cannot be compared as if the reference task were unchanged. Probability evaluation adds log loss, Brier score, calibration diagnostics, and close second-choice analysis alongside strict metrics.
Three ways to ask, with different outputs.
The application and intended inputs are Turkish. These example requests illustrate the task boundaries; they are not claims about particular products or generated test outputs.
| View | Example request | Expected workflow |
|---|---|---|
| Product recommendations | Yağlı ve sivilceye eğilimli cildim için temizleyici öner. | Find cleanser listings and assess reviews for the stated needs; show supported products and cautions |
| Evidence search | Beyaz iz bırakmadığını söyleyen güneş kremi yorumlarını göster. | Retrieve relevant experiences and show a cited summary beside original quotations |
| Chat | Müşteriler tahriş konusunda ne söylüyor? Peki hassas ciltle ilgili? | Retrieve evidence for each turn while retaining the follow-up context |
| Text classification | Güzel nemlendiriyor ama makyaj altında topaklanıyor. | Return review sentiment probabilities and overlapping topic predictions |
A fifth view browses records by topic and product filters. The main interface uses the selected models, while comparisons and experimental retrieval profiles remain in command-line tools. GPU inference is supported, including verified Qwen CUDA offload in local diagnostics.
Make the evidence visible, including its limits.
RAG experiments tested a frozen reranker, quote-first selection, query-aware assessment, and event-polarity rules. A larger suite of 60 new queries found both improvements and regressions from the event guard. No independent human RAG or recommendation accuracy score is established.
Negation, mixed experiences, unfamiliar wording, and requests with several requirements remain difficult. Diagnostic review found an unfavorable hydration experience being treated as favorable and partial evidence being treated as support for a combined request. Exact quotations can be correct while the suitability judgment is wrong. Retrieved review counts are not representative product outcome rates.
The repository includes source code, examples, setup instructions, and the selected classifier's LoRA adapter, prediction heads, and tokenizer. The public base model downloads on first use, so standalone classification runs without retraining or corpus files. Full RAG and recommendations require a locally supplied compatible corpus; private records, metadata, and experiment artifacts are excluded.