Explore

Filter creative automation

HVF FILTER IMAGE GENERATION

Currently we are relying on third party Creative team to generate images for HVFs. Out of the ~400 HVFs we currently have we only have images for about 200. It's a very slow process of generating images as right now it is a manual process to generate these images.

After the Filter Label & Value Rationalisation effort the HVF Filter count is going to shoot to Thousands. At that point this effort of manual image generation will be futile. Thus, we need to come up with a scalable cost effective way that can quickly generate images for these filters at scale

SOLUTION

Use open source stable diffusion models + our own catalog images to fulfil HVF Filter Images.

Current Approach:

Use Hyper-SD Stable Diffusion .(

https://huggingface.co/ByteDance/Hyper-SD⁠

) model to generate images. These models require a positive and a negative prompt. To ensure high quality image results these prompts needs to be created in accordance to the given Filter Key and Value.

⁠

Example HVF Image for Mens Kurta Set

⁠

This image needs to be post-processed according to the Design Criteria

Add border to Image

Round the Edges

Overlay the text in a particular font and at a particular location

Add gradient to the image

⁠

after adding gradient

⁠

SEARCH NER DS Model

Problem Statement: Identify Attribute Keys and Values being talked about in User Search Queries

This problem is analogous to the famous NLP problem of NER (Named Entity Recognition) where we need to build a Language model capable of tagging words in a search query to a particular Attribute Key.

For Ex: query→ black sarees for women

Attributes Tagged → {Color: black, Category: sarees, Gender: Women}

Areas Where this Model can be Used:

Identifying important attributes mentioned in User Search Query:

For each query, get attributes tagged, map query to SSCat, aggegrate attributes to SSCat to understand important attributes for each SSCat

Attribute Exclusion in Filter Scenario:

For a particular Search/Collection/CLP, if the name already consists of a particular attribute, then don’t show it as a filter Key.

For eg user searches for Red Sarees. Don’t show filter for Colors

Methods of Tackling Problem:

Discriminative NER

Explore the entire space of possible Attribute Keys and identify them as entities (Color, Size, Category, Gender etc)

For each Attribute Key, there can be n number of Attribute Values (which is not a exhaustive list)

Classify each word of the phrase as an entity (No Attribute, Color, Size, Category, etc)

Pros:

Discriminative models are very fast and can run quickly on a huge set of data

Cons:

Given the non-exhaustive nature of Attribute Labels and Label Values, the training data preparation step itself is very difficult

NER task where the number of entities go to 100s of entities yield low performance (especially in tail entitites

A large number of training samples are required for each entity to train this model

Difficult to get good results for non English tail queries.

Generative NER

It is an unsupervised way of solving the task of NER where we specifically don’t give the model an exhaustive set of entities to recognise from.

As a language model, it inherently learns the space of queries and possible Entities.

We prompt the model to generate both Attribute Keys and Values present in the query.

Pros:

the model has capability to identify Entities (labels) and corresponding values (label values) that it did not see during training, as the model becomes expert of the language

Not a lot of training samples required for training

It works for non-english tail queries also

Cons:

Generative models are slower compared to Discriminative model

Model Formulation

⁠

⁠

Model Training

Mistral Instruct 7B v0.3 used for training model

Model fine-tuned using PEFT and LoRA

41M parameters fine tuned out of 7B params

Used FastLanguageModel library for faster inference

Trained on ~110k head queries (ds_silver.search_top_queries)

Inference Time on a single query ~100 ms

Model Evaluation

Precision and Recall are the main metrics used to evaluate the performance of the model.

However, in the task of unsupervised Generative NER, the task of evaluation is not straightforward as explained by the following example.

Query: Black cotton night suit for women's

Ground Truth Entities: {Color:Black, Fabric:cotton, Category:Night Suit, Gender:Women}

Predicted Entites: {Colour:Black, Material:cotton, Category:Night Suit, Gender:Women’s}

Since the model is a generative one, it can generate different entity name which actually would mean the same as the ground truth entity. Even the word tagged as the corresponding entity can be a bit different.

Generic Precision Calculation for Key: Pred{Category, Gender}/GT{Color, Fabric, Category, Gender} => 0.50

Generic Precision Calculation for Value: Pred{Category-Night Suit}/GT{Color:Black, Fabric:cotton, Category:Night Suit, Gender:Women} =>0.25

This generic calculation is wrong since the model predicted all keys and values correctly. We need to match the Key and Values correctly

Ideal Calculation Precision for Key => 1

Ideal Calculation Precision for Value => 1

⁠

⁠

Results

Head Queries

Head Queries

Metric

Key

Value

Notes

Generic Precision

87.48

76.73

Open

New Precision

89.53

80.25

Open

Generic Recall

87.93

77.61

Open

New Recall

87.95

81.1

Open

There are no rows in this table

⁠

Torso Queries

Torso Queries

Metric

Key

Value

Notes

Generic Precision

80.92

71.44

Open

New Precision

83.33

75.29

Open

Generic Recall

82.68

72.05

Open

New Recall

83.29

75.62

Open

There are no rows in this table

⁠

Model Output Instances

Model Output Instances on Head queries

oneplus n 20 oneplus → {'category': 'phone', 'brand': 'oneplus', 'model': 'n 20'}

south indian golden bracelet women → {'category': 'bracelet', 'gender': 'women', 'color': 'golden', 'style': 'south indian'}

high neck maroon blouse → {'category': 'blouse', 'color': 'maroon', 'style': 'high neck'}

jewellery earring jhumka gold → {'category': 'jewellery', 'type': 'earring', 'material': 'gold', 'style': 'jhumka'}

Model Output Instances on Torso queries

motorcycle sticker blue→ {'category': 'sticker', 'color': 'blue', 'type': 'motorcycle'}

plain shirt blue color →{'category': 'shirt', 'color': 'blue', 'pattern': 'plain'}

polyester backpack for women → {'category': 'backpack', 'material': 'polyester', 'gender': 'women'}

1 meter velvet blouse piece → {'category': 'blouse piece', 'material': 'velvet', 'size': '1 meter'}

kid boy sandal → {'category': 'sandal', 'gender': 'boy', 'age group': 'kid'}

Model Output Instances on Non-English queries

kala color saree → {'category': 'saree', 'color': 'kala'}

lal kapde → {'Category': 'kapde', 'Color': 'lal'}

boy ke denim shirt → {'Category': 'shirt', 'Material': 'denim', 'Gender': 'boy'}

Want to print your doc?
This is not the way.

Try clicking the ··· in the right corner or using a keyboard shortcut (

CtrlP

) instead.