Dogs play an important emotionl role in human life, so understanding their feelings can significantly enhance the bond between humans and dogs. In this project, we developed an image classification model that identifies a dog’s emotion—particularly happiness—from its facial expressions. The idea is based on observation that dogs display a wide variety of facial expressions compared to other animals.
We collected images using unsplash API by searching keywords like happy dog, angry dog, playing dog, etc.
While collected images included some irrelevant images, data cleaning procedure is needed. Therefore, we refined dataset through the following steps:
Representative image selection: For both happy and angry labels, we manually selected several images to use as representative feature vector.
Feature extraction: Using a pretrained ResNet18 model, I extracted features from collected images, happy and angry respectively. Then calculated cosine similarity between each image and representative feature vectors. Finally I used those images with the highest similarity scores for training dataset.
Final training set:
Validation Accuracy: 90%
Test Accuracy: 60%
The performance gap was mainly due to breed imbalance in training set. The model underperformed on Pomeranians, Malteses and other breeds that were not well represented in training dataset.
Also, many test images displayed neutral or ambiguous expressions that didn’t clearly fall into either happy or angry
most images display neutral or ambiguous facial expressions
To address these issues: