Emotion is a complex psychophysiological response to external stimuli, essential for human survival, social interaction, and human-computer interaction. Emotion recognition play a critical role in both biological systems and artificial agents. However, existing research often treats these systems independently, limiting opportunities for interaction and hindering the development more advanced models. This study employs representational similarity analysis (RSA) to bridge this gap by comparing emotional representations between the human brain and neural networks, aiming to improve understanding of emotion recognition in deep learning models. By correlating the emotion recognition model EmoNet with EEG signals from the human brain during emotional image processing, we reveal EmoNet's hierarchical structure for emotion recognition. Introducing AlexNet, which is trained for object recognition, further highlights EmoNet's specificity in processing emotional images, as it shows human-like representation for emotional but not neutral images. In contrast, AlexNet exhibits similar responses to all image categories, regardless of emotional content. The results demonstrated that RSA is a powerful tool for aligning human emotional processing with deep neural networks, offering new avenues for improving the interpretability and performance of emotional AI models. Moreover, they underscore EmoNet’s potential to simulate human emotional responses, paving the way for future research to enhance emotion recognition models by incorporating human emotional evaluations into their training processes, thereby improving efficiency and specificity.