https://jjcit.org/paper/175 10.5455/jjcit.71-1655376900 Prashantha S. J.,H. N. Prakash Deep-learning features,Handcrafted features,Canonical-correlation analysis,Discriminant-correlation analysis,Support vector machines 3 5175 1447 16-Jun.-2022 18-Aug.-2022 12-Sep.-2022 In this paper, we propose an efficient fusion framework for brain magnetic resonance (MR) image classification using deep learning and handcrafted feature extraction methods; namely, histogram of oriented gradients (HOG) and local binary patterns (LBPs). The proposed framework aims to: (1) determine the optimal handcrafted features by Genetic Algorithm (GA) (2) discover the fully connected (FC) layers’ features using fine-tuned convolutional neural network (CNN) (3) employ the canonical correlation analysis (CCA) and the discriminant correlation analysis (DCA) methods in feature-level fusion. Extensive experiments were conducted and the classification performance was demonstrated on three benchmark datasets; viz., RD-DB1, TCIA-IXI- DB2 and TWB-HM-DB3. Mean accuracy of 68.69%, 90.35% and 93.15% from CCA and 77.22%, 100.00% and 99.40% from DCA was achieved by the Support Vector Machines (SVM) sigmoid kernel classifier on RD-DB1, TCIA-IXI-DB2 and TWB-HM-DB3, respectively. The obtained results of the proposed framework outperform when compared with other state-of-the-art works.