The early detection of oral malignancy by physicians is a strenuous task. The analysis of histopathological oral malignancy images using image processing and deep learning techniques can be an add-on facility for doctors to diagnose oral cancer. In this work, a deep learning model is used, designing a modified Xception net with swish activation function and generalised mean pool for the detection of oral malignancy. To prove the superiority of the model, three stages of comparative analysis are carried out. In the first stage, the model is compared with a few advanced models explicitly Alexnet, Resnet50, Resnet101, VGG16, VGG19, Inception net and original Xception. In the second stage, loss and accuracy graphs analysis is done and in the third stage, the proposed model's accuracy is compared with other model's accuracy available in the literature. It is found that the modified Xception net got upgraded performance by an accuracy of 98.97%.