HyCaTNet: A Hierarchical CNN-Transformer Network for Segmentation of Carotid Artery in Ultrasound Images

Carotid artery ultrasound image is a vital tool for the non-invasive assessment of vascular systems and is commonly used in early detection, risk stratification and monitoring of atherosclerotic cardiovascular disease. Stroke is a major cause of death and permanent disability in the world today and carotid artery atherosclerosis is the cause of a large proportion of ischemic strokes. Carotid artery ultrasound allows visualization of the morphology of the arterial wall, the lumen and the properties of atherosclerotic plaques in real-time, without causing any exposure to ionizing radiation in patients. Current deep learning methods are either based on convolutional models without long-range spatial reasoning or on pure Transformer models, which have poor performance when they are trained with limited number of ultrasound-specific datasets. To address this issue, we propose a novel dual-branch encoder named HyCaTNet (Hierarchical CNN-Transformer Network), which incorporates an EfficientNet-B4 CNN branch to exploit local texture difference and a Swim-S Transformer branch to reason the global context. The lumen–intima boundary (LIB), the media–adventitia boundary (MAB), the intima–media thickness (IMT) and the lumen diameter and plaque regions are important for quantitative analysis of carotid ultrasound images. Such are clinically relevant biomarkers that can measure subclinical atherosclerosis, predicting cardiovascular events, and measuring therapeutic response [1-4]. A few deep learning architectures based on the U-net model have been used in the segmentation of the carotid artery and proved by different metrics like Dice Score, IOU etc.