A Novel Framework for 3D-2D Vertebra Matching

Hanchao Yu; Yang Fu; Haichao Yu; Yunchao Wei; Xinchao Wang; Jianbo Jiao; Matthew Bramlet; Thenkurussi Kesavadas; Honghui Shi; Zhangyang Wang; Bihan Wen; Thomas Huang

doi:10.1109/MIPR.2019.00029

A Novel Framework for 3D-2D Vertebra Matching

Hanchao Yu, Yang Fu, Haichao Yu, Yunchao Wei, Xinchao Wang, Jianbo Jiao, Matthew Bramlet, Thenkurussi Kesavadas, Honghui Shi, Zhangyang Wang, Bihan Wen, Thomas Huang

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Citations (Scopus)

Abstract

3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

Original language	English
Title of host publication	Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	121-126
Number of pages	6
ISBN (Electronic)	9781728111988
DOIs	https://doi.org/10.1109/MIPR.2019.00029
Publication status	Published - 22 Apr 2019
Event	2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 - San Jose, United States Duration: 28 Mar 2019 → 30 Mar 2019

Publication series

Name	Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

Conference

Conference	2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
Country/Territory	United States
City	San Jose
Period	28/03/19 → 30/03/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

3D 2D registration
Hough transform
object detection

ASJC Scopus subject areas

Computer Networks and Communications
Signal Processing
Media Technology

Access to Document

10.1109/MIPR.2019.00029

Cite this

Yu, H., Fu, Y., Yu, H., Wei, Y., Wang, X., Jiao, J., Bramlet, M., Kesavadas, T., Shi, H., Wang, Z., Wen, B., & Huang, T. (2019). A Novel Framework for 3D-2D Vertebra Matching. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 (pp. 121-126). Article 8695312 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/MIPR.2019.00029

Yu, Hanchao ; Fu, Yang ; Yu, Haichao et al. / A Novel Framework for 3D-2D Vertebra Matching. Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers (IEEE), 2019. pp. 121-126 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).

@inproceedings{f76f63f178fc4630881d58183c864027,

title = "A Novel Framework for 3D-2D Vertebra Matching",

abstract = "3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.",

keywords = "3D 2D registration, Hough transform, object detection",

author = "Hanchao Yu and Yang Fu and Haichao Yu and Yunchao Wei and Xinchao Wang and Jianbo Jiao and Matthew Bramlet and Thenkurussi Kesavadas and Honghui Shi and Zhangyang Wang and Bihan Wen and Thomas Huang",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 ; Conference date: 28-03-2019 Through 30-03-2019",

year = "2019",

month = apr,

day = "22",

doi = "10.1109/MIPR.2019.00029",

language = "English",

series = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "121--126",

booktitle = "Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019",

}

Yu, H, Fu, Y, Yu, H, Wei, Y, Wang, X, Jiao, J, Bramlet, M, Kesavadas, T, Shi, H, Wang, Z, Wen, B & Huang, T 2019, A Novel Framework for 3D-2D Vertebra Matching. in Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019., 8695312, Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, Institute of Electrical and Electronics Engineers (IEEE), pp. 121-126, 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019, San Jose, United States, 28/03/19. https://doi.org/10.1109/MIPR.2019.00029

A Novel Framework for 3D-2D Vertebra Matching. / Yu, Hanchao; Fu, Yang; Yu, Haichao et al.
Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers (IEEE), 2019. p. 121-126 8695312 (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Novel Framework for 3D-2D Vertebra Matching

AU - Yu, Hanchao

AU - Fu, Yang

AU - Yu, Haichao

AU - Wei, Yunchao

AU - Wang, Xinchao

AU - Jiao, Jianbo

AU - Bramlet, Matthew

AU - Kesavadas, Thenkurussi

AU - Shi, Honghui

AU - Wang, Zhangyang

AU - Wen, Bihan

AU - Huang, Thomas

PY - 2019/4/22

Y1 - 2019/4/22

N2 - 3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

AB - 3D-2D medical image matching is a crucial task in image-guided surgery, image-guided radiation therapy and minimally invasive surgery. The task relies on identifying the correspondence between a 2D reference image and the 2D projection of 3D target image. In this paper, we propose a novel image matching framework between 3D CT projection and 2D X-ray image, tailored for vertebra images. The main idea is to learn a vertebra detector by means of deep neural network. The detected vertebra is represented by a bounding box in the 3D CT projection. Next, the bounding box annotated by the doctor on the X-ray image is matched to the corresponding box in the 3D projection. We evaluate our proposed method on our own-collected 3D-2D registration dataset. The experimental results show that our framework outperforms the state-of-the-art neural network-based keypoint matching methods.

KW - 3D 2D registration

KW - Hough transform

KW - object detection

UR - http://www.scopus.com/inward/record.url?scp=85065605641&partnerID=8YFLogxK

U2 - 10.1109/MIPR.2019.00029

DO - 10.1109/MIPR.2019.00029

M3 - Conference contribution

AN - SCOPUS:85065605641

T3 - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

SP - 121

EP - 126

BT - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

Y2 - 28 March 2019 through 30 March 2019

ER -

Yu H, Fu Y, Yu H, Wei Y, Wang X, Jiao J et al. A Novel Framework for 3D-2D Vertebra Matching. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019. Institute of Electrical and Electronics Engineers (IEEE). 2019. p. 121-126. 8695312. (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). doi: 10.1109/MIPR.2019.00029

A Novel Framework for 3D-2D Vertebra Matching

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this