Corresponding Author: Michael S. Kook, MD. Department of Ophthalmology, Asan Medical Center, University of Ulsan College of Medicine, #388-1 Pungnap 2-dong, Songpa-gu, Seoul 138-736, Korea. Optical coherence tomography (OCT) has become an important diagnostic method for glaucoma and posterior segment disease [1-8]. It allows in vivo cross sectional imaging of eyes with a non-contact, non-invasive technique. Since glaucoma is a slowly progressing disease, assessment and monitoring of changes in OCT retinal nerve fiber layer (RNFL) parameters might play an important role in progression detection of glaucomatous damage.
The most crucial aspect of a useful device for glaucoma progression detection is measurement reproducibility, which involves discriminating true pathologic change from test-retest variability. Although numerous studies have reported good reproducibility and glaucoma diagnostic ability of OCT in cross-sectional studies [9-12], the capability of OCT for glaucoma progression detection has not yet been fully elucidated. Furthermore, we have the impression in everyday clinical practice that measurement reproducibility of OCT may not be as good as that of a well-designed study obtained by good quality imaging.
A possible cause for seemingly worse reproducibility in real clinical settings over prospective, case-control studies may be that the scan circle is difficult to locate at the same position of the previous measurement since it is manually placed at each test. Another putative cause is the signal strength of the OCT images since OCT RNFL thickness is heavily influenced by signal strength, and therefore image quality. Both scan circle placement and signal strength have been reported to substantially affect measurement variability in previous publications [13,14].
Recently, a newer version of OCT incorporating spectral domain (SD) technology has been introduced into clinical practice. SD-scanning laser ophthalmoscopic (SLO)/OCT is a commercially available SD-OCT. This technology provides a SLO fundus image and OCT image simultaneously. The SLO imaging guides follow-up OCT scanning being performed at the same location of base-line images with a 're-test mode' which is incorporated into the machine. This SLO guided registration of OCT data over different scans might theoretically provide more accurate retinal structural information and therefore allow for better measurement reproducibility. As described earlier, the longitudinal measurement reproducibility of an imaging device is very important for the purpose of detecting glaucomatous change over time. Thus, this study evaluates both the short- and long-term measurement variability of peripapillary RNFL thickness in healthy eyes with adjustment of signal strength by statistical modeling and the scan circle placement effect using the SLO guided 're-test mode' by SD-SLO/OCT.
Materials and Methods
Subjects
One randomly chosen eye of 75 healthy volunteers was entered for the final analysis in our current study. Subjects were staff, their family, spouses of patients, or volunteers from the eye clinic between September 2008 and September 2009 at the Asan Medical Center, Seoul, Korea. At the initial evaluation, all subjects underwent a complete ophthalmologic examination including medical, ocular, and family history; visual acuity (VA) testing; the Humphrey field analyzer (HFA) Swedish Interactive Threshold Algorithm (SITA) 24-2 test (Carl Zeiss Meditec, Dublin, CA, USA); intraocular pressure (IOP) measurements using Goldmann applanation tonometry (GAT); stereoscopic optic nerve photography; and SD-SLO/OCT. For inclusion in the study, all participants had to meet the following criteria: best-corrected VA of 20 / 30 or better, with a spherical equivalent within ±5 diopters (D) and a cylinder correction within +3 D; IOP less than 22 mmHg, with no history of IOP elevation; presence of a normal anterior chamber and open-angle on slit-lamp and gonioscopic examinations; normal-appearing optic disc; normal VF with reliable HFA test results with a false-positive error <15%, a false-negative error <15%, and a fixation loss <20%. Subjects with any other ophthalmic disease that could result in HFA defects, or with a history of diabetes mellitus, were excluded. One eye was randomly selected if both eyes were found to be eligible for study.
All participants gave written informed consent before enrollment. All procedures conformed to the Declaration of Helsinki and the study was approved by the Institutional Review Board of the Asan Medical Center, Seoul, Korea.
Spectral domain scanning laser ophthalmoscopic optical coherence tomography
SD-SLO/OCT (OPKO, Ophthalmic Technologies Inc., Toronto, Canada) is equipped with single light source and parallel dual beam channel for OCT and SLO imaging. The axial resolution in tissue is 5 to 6 µm and scan speed is 27,000 A-scans/sec. Three continuous RNFL thickness measurements along a circle 3.45 mm in diameter centered at the optic disc are obtained and averaged to produce single RNFL thickness by the scan speed of 32 frame/sec. Once the single measurement is performed, the re-test mode allows for follow-up measurements at the same location by SLO guided registration of data.
Based on the optic disc image from the first capture, the software automatically aligns a new SLO image using the blood vessels as a marker. The operator has the opportunity to correct the placement. Once the placement is accepted, the software uses the new SLO image in the aligned area to track, again using the blood vessels as a guide.
The data were obtained in two ways at every visit, with the re-test mode and separate scanning without using the re-test mode (independent mode). We excluded all poor-quality scans, defined as those with a signal strength less than 8, overt misalignment of the surface detection algorithm on at least 15% of consecutive A scans or 20% of cumulative A scans, or overt displacement of the measurement circle, as assessed subjectively. Ten of the initially enrolled 85 subjects had poor image quality or eye movements during the image acquisition and were excluded from the analysis. Thus the data of 75 subjects were included in the final analysis. The average and quadrant peripapillary RNFL thickness measurements were reported in the analysis printout. Pharmacologic dilation was performed if the pupil was small.
Image acquisition
All subjects were scanned 3 times by each mode (re-test mode and independent mode) with SD-SLO/OCT with 5-minute breaks between each measurement at first visit (intra-session variability). The order of the scanning modes was randomly determined. Measurement involved having the subject seated with the chin in a chin rest and the machine properly aligned. The subject was then instructed to fixate the eye being measured on a target, to bring the optic nerve head within view of the examiner. A single well-trained operator acquired all measurements for the purpose of testing reproducibility of the scanning modes of the instrument, rather than that of the operators. The subject was asked to sit back after each session before acquiring the subsequent scans. The subjects were scanned by both modes at two additional visits within a 2 week interval (inter-session reproducibility) by the same protocol as the intra-session reproducibility measurements. For testing longitudinal variability, three separate RNFL thickness measurements were performed over 6 months in these subjects using the same protocol with the same device. When assessed by re-test mode, the first qualified measurement of the first visit was used for registration of subsequent data acquisition for intra- and inter-session and longitudinal variability measurements. When assessed by independent mode, each scan circle was placed independently. Calibration of the device was performed monthly by the manufacturer.
Analysis
The number of subjects (n) and measurements (k) were selected to assure that the lower confidence interval (CI) for an intraclass correlation coefficient (ICC) of 0.8 would not be lower than 0.75, which is the generally accepted lower cutoff for good reproducibility [15]. With three measurements by each mode per subject, 75 subjects are needed to yield an ICC of 0.8 with a lower CI of 0.74. Therefore, our study sample size involving three measurements per subject by both modes and a minimum of 75 subjects was thought to be a reasonable estimate of sample size calculation.
Average and 4 quadrants RNFL thickness were used for the analysis. Coefficient of variance (CV, square root of the residual mean square value divided by the mean thickness), reproducibility coefficient (RC, 2.77× within-subject standard deviation) [16,17] and ICC, determined by both re-test and independent modes, were computed for average and 4 quadrants RNFL thickness after accounting for age, gender, and SD-SLO/OCT scan quality (signal strength) as covariates using a mixed-effects model. The quartile method using 1,000 times bootstrapping was implemented for estimating the 95% CI. For the purpose of comparing variability between the re-test and independent modes, within-subject variance of RNFL parameters in intra-, inter-session and long-term measurements were compared by paired t-test on log transformed square differences. A p-value of <0.05 was considered to be statistically significant. All statistical analyses were performed using SAS ver. 9.1 (SAS Inc., Cary, NC, USA) and SPSS ver. 15.0 (SPSS Inc., Chicago, IL, USA).
Results
As described earlier, 10 subjects from the initially enrolled 85 subjects were excluded due to poor quality of SD-SLO/OCT image. Of the 75 subjects who entered the subsequent analysis, 40 were women and 35 were men. Mean age (±standard deviation) was 43.7 120 (±15.1). Table 1 shows the demographics of the study participants.
SD-SLO/OCT demonstrated good intra-session reproducibility in average and 4 quadrants in both re-test and independent modes (Table 2). The CV ranged from 1.76 to 5.72% in independent mode and 1.75 to 5.58% in re-test mode. The ICC ranged from 0.819 to 0.966 in independent mode and 0.877 to 0.973 in re-test mode. Variability was not significantly different between re-test and independent mode in each quadrant and average RNFL thickness measurement (p-value: temporal, 0.058; superior, 0.069; nasal, 0.066; inferior, 0.089; average, 1.00).
In the inter-session variability assessment, the CV ranged from 1.89 to 5.78% in independent mode and 1.90 to 5.54% in re-test mode. The ICC ranged from 0.804 to 0.963 in independent mode and 0.828 to 0.961 in re-test mode (Table 3). Variability was not significantly different between re-test and independent mode in each quadrant as well as average RNFL thickness measurement (p-value: temporal, 0.174; superior, 0.083; nasal, 0.164; inferior, 0.190; average, 0.892).
In the longitudinal assessment, variability was significantly lower when measured by re-test mode than by independent mode in all parameters except for the nasal quadrant (p-value: temporal, 0.004; superior, <0.001; nasal, 0.052; inferior, 0.001; average, 0.011). The CV ranged from 2.79 to 7.08 % in independent mode and 2.33% to 6.22% in re-test mode. ICC ranged from 0.72 to 0.825 in independent mode and 0.794 to 0.881 in re-test mode (Table 4). Longitudinal variability was generally higher in each quadrant and average RNFL thickness than intra- or inter-session variability when assessed by both re-test and independent modes.
Discussion
RNFL change can be an important indicator for glaucoma progression detection. Among several devices which can measure RNFL thickness, OCT has become one of the most commonly used modalities, and SD-OCT is the latest version of currently available OCT. SD-OCT has a faster scan speed and higher resolution than conventional TD-OCT, which may provide more accurate data registration and subsequent improved measurement reliability. However, it has not been completely proven whether these presumed merits would be of practical help in increasing reproducibility. SD-SLO/OCT is equipped with a re-test mode for peripapillary RNFL measurement, which allows follow-up imaging to be performed at the same location by retinal vascular pattern identification by a SLO guided tacking system. As previous studies have demonstrated that subsequent measurement reproducibility is affected by scan circle placement [13,14], our study was intended to validate whether a re-test mode for RNFL measurements would improve longitudinal variability compared with independent scan circle placement, which has previously been used in TD-OCT. Additionally, since OCT images and subsequent RNFL thickness are known to be heavily influenced by signal strength [14,18-20], we attempted to adjust this effect by adding signal strength as one of the covariates in the mixed-effects model.
According to our results, SD-SLO/OCT showed good reproducibility of RNFL thickness measurements either by re-test or independent mode in healthy individuals. After we adjusted those covariates to include signal strength, our results indicated that variability was not significantly different between re-test and independent mode in short-term measurements (intra- and inter-session variability). However, longitudinal variability was significantly improved by re-test mode in all parameters except for nasal quadrant compared with independent mode. Therefore, we may speculate that although the effect of same scan circle placement was not evident in short-term measurements, long-term variability would be significantly improved by the SLO guided re-test mode.
Although we adjusted the signal strength and scan circle placement which are regarded as the main sources of measurement variability [13,14,18-21], longitudinal variability was still higher than intra- or inter-session variability. We believe this is an important observation. Other unknown factors may affect long-term measurement variability. One possible factor could be the difference in reflectance induced by different measurement angles obtained at later times. Software related issues, such as the consistency of image processing or the segmentation algorithm, may also affect longitudinal measurement reproducibility. There is also a possibility that the signal strength effect was not fully compensated for by our statistical modeling. Although scan speed was tremendously improved, there is a possibility that circular measurement was not performed by fine saccadic eye motion at the presumed location. RNFL thickness may also have had real changes over 6 months; however, we believe that this possibility was very low, since all participants were healthy subjects and we did not observe any structural changes by optic disc exam during the 6 month follow-up period. Another possibility involves the issue of calibration, which was performed on a monthly basis; however, this time schedule may not be optimal.
We compared the CV of our measurements with the results of other studies. CV is the ratio of mean and standard deviation and thus may be less affected by measurement scale. Leung et al. [22] reported a CV of 1.8 to 4.7% when they evaluated inter-visit RNFL measurement reproducibility by Cirrus OCT. This variation is comparable to our results of 1.89 to 5.78%. Vizzeri et al. [23] reported a CV of 1.2% to 4.4% when they evaluated RNFL measurement reproducibility by Cirrus OCT. Gonzalez-Garcia et al. [24] reported a CV of 1.5% to 4.6% by RTVue OCT and Menke et al. [25] a CV of 2.0% to 6.7% by Topcon SD-OCT. Unlike conventional TD-Stratus OCT, several brands of SD-OCT devices from different companies are currently available. Thus, different study outcomes from different SD-OCTs may not be comparable. For instance, if the thickness itself is systemically thicker in certain device, the reproducibility coefficient could be higher, which may be interpreted as worse reproducibility. Therefore, caution needs to be taken when comparing the results of one study to another.
Our results showed that nasal quadrants showed the highest variability in intra-, inter-session, and longitudinal examination and this finding was in agreement with the results of others performed by SD-OCT as well as by TD-OCT [11,12,22-24]. This high measurement variability of the nasal quadrant may be explained by the data acquisition mode. Large vessels in the optic nerve head dominate on the nasal side. Thus, data from this side are generally of lower precision and suffer from poor reproducibility.
The glaucoma diagnostic capability of RNFL measurement by OCT has been proven to be effective in numerous studies [1,2,4-8,26]. However, the capability of glaucomatous progression detection by OCT with RNFL thickness measurements has yet to be established. Rapid iterations of devices do not provide the chance for individuals to be traced longitudinally. However, we suggest that longitudinally-confirmed measurement stability is crucial for OCT to be a useful device for glaucoma diagnosis and progression detection. In our study, we confirmed that same scan circle placement by re-test mode would enhance long-term measurement reproducibility compared with independent scan circle placement. However, longitudinal variability was still higher than for short-term measurements. All diagnostic devices have inherent measurement variability. Therefore, we suggest that change beyond test-retest variability could be regarded as a reasonable cut-off value for defining 'true change.' This strategy for defining 'true change' as the 'change beyond test-retest variability' was implemented in numerous publications [3,27,28]. Measurement variability is usually used as a cut-off value for defining change in event analysis, which is frequently used in glaucoma progression detection [28,29]. If we assume that longitudinal variability is closer to real clinical settings since patients are usually followed up at intervals of several months, it would be better to use longitudinal variability rather than inter- or intra- session measurement variability in defining 'true change'. In this regard, our observation that longitudinal variability is greater than inter- or intra- session measurement variability has important clinical relevance in the detection of glaucoma progression.
There are a few limitations in our study. First, a single operator scanned all exams for the purpose of testing reproducibility of the instrument, rather than of operators, and thus inter-operator variability was not considered in our study and this issue needs to be evaluated in the future. Second, since this a prospectively designed study, the reproducibility error may not be the same in real clinical settings. In real life, moderate to severe cataracts, corneal opacities, or vitreous opacities can prevent or introduce error in image acquisition. Patient cooperation during the examination, especially among elderly individuals, may not be optimal. Despite the encouraging data on both short- and long-term variability in healthy individuals, our data may not reflect what would be expected in a clinical setting. As the reproducibility data on the glaucoma subjects are more clinically relevant, the results based on glaucomatous subjects may differ from those of healthy individuals using the re-test mode of the SD-SLO/OCT device. As the aim of our study was to test the variability of the scanning modes of the instrument, and not operator variability, a single well-trained operator (MHC) acquired both re-test and independent modes. However, without masking, this might have introduced a bias in recording measurements. Finally, with the aging process, there is a natural loss of RNFL thickness. This potential problem has not been taken into account in the calculation of longitudinal variability, although the interval for the repeat exam was relatively short in our current study. The introduction of a normative database based on this type of longitudinal variability study taking into account normal aging changes might increase diagnostic power for glaucoma progression detection.
In conclusion, SLO guided SD-OCT demonstrated excellent short- and long-term reproducibility. Long-term reproducibility was substantially improved by SLO guided data registration. The re-test mode of RNFL thickness measurement in spectral domain OCT employing a SLO tracking system may be helpful for reducing test-retest variability originated by inconsistent scan circle placement. Future studies are needed to assess long-term reproducibility of SLO guided SD-OCT in glaucomatous eyes.
PDF Links
PubReader
XML Download
Full text via DOI
Full text via PMC
Download Citation
Print
