Skip to content

Advertisement

  • Research
  • Open Access

Comparison among ACR1997, SLICC and the new EULAR/ACR classification criteria in childhood-onset systemic lupus erythematosus

  • 1Email authorView ORCID ID profile,
  • 1,
  • 1,
  • 2 and
  • 1
Contributed equally
Advances in Rheumatology201959:20

https://doi.org/10.1186/s42358-019-0062-z

  • Received: 7 November 2018
  • Accepted: 1 May 2019
  • Published:

Abstract

Background

To date there are no specific classification criteria for childhood-onset systemic lupus erythematosus (cSLE). This study aims to compare the performance among the American College of Rheumatology (ACR) 1997, the Systemic Lupus International Collaborating Clinics criteria (SLICC) and the new European League Against Rheumatism (EULAR)/ACR criteria, in a cSLE cohort.

Methods

We conducted a medical chart review study of cSLE cases and controls with defined rheumatic diseases, both ANA positive, to establish each ACR1997, SLICC and EULAR/ACR criterion fulfilled, at first visit and 1-year-follow-up.

Results

Study population included 122 cSLE cases and 89 controls. At first visit, SLICC criteria had higher sensitivity than ACR 1997 (89.3% versus 70.5%, p <  0.001), but similar specificity (80.9% versus 83.2%, p = 0.791), however performance was not statistically different at 1-year-follow-up. SLICC better scored in specificity compared to EULAR/ACR score ≥ 10 at first visit (80.9% versus 67.4%, p = 0.008) and at 1-year (76.4% versus 58.4%, p = 0.001), although sensitivities were similar. EULAR/ACR criteria score ≥ 10 exhibited higher sensitivity than ACR 1997 (87.7% versus 70.5%, p <  0.001) at first visit, but comparable at 1-year, whereas specificity was lower at first visit (67.4% versus 83.2%, p = 0.004) and 1-year (58.4% versus 76.4%, p = 0.002). A EULAR/ACR score ≥ 13 against a score ≥ 10, resulted in higher specificity, positive predictive value, and cut-off point accuracy. Compared to SLICC, a EULAR/ACR score ≥ 13 resulted in lower sensitivity at first visit (76.2% versus 89.3%, p <  0.001) and 1-year (91% versus 97.5%, p = 0.008), but similar specificities at both assessments. When compared to ACR 1997, a EULAR/ACR total score ≥ 13, resulted in no differences in sensitivity and specificity at both observation periods.

Conclusions

In this cSLE population, SLICC criteria better scored at first visit and 1-year-follow-up. The adoption of a EULAR/ACR total score ≥ 13 in this study, against the initially proposed ≥10 score, was most appropriate to classify cSLE. Further studies are necessary to address if SLICC criteria might allow fulfillment of cSLE classification earlier in disease course and may be more inclusive of cSLE subjects for clinical studies.

Keywords

  • Systemic lupus erythematosus
  • Childhood
  • Adolescence
  • Classification criteria

Background

Systemic lupus erythematosus (SLE) is a chronic autoimmune disease, with a broad spectrum of clinical patterns. SLE affects women predominantly at reproductive age but may present at any age [1]. Childhood-onset SLE (cSLE) represents approximately 20% of SLE cases [2] and displays a higher frequency of atypical manifestations, more severe presentation and course, higher rates of disease activity and cumulative damage, than that reported for adult-onset disease [3, 4].

As the purpose of classification criteria is to identify a well-defined patient population suitable for research, specificity thus generally outweighs sensitivity in determining classification performance.

The SLE classification criteria set most commonly used is the one established by the American College of Rheumatology (ACR) [5]. Besides the development in adult-onset SLE and scarce validation in cSLE, concerns arose about the redundancy of photosensitivity with skin rashes, non inclusion of several clinically relevant integument and nervous system lupus manifestations, as well as hypocomplementemia, inadequate quantification of urine protein by dipstick, and, classification as SLE for patients without positive autoantibodies [6].

Alternative methods for SLE classification have been developed predominantly in adults, such as the SLICC (Systemic Lupus International Collaborating Clinics) criteria [7] and the new EULAR/ACR criteria [812].

The main changes proposed in SLICC criteria were the expansion of cutaneous and neurological criteria, allocation of cytopenias and autoantibodies in individual criterion, inclusion of alopecia and hypocomplementemia, and the classification for patients with only documented lupus nephritis with ANA or anti-dsDNA [7]. SLICC criteria yielded higher sensitivity (97% versus 83%) but lower specificity (84% versus 96%) than ACR 1997 criteria in the original validation set [7]. In subsequent studies in adult and cSLE, SLICC higher sensitivity was also found, especially for early SLE, however, results were conflicting regarding specificity [1320]. The SLICC criteria have also been criticized because they were derived comparing the expert’s decision (“gold standard”) with a standardized group of manifestations [6]. Moreover, as SLICC criteria emphasize immunological and hematological events, it might be possible that subjects classified through SLICC criteria may exhibit less clinically significant multisystem involvement compared with subjects classified through ACR criteria [6].

In 2017, the EULAR and ACR joined in a four-phase project to develop more sensitive (especially for initial classification) and more specific SLE classification criteria [812]. The first phase of this project was designed to gather potential candidate items broadly, through an SLE expert Delphi exercise and an international early SLE cohort study. The second phase consisted of item reduction by nominal group technique. The third phase was for item definition (literature based) and weighting (multiparameter decision analysis) and the fourth phase for item testing and validation against ACR 1997 and SLICC [10]. These criteria rely on a scoring system for clinical and laboratory domains [21], and a positivity of antinuclear antibody (ANA) at a titer 1:80 or higher by immunofluorescence (IFA) as an entry criterion [22]. The patient is classified as SLE if the total score is equal to or greater than 10 [10, 11]. EULAR/ACR criteria were tested, simplified and validated in a large (n = 2.218) international cohort. Performance characteristics found a sensitivity similar to the SLICC criteria (98% versus 95% for SLICC and 85% for ACR 1997) while maintaining the specificity of the ACR 1997 criteria (97% versus 95% for ACR 1997 and 90% for SLICC) [10]. Limitations indicated by the authors were the possible misclassification of patients with overlapping syndromes and the non-inclusion of new biomarkers [10, 19]. Other authors pointed out that the lack of extensive data on the longitudinal expression of ANA could affect the application of classification criteria in which ANA expression is the entry point [23].

This study aims to compare the performance of ACR 1997, SLICC criteria and the new EULAR/ACR criteria, to identify patients with cSLE at first visit and 1-year-follow-up.

Methods

Inclusion criteria

Children and adolescents, with cSLE (cases) or other defined rheumatic diseases (controls), with ANA reactivity at ≥1:80 serum dilution, followed-up at the Pediatric Rheumatology Unit of our University Hospital, from 2000 to 2017, were consecutively selected, from the number of patients evaluated in the clinic during the inclusion period.

To be included, cases and controls patients needed to have a well-established clinical diagnosis, performed and confirmed by three highly experienced pediatric rheumatologists, with over than 20 years experience in pediatric rheumatology and cSLE, of the medical staff of the outpatient clinic. All baseline and evolutionary information (physical examination, laboratory parameters, and imaging), was routinely discussed and re-evaluated at each visit, by attending pediatric rheumatologists, which established the diagnosis based on continuous follow-up of all patients and total agreement about diagnosis, supported by internationally accepted criteria [2429].

Exclusion criteria

Patients with overlapping syndromes or undifferentiated disease and those patients followed-up for less than 1 year were excluded.

Data collection

We performed a medical chart review for all eligible patients, and information collected in a standardized file. Data collection was retrospective, extracted by two of the authors and reviewed by the other three authors before classification sets scoring. Discrepancies were solved by team discussion. Finally, all patient’s files (from cSLE cases and controls) were rated for each ACR 1997, SLICC and EULAR/ACR criterion that was or was not met, as laid out by the respective classification rule, at first visit, and at 1-year-follow-up. Cases and controls were classified as SLE if met ≥4 criteria for ACR 1997, ≥ 4 criteria or documented lupus nephritis with ANA, anti-dsDNA or both for SLICC and ≥ 10 or ≥ 13 total score for EULAR/ACR.

Baseline/first visit data were those obtained from the clinical history, physical examination and laboratory tests requested by attending pediatric rheumatologists at first visit. The immunologic assessments evaluated for cSLE and control patients were antinuclear antibody (ANA), anti-dsDNA, anti-Sm, anticardiolipin IgM and IgG, lupus anticoagulant, anti- β2-glycoprotein-I IgM and IgG (for patients who started follow-up after the year of 2012), direct Coombs test, levels of complement proteins C3 and C4, and VDRL (Venereal Disease Research Laboratory).

Criteria definitions

Definitions for each clinical or laboratory criterion were those provided by the respective criteria set. 1) ANA by indirect immunofluorescence, on human cell epithelioma (HEp-2) cells substrate, defined as positive if staining reactivity at ≥1:80; 2) Anti-dsDNA by indirect immunofluorescence, on Crithidia lucilae substrate, described as positive if staining reactivity at > 1:10 serum dilution; 3) Anti-Sm by enzyme-linked immunosorbent assay (ELISA), considered positive if above kit manufacturer cut-off value. 4) Anticardiolipin (aCL) IgM and IgG by ELISA, a cut-off value of 20 MPL or GPL for ACR1997 and SLICC criteria, and a cut-off value of 40 MPL or GPL for EULAR/ACR criteria set.

EULAR/ACR criteria: A positivity of antinuclear antibody (ANA) at a titer 1:80 or higher by immunofluorescence (IFA) is required as an entry criterion [22]. These criteria rely on weighted additive criteria divided into seven clinical domains and three immunological domains, for which attribution to SLE is critical [21]. For each domain, only the individual criterion of highest value is considered for the total score [10, 11]. Clinical domains include: 1) unexplained fever; 2) arthritis; 3) serositis (pleural or pericardial effusion, acute pericarditis); 4) mucocutaneous (acute cutaneous lupus, subacute/discoid lupus, alopecia, oral ulcers); 5) central nervous system involvement (seizures, psychosis and delirium); 6) hematological involvement (autoimmune hemolytic anemia, thrombocytopenia and leukopenia); 7) nephritis (proteinuria > 0.5 g/d, nephritis class III/IV, nephritis class II/V). Lupus nephritis class II/IV over class II/V gain the highest weight. Immunological domains consist of 1) autoantibodies (anti-dsDNA, anti-Sm); 2) low complement (both C3 and C4 OR only C3 or C4); 3) Antiphospholipid antibodies (anticardiolipin IgG ≥ 40 GPL or anti-β2 glycoprotein I IgG ≥ 40 GPL or lupus anticoagulant). The patient is classified as SLE if the total score is equal to or greater than 10, in the presence of at least one clinical criterion, and each criterion may occur serially or simultaneously and on at least one occasion [10, 11].

Statistical analysis

The Statistical Package for the Social Sciences (SPSS) 20.0 for Windows (IBM, Armonk, NY, USA) was used. Data were shown in median (range, IQR) for continuous variables, and further comparisons between groups were conducted using Student’s t-test. Differences between proportions or categorical variables were analyzed by Fisher exact test. We performed the calculation of two types of accuracy: global accuracy and cut-off point accuracy [30]. The global accuracy was calculated for all three criteria sets, through the area under the ROC curve (AUC), to estimate how high is the discriminative power of each criteria set. However, as the AUC tell us nothing about individual cut-off points, we calculated the cut-off point accuracy for ACR 1997 ≥ 4 criteria, for SLICC ≥4 criteria and EULAR/ACR total score (total score ≥ 10, ≥11 and ≥ 13), through two-by-two contingency tables. McNemar’s test was applied to assess differences in sensitivity and specificity between ACR1997, SLICC and EULAR/ACR classification sets. A p <  0.05 value was regarded as significant, and all analyses were two-tailed.

Ethics

The local Research Ethics Committee approved the study protocol before study commencement, under Number 2.421.080, on December 7th, 2017.

Results

Sample description

133 cSLE and 96 controls fulfilled inclusion criteria, but there were 11 (9.0%) losses for cases group and 7 (8.2%) for control group due to missing information about some variables in medical charts. Study population included 122 cases with a well-established clinical diagnosis of cSLE (82.8% female, median onset age of 10.32 years (2.8–17.1) and median follow-up time of 6.0 years) and 89 controls (75.3% female, median onset age of 9.0 years and median follow-up time of 6.0 years), see Table 1.
Table 1

General characteristics of case and control groups

 

Cases

(N = 122)

Controls

(N = 89)

P value

Sex ratio (female: male)

101:21

67:22

0.226

Median onset age, years

10.32

9.00

<  0.001

(range)

(2.8–17.1)

(1.1–15.8)

Interquartile range (years)

3.2

6.3

Median age at diagnosis, years

10.63

9.5

0.024

(range)

(4.1–17.3)

(1.9–17.8)

Interquartile range (years)

3.0

7.0

Median time to diagnosis, months

3.00

9.00

<  0.001

(range)

0–60

1–68

Interquartile range (years)

4.0

18.5

Median follow-up time, years

6.00

6.00

0.91

(range)

1–13

2–16

Interquartile range (years)

4.0

4.0

Cases were patients with a well-established clinical diagnosis of childhood-onset SLE (cSLE). Controls had systemic-onset juvenile idiopathic arthritis, JDM juvenile dermatomyositis, JSS juvenile systemic sclerosis, MCTD mixed connective tissue disease, SS Sjögren syndrome, APS primary antiphospholipid syndrome, or primary vasculitis (Behçet, polyarteritis nodosa, Takayasu’s arteritis and granulomatosis with polyangiitis)

Controls had the following diagnoses: 8 systemic-onset juvenile idiopathic arthritis (SoJIA), 34 juvenile dermatomyositis (JDM), 10 juvenile systemic sclerosis (JSS), 14 mixed connective tissue disease (MCTD), 12 Sjögren syndrome (SS), 3 primary antiphospholipid syndromes (APS), 8 primary vasculitides.

ACR 1997 criteria

The most commonly observed ACR criteria in cSLE cases, both at the first visit and at 1-year-follow-up, respectively, were: arthritis (77 and 86.1%), immunologic (59 and 76.2%), hematologic (59 and 73.8%) and malar rash (36.9 and 52.5%). Arthritis (64 and 70.8%), malar rash (19.1 and 29.2%), and hematological (18 and 21.3%) were the most observed criteria in the control group.

At first visit, the mean number of ACR criteria was 4.54 ± 0.16 for cases and 2.52 ± 0.12 for controls (p <  0.001). At 1-year-follow-up, this average was 5.71 ± 0.14 for cases and 2.82 ± 0.12 for controls (p <  0.001).

Fifteen controls were misclassified as JSLE at first visit: 1 SoJIA, 8 JDM, 1 JSS, 4 MCTD, and 1 SS. Six additional controls were misclassified at 1-year-follow-up: 1 JDM and 5 MTCD.

SLICC criteria

The most frequent SLICC criteria in cSLE, both at first visit and at 1-year-follow-up, were, respectively: arthritis (78.7 and 87.7%), hypocomplementemia (56.6 and 60.7%), acute cutaneous lupus (49.2 and 65.6%), alopecia (42.6 and 44.3%) and anti-dsDNA (37.7 and 53.3%). At first visit, cSLE cases fulfilled a mean of 6.07 ± 0.21 and controls 2.69 ± 0.13 SLICC criteria (p < 0.001). At 1-year-follow-up, cases had a mean of 7.42 ± 0.20 and controls 3.00 ± 0.14 (p = 0.012) SLICC criteria.

Seventeen controls were misclassified as cSLE at first visit: one SoJIA, 2 JDM, 2 JSS, 9 MCTD, 1 primary APS, one SS, and one primary vasculitis. Four additional controls were misclassified at 1-year-follow-up: 2 JDM and 2 MCTD.

Table 2 presents the prevalence of SLICC criteria in cSLE cases and controls at baseline/first visit and 1-year-follow-up.
Table 2

Prevalence of clinical and immunological criterion of Systemic Lupus International Collaborating Clinics (SLICC) Classification Criteria in 122 cSLE cases and 89 controls, at baseline/first visit and 1-year-follow-up

Criterion

Cases

Controls

P value

Cases

Controls

P value

Baseline/First visit

Baseline/First visit

1-year follow-up

1-year follow-up

n (%)

n (%)

n (%)

n (%)

Acute cutaneous lupus

60 (49.2)

22 (24.7)

<  0.001

80 (65.6)

30 (33.7)

<  0.001

 Malar rash

45 (36.9)

17 (19.1)

0.006

64 (52.5)

26 (29.2)

0.001

 Photosensitive lupus rash

29 (23.8)

14 (15.7)

0.169

34 (27.9)

16 (18.0)

0.104

 Subacute cutaneous lupus

18 (14.8)

1 (1.1)

< 0.001

21 (17.2)

1 (1.1)

<  0.001

Chronic cutaneous lupus

13 (10.7)

1 (1.1)

0.005

16 (13.1)

1 (1.1)

0.001

 Discoid rash

13 (10.7)

1 (1.1)

0.005

15 (12.3)

1 (1.1)

0.003

Oral ulcers

44 (36.1)

11 (12.4)

<  0.001

64 (52.5)

12 (13.5)

<  0.001

Alopecia

52 (42.6)

12 (6.8)

<  0.001

54 (44.3)

14 (9.1)

<  0.001

Synovitis

96 (78.7)

59 (66.3)

0.058

107 (87.7)

65 (73.0)

0.011

Serositis

30 (24.6)

3 (3.4)

<  0.001

41 (33.6)

4 (4.5)

<  0.001

 Pleuritis

22 (18.0)

2 (2.3)

<  0.001

32 (26.2)

3 (3.4)

<  0.001

 Pericarditis

8 (6.6)

0

0.022

9 (7.4)

2 (2.3)

0.124

Renal disorder

33 (27.0)

1 (1.1)

<  0.001

49 (40.2)

2 (2.3)

<  0.001

 Proteinuria

33 (27.0)

1 (1.1)

<  0.001

49 (40.2)

2 (2.3)

<  0.001

 Red blood cell casts

10 (8.2)

0

<  0.001

15 (12.3)

1 (1.1)

<  0.001

Neuropsychiatric

10 (8.2)

1 (1.1)

0.027

23 (18.9)

1 (1.1)

<  0.001

 Seizures

5 (4.1)

0

0.075

17 (13.9)

0

<  0.001

 Psychosis

2 (1.6)

0

0.510

6 (4.9)

0

0.041

 Myelitis

1 (0.8)

0

0.999

3 (2.5)

0

0.265

 Confusional state

1 (0.8)

0

0.999

1 (0.8)

0

0.999

Hemolytic anemia

44 (36.1)

4 (4.5)

<  0.001

61 (50.0)

4 (4.5)

<  0.001

Leukopenia/lymphopenia

39 (32.0)

(10.1)

<  0.001

53 (43.4)

11 (12.4)

<  0.001

Thrombocytopenia

18 (14.8)

4 (4.5)

0.021

20 (16.4)

4 (4.5)

0.008

Anti-dsDNA

46 (37.7)

4 (4.5)

<  0.001

65 (53.3)

5 (5.6)

<  0.001

Anti-Sm

29 (23.8)

5 (5.6)

<  0.001

38 (31.1)

5 (5.6)

<  0.001

Antiphospholipid

30 (24.6)

5 (5.6)

<  0.001

38 (31.1)

4 (4.5)

<  0.001

 Anticardiolipin IgM

11 (9.0)

2 (2.3)

0.047

12 (9.8)

2 (2.3)

0.046

 Anticardiolipin IgG

23 (18.9)

2 (2.3)

<  0.001

27 (22.1)

3 (2.5)

<  0.001

 Lupus anticoagulant

6 (4.9)

5 (5.6)

0.999

8 (6.6)

5 (5.6)

0.999

 Anti-β2 GPI I IgM

3/61 (4.9)

1/42 (2.4)

0.644

3/61(4.9)

1/42(2.4)

0.644

 Anti- β2 GPI I IgG

3/61 (4.9)

2/42 (4.9)

0.999

3/61(4.9)

2/42(4.9)

0.999

 False-positive VDRL

4 (3.3)

1 (1.1)

0.4

4 (3.3)

1 (1.1)

0.4

Low complement

69 (56.6)

4 (4.5)

<  0.001

74 (60.7)

5 (5.6)

<  0.001

Direct Coombs test

34 (27.9)

6 (6.7)

<  0.001

38 (31.1)

6 (6.7)

<  0.001

The Pearson chi-square test analyzed differences between proportions, and the difference was regarded as statistically significant when p < 0.05

No patients had toxic epidermal necrolysis, hypertrophic lupus, mucosal lupus, lupus tumidus, and chilblains lupus. One cSLE case had bullous lupus, another one cSLE had panniculitis, and one control had mononeuritis multiplex at 1-year-follow-up. Anti-β2 glycoprotein-I IgM and IgG were tested in 61/122 (50%) cSLE cases and 42/89 (47.2%) controls, at baseline/first visit and 1-year-follow-up

EULAR/ACR criteria

To better evaluate the use of EULAR/ACR criteria in our cSLE population, we searched through ROC curve analysis, the total score cut-off that might optimize sensitivity and specificity, and we noticed that a total score ≥ 13, against the initially proposed ≥10 score, was more suitable for our cSLE patients. Besides, we also decided to perform a sensitivity analysis for different cut-off points of EULAR/ACR total score.

At first visit, if a EULAR/ACR total score ≥ 13 is adopted, a lower sensitivity (76.2% versus 87.7% for score ≥ 10, p < 0.0001) but a higher specificity (87.6% versus 67.4% for score ≥ 10, p < 0.0001), higher positive predictive value (89.4% versus 78.7% for score ≥ 10) and higher cut-off point accuracy would result.

At 1-year-follow-up, if an EULAR/ACR total score ≥ 13 is adopted, a similar sensitivity (91.0% versus 95.1% for score ≥ 10, p = 0.063), but again a higher specificity (83.2% versus 58.4% for score ≥ 10, p < 0.0001), higher positive predictive value (88.1% versus 75.8% for score ≥ 10) and higher cut-off point accuracy would result.

At first visit, cSLE cases achieved a mean EULAR/ACR total score of 19.64 ± 0.78 and controls 7.93 ± 0.54 (p = 0.002). At 1-year-follow-up, cases had a mean total score of 24.42 ± 0.74 and controls 9.19 ± 0.57 (p = 0.038).

Twenty-nine controls were misclassified as cSLE at first visit if total score ≥ 10 (3 SoJIA, 10 JDM, 2 JSS, 8 MCTD, 1 primary APS, 4 SS and 1 primary vasculitis), against eleven misclassified controls, if total score ≥ 13 (2 SoJIA, 1 JDM, 1 JSS, 6 MCTD, and 1 SS). Eight additional controls were misclassified at 1-year-follow-up if score ≥ 10 (4 JDM, 1 MCTD, 2 SS and 1 primary vasculitis), against 4 other controls if score ≥ 13 (2 JDM and 2 MCTD).

Table 3 displays the global accuracy for each criteria set.
Table 3

Global accuracy for ACR 1997, SLICC and the new EULAR/ACR criteria, at baseline/first visit and 1-year-follow-up

Classification System

Follow-up time

Global accuracy (IC 95%)

ACR 1997

Baseline/First visit

0.830 (0.776–0.884)

1-year-follow-up

0.933 (0.896–0.969)

EULAR/ACR

Baseline/First visit

0.874 (0.827–0.921)

1-year-follow-up

0.929 (0.894–0.964)

SLICC

Baseline/First visit

0.910 (0.870–0.949)

1-year-follow-up

0.952 (0.923–0.982)

Global accuracy was measured through the area under the ROC curve

IC 95–95% confidence interval, ACR American College of Rheumatology, SLICC Systemic Lupus International Collaborating Clinics criteria, EULAR/ACR European League Against Rheumatism (EULAR)/ACR

First visit - ACR 1997 versus SLICC criteria

SLICC criteria exhibited higher sensitivity (89.3% versus 70.5%, p < 0.001), but similar specificity (80.9% versus 83.2% ACR 1997, p = 0.791). SLICC criteria resulted in less misclassifications (30 versus 51, p < 0.001).

First visit - ACR 1997 versus EULAR/ACR total score ≥ 10

EULAR/ACR criteria total score ≥ 10 exhibited higher sensitivity than ACR 1997 (87.7% versus 70.5%, p < 0.001). However, ACR 1997 specificity was higher at first visit (83.2% versus 67.4%, p = 0.004). The number of misclassified patients was similar (44 for EULAR/ACR versus 51 for ACR 1997, p = 0.15).

First visit - ACR 1997 versus EULAR/ACR total score ≥ 13

An EULAR/ACR total score ≥ 13, resulted in no statistically significant difference in sensitivity (76.2% versus 70.5% for ACR 1997,p = 0.189); and specificity (87.6% versus 83.2% for ACR 1997,p = 0.424. EULAR/ACR ≥13 resulted in less misclassifications (40 versus 51, p = 0.023).

First visit - SLICC versus EULAR/ACR total score ≥ 10

Sensitivities were similar at first visit (89.3% for SLICC versus 89.3%, p = 0.687). However, SLICC specificity was higher at first visit (80.9% versus 67.4%, p = 0.008), with less misclassifications (30 versus 44, p = 0.003).

First visit - SLICC versus EULAR/ACR total score ≥ 13

The higher SLICC sensitivity persisted (89.3% versus 76.2%, p < 0.001). There were no differences in specificity (87.6% versus 80.9% for SLICC, p = 0.109). There were less misclassifications for SLICC (30 versus 40, p = 0.032).

1-year-follow-up - ACR 1997 versus SLICC criteria

Sensitivity (97.5% versus 95.1%, p = 0.250), and specificity (76.4% both, p = 0,999) were similar, as well as the number of misclassifications (24 versus 27, p = 0.48).

Global accuracy and cut-off point accuracy were higher for SLICC criteria at both observation periods.

1-year-follow-up - ACR 1997 versus EULAR/ACR total score ≥ 10

EULAR/ACR criteria total score ≥ 10 had similar sensitivity (95.1% for both, p = 0.999). However, ACR 1997 specificity again was higher (76.4% versus 58.4%, p = 0.002), with less misclassifications (43 versus 27, p = 0.006).

1-year-follow-up - ACR 1997 versus EULAR/ACR total score ≥ 13

An EULAR/ACR total score ≥ 13, displayed no statistical significant difference in sensitivity (91.0% versus 95.1% for ACR 1997, p = 0.063), or specificity (83.2% versus 76.4% for ACR 1997, p = 0.146), or in the number of misdiagnosis (26 versus 27, p = 0.816).

1-year-follow-up - SLICC versus EULAR/ACR total score ≥ 10

Sensitivities were similar (97.5% for SLICC versus 95.1%, p = 0.250). However, SLICC specificity was higher (76.4% versus 58.4%, p = 0.001). There were less misclassifications for SLICC (24 versus 43, p < 0.001).

1-year-follow-up-SLICC versus EULAR/ACR total score ≥ 13

The higher SLICC sensitivity persisted (97.5% versus 91.0%, p = 0.008). There were no differences in specificity (83.2% versus 76.4% for SLICC, p = 0.146) and misclassifications (24 versus 26, p = 0.636).

Table 4 (first visit), Table 5 (1-year follow-up) and Additional file 1: Tables S1-S5 summarizes comparative performance measures of the three classification criteria sets.
Table 4

Performance measures for ACR 1997, SLICC and new EULAR/ACR criteria, at baseline/first visit

Classification System

Cut-off point

Sensitivity (IC 95%)

Specificity (IC 95%)

PPV (%)

NPV (%)

Cut-off point accuracy (IC 95%)

ACR 1997

≥ 4

70.5%

83.2%

85.2%

67.3%

75.8

criteria

(61.6–78.4)

(73.7–90.3)

  

(69.5–81.4)

EULAR/ACR

Total

87.7

67.4

78.7

80.0

79.2

score ≥ 10

(80.5–93.0)

(56.7–77.0)

  

(73.0–84.4)

Total

76.2

87.6

89.4

72.9

81.0

score ≥ 13

(67.9–83.5)

(79.0–93.7)

  

(75.1–86.1)

SLICC

≥ 4

89.3

80.9

86.5

84.7

85.8

criteria

(82.5–94.2)

(71.2–88.5)

  

(80.3–90.2)

IC 95–95% confidence interval, PPV positive predictive value, NPV negative predictive value, ACR American College of Rheumatology, SLICC Systemic Lupus International Collaborating Clinics criteria, EULAR/ACR European League Against Rheumatism (EULAR)/ACR

Table 5

Performance measures for ACR 1997, SLICC and new EULAR/ACR criteria, at 1-year-follow-up

Classification System

Cut-off point

Sensitivity (IC 95%)

Specificity (IC 95%)

PPV (%)

NPV (%)

Cut-off point accuracy (IC 95%)

ACR 1997

≥ 4 criteria

95.1

76.4

84.7

91.9

87.2

(89.6–98.2)

(66.2–84.8)

  

(81.9–91.4)

EULAR/ACR

Total score ≥ 10

95.1

58.4

75.8

89.7

79.6

(89.6–98.2)

(47.5–68.8)

  

(73.6–84.8)

Total score ≥ 13

91.0

83.2

88.1

87.1

87.7

(84.4–95.4)

(73.7–90.3)

  

(82.5–91.8)

SLICC

≥ 4 criteria

97.5

76.4

85.0

95.8

88.6

(93.0–99.5)

(66.2–84.8)

  

(83.6–92.6)

IC 95–95% confidence interval, PPV positive predictive value, NPV negative predictive value, ACR American College of Rheumatology, SLICC Systemic Lupus International Collaborating Clinics criteria, EULAR/ACR European League Against Rheumatism (EULAR)/ACR

Discussion

Recently it was published that the new EULAR/ACR could be more sensitive and specific for adult SLE classification (especially for initial classification) than previous criteria sets. Concerning the need for more specific criteria for cSLE classification, we compared three available classification criteria sets. To the best of our knowledge, this is the first study regarding the assessment of performance among three classification systems (ACR 1997, SLICC and especially the new EULAR/ACR criteria) applied exclusively to cSLE patients, in two different observation periods (first visit and 1-year-follow-up).

Few studies have evaluated the performance of different SLE classification criteria in cSLE, showing variables results. For cSLE, three studies demonstrated that SLICC criteria classify patients earlier than ACR 1997 [1820]. In the multicentre European study by Sag and colleagues, the sensitivity of SLICC was higher (98.7% versus 85.3%, p < 0.001) but specificity was lower (76.6% versus 93.4%, p < 0.001) compared to ACR 1997, at time of diagnosis [18]. The lower specificity of SLICC criteria was mainly attributed to the fulfillment by controls with hemolytic-uremic syndrome or JDM [18]. Our group has previously assessed ACR 1997 versus SLICC classification at first visit and 1-year-follow-up, in a Brazilian single center cSLE cohort. Sensitivity for SLICC was higher than ACR 1997, 82.7% versus 58.0% at first visit (p < 0.001), but similar at 1-year (96.3% versus 91.3%, p = 0.125). Specificity was not significantly different [19]. The UK juvenile SLE cohort evaluated only sensitivity, and SLICC also were more sensitive than ACR 1997, both at diagnosis (92.9% versus 84.1%, p < 0.001) and at last visit (100% versus 92%, p < 0.001) [20].

Hartman and colleagues [17] conducted a systematic literature review for studies comparing ACR 1997 and SLICC criteria with clinical diagnosis in adult SLE and cSLE patients with disease duration up to 5 years. A meta-analysis estimated the sensitivity and specificity of these criteria sets and their variables. Four cSLE studies (568 cSLE patients, 339 controls), including our group previous study [19], were included, showing a higher sensitivity for early classification with SLICC criteria (99.9% versus 84.3%), but lower specificity than ACR 1997 (82.0% versus 94.1%).

In this present study, the lower SLICC specificity was not found, in contrast with most previous studies. Some reasons to explain why SLICC specificity was comparable to ACR1997 and higher than EULAR/ACR ≥ 10 in our study might be a different control group composition, the assessment of performance in distinct observation periods and especially the cut-off point for total EULAR/ACR score derived from our data.

SLICC criteria also exhibited the highest sensitivity for earlier classification (at first visit), in comparison to ACR 1997 (p < 0.001) and EULAR/ACR ≥ 13 (p < 0.001), but similar to EULAR/ACR proposed total score ≥ 10 (89.3% versus 87.7%, p = 0.687).

EULAR/ACR total score cut-off point influenced its performance measures. The selection of a EULAR/ACR total score ≥ 13, as determined by ROC curve analysis, against the initially proposed ≥10 score, resulted in higher specificity, positive predictive value, and accuracy. SLICC criteria exhibited higher global accuracy at both observation periods. Concerning the EULAR/ACR total score cut-off point being compared (whether ≥10 or ≥ 13), application of SLICC criteria still better scored in cut-off point accuracy both at first visit and at 1-year-follow-up, in our cSLE population.

The expanded scope of clinical (especially cutaneous and CNS manifestations) and immunologic manifestations included in SLICC criteria, besides the allocation of cytopenias and antibodies into separate criterion might have allowed a higher sensitivity, especially earlier in the disease course. The SLICC rule to classify only patients that have at least one immunologic criterion, preventing SLE classification based solely on clinical manifestations, and the ending of the “double counting” of photosensitive malar rash as two criteria (as in ACR 1997), may have increased SLICC specificity.

The differences in performance among those three sets of classification criteria might be due to changes either in the definition of organ involvement, to cut-off points for positive autoantibodies or to the form in which several clinical manifestations and laboratory parameters are gathered within each criteria set.

We are aware that this study is limited by the retrospective design and the relatively small number of patients and controls included. However, the extraction and collection of data by two authors, and the confirmation by the other three authors, before criteria set scoring, minimized this methodological limitation.

Second, a structural problem in designing and validating classification criteria, and thereby in interpreting our study, is an inherent lack of an objective diagnosis as the standard of reference other than clinical diagnosis, so that the treating physician’s diagnosis, is still adopted by most studies. We also decided to use as our standard of reference, the diagnosis consolidated during continuous follow-up of all patients by a group of highly experienced pediatric rheumatologists. It may be argued that this could lead to inconsistency; yet, it does allow evaluation of classification criteria in a real-world setting. Finally, we decided to compose our control group with rheumatic diseases that impose difficult differential diagnosis with JSLE, simulating the daily clinical practice; however, we did not include controls with undifferentiated disease or overlapping syndromes.

Considering the peculiarities of cSLE (more severe presentation and course, higher rates of disease activity and cumulative damage, higher frequency of atypical and constitutional manifestations) and taking into account that these three sets of criteria were developed in adult SLE, it should be considered modifications to increase early sensitivity and specificity for cSLE classification.

Conclusion

In this cSLE population, SLICC criteria better scored at first visit and 1-year-follow-up. The adoption of a EULAR/ACR total score ≥ 13 in this study, against the initially proposed ≥10 score, was most appropriate to classify cSLE. Further studies are necessary to address if SLICC criteria might allow fulfillment of cSLE classification earlier in disease course and may be more inclusive of cSLE subjects for clinical studies.

Notes

Abbreviations

APS: 

Antiphospholipid syndrome

JDM: 

Juvenile dermatomyositis

JSS: 

Juvenile systemic scleroderma

MCTD: 

Mixed connective tissue disease

SoJIA: 

Systemic onset juvenile idiopathic arthritis

SS: 

Sjögren syndrome

Declarations

Acknowledgements

We acknowledge Dr. Mariana Aires M.D. PhD and Dr. Elaine Sobral M.D. PhD, for helpful inputs.

Funding

There are no sources of funding for the research to be declared.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

ARF, MCFR, FRS, MGPL and SKFO contributed to the conception and design of study. ARF, MCFR, FRS, MGPL and SKFO contributed to acquisition of data. ARF and MGPL contributed to analysis of data and with the interpretations of data. ARF and MGPL were the major contributors in writing the manuscript. All authors revised the manuscript critically for intellectual content. All authors read and approved the final manuscript and the listing of authors.

Ethics approval and consent to participate

The local Research Ethics Committee approved the study protocol before study commencement, under Number 2.421.080, on December 7th, 2017.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Pediatric Rheumatology Unit, Instituto de Puericultura e Pediatria Martagão Gesteira, Universidade Federal do Rio de Janeiro (UFRJ), Rua Bruno Lobo, 50–Cidade Universitária, Rio de Janeiro, Brazil
(2)
Internal Medicine Post–graduation Program, Faculty of Medicine, Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil

References

  1. Hedrich CM, Smith EMD, Beresford MW. Juvenile-onset systemic lupus erythematosus (jSLE)-Pathophysiological concepts and treatment options. Best Pract Res Clin Rheumatol. 2017;31:488–504.View ArticleGoogle Scholar
  2. Harry O, Yasin S, Brunner H. Childhood-Onset Systemic Lupus Erythematosus: A Review and Update. J Pediatr. 2018;196:22–30 e2.View ArticleGoogle Scholar
  3. Brunner HI, Gladman DD, Ibanez D, Urowitz MD, Silverman ED. Difference in disease features between childhood-onset and adult-onset systemic lupus erythematosus. Arthritis Rheum. 2008;58:556–62.View ArticleGoogle Scholar
  4. Bundhun PK, Kumari A, Huang F. Differences in clinical features observed between childhood-onset versus adult-onset systemic lupus erythematosus: A systematic review and meta-analysis. Medicine (Baltimore). 2017;96:e8086.View ArticleGoogle Scholar
  5. Hochberg MC. Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum. 1997;40:1725.View ArticleGoogle Scholar
  6. Larosa M, Iaccarino L, Gatoo M, Punzi L, Doria A. Advances in the diagnosis and classification of systemic lupus erythematosus. Expert Rev Clin Immunol. 2016;12:1309–20.View ArticleGoogle Scholar
  7. Petri M, Orbai AM, Alarcon GS, Gordon C, Merrill JT, Fortin PR, et al. Derivation and validation of the systemic lupus international collaborating clinics classification criteria for systemic lupus erythematosus. Arthritis Rheum. 2012;64:2677–86.View ArticleGoogle Scholar
  8. Aringer M, Dörner T, Leuchten N, Johnson SR. Toward new criteria for systemic lupus erythematosus—a standpoint. Lupus. 2016;25:805–11.View ArticleGoogle Scholar
  9. Tedeschi SK, Johnson SR, Boumpas DA, Daikh D, Dörner T, Jayne D, et al. Developing and refining new candidate criteria for SLE classification: an international collaboration. Arthritis Care Res (Hoboken). 2018;70:571–81.View ArticleGoogle Scholar
  10. Aringer M, Costenbader KH, Brinks R, Boumpas D, Daikh D, Jayne D, et al. Validation of New Systemic Lupus Erythematosus Classification Criteria. In: 2018 ACR/AHRP annual meeting. Chicago; 2018. https://acrabstracts.org. Accessed 24 Oct 2018.
  11. Aringer M, Dörner T. Systemic Lupus Erythematosus (SLE) - New Classification Criteria [in German]. Dtsch Med Wochenschr. 2018;143:811–4.View ArticleGoogle Scholar
  12. Mosca M, Costenbader KH, Johnson SR, Lorenzoni V, Sebastiani GD, Hoyer BF, et al. How do patients with newly diagnosed systemic lupus erythematosus present? A multicenter cohort of early systemic lupus erythematosus to inform the development of new classification criteria. Arthritis Rheumatol. 2019;71(1):91–8.View ArticleGoogle Scholar
  13. Amezcua-Guerra LM, Higuera-Ortiz V, Arteaga-García U, Gallegos-Nava S, Hübbe-Tena C. Performance of the 2012 SLICC and the 1997 ACR classification criteria for systemic lupus erythematosus in a real-life scenario. Arthritis Care Res (Hoboken). 2015;67:437–41.View ArticleGoogle Scholar
  14. Inês L, Silva C, Galindo M, López-Longo FJ, Terroso G, Romão VC, et al. Classification of Systemic Lupus Erythematosus: Systemic Lupus International Collaborating Clinics versus American College of Rheumatology Criteria. A Comparative Study of 2.055 patients from a real-life, international systemic lupus erythematosus cohort. Arthritis Care Res (Hoboken). 2015;67:1180–5.View ArticleGoogle Scholar
  15. Ighe A, Dahlström O, Skogh T, Sjöwall C. Application of the 2012 Systemic Lupus International Collaborating Clinics classification criteria to patients in a regional Swedish systemic lupus erythematosus register. Arthritis Res Ther. 2015;17:3.View ArticleGoogle Scholar
  16. Oku K, Atsumi T, Akiyama Y, Amano H, Azuma N, Bohgaki T, et al. Evaluation of the alternative classification criteria of systemic lupus erythematosus established by systemic lupus international collaborating clinics (SLICC). Mod Rheumatol. 2018;28:642–8.View ArticleGoogle Scholar
  17. Hartman EAR, Van Royen-Kerkhof A, Jacobs JWG, Welsing PMJ, Fritsch-Stork RDE. Performance of the 2012 systemic lupus international collaborating clinics classification criteria versus the 1997 American College of Rheumatology Classification Criteria in adult and juvenile systemic lupus erythematosus. A systematic review and meta-analysis. Autoimmun Rev. 2018;17:316–22.View ArticleGoogle Scholar
  18. Sag E, Tartaglione A, Batu ED, Ravelli A, Khalil SM, Marks SD, Ozen S, et al. Performance of the new SLICC classification criteria in childhood systemic lupus erythematosus: a multicentre study. Clin Exp Rheumatol. 2014;32:440–4.PubMedGoogle Scholar
  19. Fonseca AR, Gaspar-Elsas MIC, Land MGP, de Oliveira SKF. Comparison between three systems of classification criteria in juvenile systemic lupus erythematous. Rheumatology (Oxford). 2015;54:241–7.View ArticleGoogle Scholar
  20. Lythgoe H, Morgan T, Heaf E, Lloyd O, Al-Abadi E, Armon K, et al. Evaluation of the ACR and SLICC classification criteria in juvenile-onset systemic lupus erythematosus: a longitudinal analysis. Lupus. 2017;26:1285–90.View ArticleGoogle Scholar
  21. Schmajuk G, Hoyer BF, Aringer M, Johnson SR, Daikh DI, Dörner T. Multi-center Delphi Exercise Reveals Important Key Items for Classifying Systemic Lupus Erythematosus. Arthritis Care Res (Hoboken). 2018;70:1488–94.View ArticleGoogle Scholar
  22. Leuchten N, Hoyer A, Brinks R, Schoels M, Schneider M, Smolen J, et al. Performance of anti-nuclear antibodies for classifying systemic lupus erythematosus: a systematic literature review and meta-regression of diagnostic data. Arthritis Care Res (Hoboken). 2018;70:428–38.View ArticleGoogle Scholar
  23. Pisetsky DS, Lipsky PE. The Role of ANA Determinations in Classification Criteria for SLE. Arthritis Care Res (Hoboken). 2018. https://doi.org/10.1002/acr.23559.
  24. Petty RE, Southwood TR, Manners P, Baum J, Glass DN, Goldenberg J, et al. International league of associations for rheumatology classification of juvenile idiopathic arthritis: second revision, Edmonton, 2001. J Rheumatol. 2004;31:390–2.PubMedGoogle Scholar
  25. Bohan A, Peter JB. Polymyositis and dermatomyositis (first of two parts). N Engl J Med. 1975;292:344–7.View ArticleGoogle Scholar
  26. Zulian F, Woo P, Athreya BH, Laxer RM, Medsger TA Jr, Lehman TJ, et al. The Pediatric Rheumatology European Society/American College of Rheumatology/European League against Rheumatism provisional classification criteria for juvenile systemic sclerosis. Arthritis Rheum. 2007;57:203–12.View ArticleGoogle Scholar
  27. Bartunkova J, Sediva A, Vencovsky J, Tesar V. Primary Sjögren’s syndrome in children and adolescents: proposal for diagnostic criteria. Clin Exp Rheumatol. 1999;17:381–6.PubMedGoogle Scholar
  28. Miyakis S, Lockshin MD, Atsumi T, Branch DW, Brey RL, Cervera R, et al. International consensus statement on an update of the classification criteria for definite antiphospholipid syndrome. J Thromb Haemost. 2006;4:295–306.View ArticleGoogle Scholar
  29. Kasukawa R, Tojo T, Miyawaki S. Preliminary diagnostic criteria for classification of mixed connective tissue disease. In: Kasukawa R, Sharp GC, editors. Mixed connective tissue disease and antinuclear antibodies. Amsterdam: Elsevier; 1987. p. 41–7.Google Scholar
  30. Šimundić AM. Measures of Diagnostic Accuracy: Basic Definitions. EJIFCC. 2009;20(19):203–11.Google Scholar

Copyright

© The Author(s) 2019

Advertisement