The mean (±Standard Deviation) and median DAI values obtained were 35.4 (±10.9) and 33.0, respectively.

The minimum and maximum values were 19 and 98.

For the ordinal DHC-IOTN, the minimum and maximum values ranged between 1 and 5.

The orthodontic treatment need according to the DAI and DHC-IOTN evaluation made by the examiner were presented on Table 1.

The intra-rater reliability assessment resulted in an intraclass correlation coefficient of 0.89 for the DAI (95% IC = 0.64 to 1.0) and 0.87 for the DHC (95% CI = 0.56 to 0.96).

Table 2 shows the comparison between the two indices, the Cohen Kappa and PABAK coefficients.


The time spent (in seconds) to assess the DAI and the DHC-IOTN were presented on Table 3.

These variables were not normally distributed (Kolmogorov-Smirnov test, P < 0.05).

The time spent to assess the DHC-IOTN was statistically lower than that for the DAI (Wilcoxon test, P < 0.001).


When comparing the two indices with the gold standard (Table 4), less agreement on the overall diagnosis of models examined for treatment needs was observed (47% according to the DAI and 52% according to the DHC-IOTN), with a significant percentage of false positives both for the DAI (41%) and the DHC-IOTN (39%).

The accuracy of the indices, as reflected by the ROC curve, was also presented (Figure 1).

In the analysis of the validity of the indices (Table 5), both had great sensitivity and very low specificity, indicating a good ability to identify orthodontic treatment need in patients.

However, the positive predictive value (PPV) for both indices was low, reducing the certainty of the sensitivity.

Otherwise, the specificity is low but the negative predictive value is high.

The new cutoff points (DAI=31, and DHC=3), have changed the properties of indexes.


Indices could be considered useful for epidemiological and public health applications when they are reliable and valid.

Considering the results presented, the DAI and DHC-IOTN could be considered reliable and validity.


For the sample size calculation of our study we could use the frequency of orthodontic treatment need as measured by the DHC-IOTN or by the DAI.

Considering that orthodontic treatment need based on research with orthodontic study models was about 15.0% [16], we opted for the frequency of orthodontic treatment need determined by the DAI [9] because it ensured a larger sample set.

The literature have pointed out that it is possible and correct to use dental models in order to validate orthodontic indices [14,17].

Besides, the reliability and agreement between the information obtained clinically and from diagnostic models are high [11].

For these reasons and due to feasibility, we carried out the study using dental models.


High intra-examiner agreement existed between the original DAI and DHC-IOTN values.

The examiner was trained and calibrated in the use of the indices before the evaluation sessions, which confirms the need for those steps before an epidemiological survey.

This step contributed to the good results.


However, the examiner was a specialist in orthodontics, and epidemiological surveys are normally conducted by general dentists, which may point to the need for more previous training.

It might be necessary to evaluate the reliability and validity of occlusal indices between general dentists as well.

It is important that the indices have a high degree of reproducibility to be useful as a research tool.


Despite the lower ICC DHC-IOTN as compared to the DAI value, the confidence intervals are coincident, showing that the reproducibility of both is similar [1,11,12,17,18].

Despite the high percentage of agreement between both indices, the Cohen Kappa could be considered fair.


However, considering that the agreement between positive classification (orthodontic treatment need) for DAI and DCH-IOTN was high, Cohen Kappa was artificially low.

So, the Prevalence-adjusted bias-adjusted Kappa was calculated and it was considered substantial.

So, the DAI and DHC-IOTN measure orthodontic treatment needs in the same way [15].


Most cases showed the need for treatment.

This high prevalence is similar to the results of other studies because these validation studies are usually conducted in orthodontics settings, where study models are provided and where most cases for treatment are, due to the need for diagnosis [14,16,17,19].


In validity studies of the occlusal indices, an important factor is the definition of the gold standard.

The literature has considered a panel of orthodontists to be the gold standard.

This assessment, as defined by several authors [20,21], has been considered the gold standard of the orthodontic treatment needs.

However, there are at least two ways to define this panel: using the Likert scale [17] or by consensus [12].

The number of orthodontists in this type of panel has varied from two [12] to eighteen [21].


Our study had a consensus panel of three orthodontists, similar to that of Freer and Freer [12].

It seems necessary to standardise the construction of these panels worldwide to better define the need for orthodontic treatment.

It is not easy to infer the effect on the validity statistics of the DHC-IOTN and DAI if the number of specialists participating in the panel were changed.

In Brazil, the post-graduate Orthodontic programs vary in content and length of study which may potentially increase the discordance among specialists' determinations of treatment need.


The comparison with the gold standard has shown an impressive amount of false positives.

This is a worrying finding because about 50% of the cases were determined to need treatment based on both indices which a committee of experts in orthodontics had not noticed.

In this case, an epidemiological survey using these indices may overestimate the need for treatment in a population.

The modification in the cutoff points has decreased the proportion of false-positive and has increased the proportion of false-negative results in both indices.

The overall concordance has slightly increased.


In the validity assessment, the DHC-IOTN showed a sensitivity of 100% and DAI, 91%, i.e., the probability of the assessment performed correctly indicate the orthodontic treatment needs is great.

Both showed low values of specificity (DAI = 14% and DHC-IOTN = 19%).


In epidemiological surveys, sensitive tests are useful because they prevent people with a problem from being disregarded.

Depending on the problem, this can be a complicating factor in finding a solution.

Moreover, specific tests are also desirable because they contribute to cost reduction both in the need for subsequent examinations and in the treatment that will be provided.

The low specificity is related to a high degree of false positives, which affects the good sensitivity.

Thus, it would be desirable to have a balance of these two characteristics, but that did not occur with the DAI and DHC-IOTN indices.

Thus, it is necessary to develop an occlusal index that evaluates orthodontic treatment needs more accurately.

This development process is not easy and could be done with participation of experts in orthodontics, public health, epidemiology, statistic from all over the world.


The positive predictive value (PPV) for the two indices is low (28 for the DAI and 31 for the DHC-IOTN).

Whereas PPV increases with increasing prevalence, this is another deficiency in the validity of these indices.

The modification in cutoff points increased the PPV and specificity for the two indices.

However, the others properties (sensibility and negative predictive value) have decreased.

The deficiencies observed in the characteristics analysed concerning the validity of a test resulted in the accuracy values (DAI = 0.61 and DHC-IOTN = 0.67).


Studies with American [2] or English [1] orthodontists showed better accuracy levels.

However, in another study [14], the accuracy of the IOTN was very similar to our results.

The validity of an index can depend on the origin of the orthodontic experts who determine as the gold standard [14].


As discussed previously, an expert's opinion is currently regarded as the best determinant of the treatment need because of the difficulty in using occlusal indices to identify and quantify the objective signs of malocclusion and orthodontic treatment needs [10,21,22].

Therefore, the aggregate decision of orthodontic specialists is generally regarded as the gold standard against which any occlusal index should be validated [20,21].

The different methods of obtaining the gold standard in the validation studies could also explain the different accuracy results for the occlusal indices [12,21].


The time spent for the assessment of the DAI was longer than for the DHC-IOTN.

This is probably because only the worst occlusal feature is recorded by the DHC-IOTN [3].

In other words, the identification is made through a hierarchical scale of occlusal anomalies, whereas several occlusal features of space and the teething are recorded to obtain the final DAI score.

Reducing the time needed for index application is always important, especially in population studies [22].

Despite not assessing the aesthetic component of the IOTN once we evaluated dental models [11], the inclusion of this component would increase the time spent in evaluation.

A disadvantage of DHC-IOTN use is that the proposed ruler for the index is not easily found, whereas the DAI is an index whose instrument for measurement (periodontal probe) is easily accessible.


It is necessary to point out some limitations of this study.

The study was conducted with a small group of Brazilian orthodontists, and the sample, although probabilistic, is representative of a single orthodontics service in Brazil.

Other studies should be conducted to assess the validity and reproducibility of the DAI and IOTN among Brazilian orthodontists.

Although there is little option for orthodontic treatment in public health in Brazil, the choice of a reliable and valid instrument is essential for a correct epidemiological diagnosis.

Additionally, the incorporation of subjective evaluation in the epidemiological diagnosis of orthodontic treatment need is absolutely relevant [23,24].

The studied indices are epidemiological tools that aim to assess the degree of treatment need and not make diagnoses or aid in orthodontic planning.

The epidemiological indices usually underestimate the studied disease, which has not occurred in this case.

Further research in this area is important so that the epidemiological findings can be utilised as a reliable tool for planning and evaluation of public health actions.


The DHC and the DAI are reproducible and have reasonable accuracy.

The biggest problem presented is the high false positive rate compared to the gold standard.

The DHC has the advantage of being an index of rapid implementation.

The time spent assessing the DAI is greater than that spent assessing the DHC.


In regard to the article entitled “The Dental Aesthetic Index and Dental Health Component of the Index of Orthodontic Treatment Need as Tools in Epidemiological Studies” by Cardoso et al. [ 1] which checks the validity and agreement of two scales for orthodontic treatment need, we would like to draw your attention to a number of discrepancies in the design and methodology which have affected the results:


The authors have chosen a sample where the prevalence of orthodontic treatment need (according to the gold standard) is 91%.

Unfortunately, this is not a true representation of the orthodontic treatment need of the population.

In a literature review of studies published between 1951 and 2000 (in total 25 studies), covering orthodontic treatment needs in different ethnic groups, Thilander et al. [ 2] reported only 10 (40%) studies reporting orthodontic treatment need above 75% while only one (4%) study (Law et al., [ 3]) in a Chinese population reported the prevalence of orthodontic treatment need above 90%, as reported by Cardoso et al. It is unfortunate that Cardoso et al. have chosen dental casts from the archive of Specialization Course in Orthodontics of a dental school—which indicates that these casts were prepared for the subjects who had undergone orthodontic treatment.

This gives rise to one question i.e. whether the 9% of casts shown to be not requiring orthodontic treatment were actually of subjects who did not require orthodontic treatment?

In our opinion, for such important research in which two scales were being evaluated and the study was carried out with the help of dental casts (post-hoc design), the valid option would have been to include a higher number of otherwise normal casts (as per the gold standard), as this was not a prospective study and could have been easily controlled.

Otherwise, the study should have been carried out as a random evaluation in a sample drawn from the population.


The authors have shown the diagnostic accuracy of the two scales to be 61% and 67%, respectively.

Now here we would like to provide a random simulation wherein the diagnostic accuracy will lead to an even higher result without using any scale in such a high prevalence situation.

Suppose, using random criteria we propose all the 100% subjects to be requiring the treatment (Table 1).


Table 1.

Diagnostic Efficacy in high prevalence cases.


Random criteria


Yes
No
Total


Gold standard


Yes
91
0
91


No
9
0
9


Total


100
0
100


Using these random criteria (which in fact are no criteria), owing to high prevalence the criteria has a diagnostic accuracy of 91%, sensitivity of 100% and specificity of 0%.

This simulation shows that in case of a high prevalence, the accuracy of a model can be a wrong inference.

The poor specificity as observed by Cardoso et al. in this article indicates that despite the use of proposed scales, the entire population has to be screened again by panel approach (gold standard) which shows that the time spent on administration of scales is of no use at all, rather it increases the overall time for treatment need assessment without adding any utility.

Secondly, in such a situation, DHC cut-off 2 seems to be a better choice as at least it does have a 100% sensitivity, which means that the panel has not to go beyond the screened subjects.


In regard to reproducibility, we are amazed to see that the authors have used test-retest validation for only 10% of the samples, which merely turns out to be 13 models.

It is difficult to assume that with the primary aim of the study being stated as “to assess the validity and reproducibility of the DAI and the DHC-IOTN in the identification of orthodontic treatment needs”, the reproducibility has been checked in only 13 observations.


4. Area under curve as depicted in Table 5 of the article (61% for DAI and 67% for DHC) indicates poor discrimination ability of the tests.

In an ROC curve, an area of 1 represents a perfect test; an area of 0.5 represents a worthless test.

According to rough estimates, an AUC value 0.60–0.70 shows poor accuracy of a diagnostic test, as no realistic classifier should have an AUC less than 0.5 (Fawcett [4]).


Under the circumstances, we do not agree with the conclusions of the authors that DHC and the DAI are reproducible and have reasonable accuracy and would recommend a rethinking of the applicability of DHC and DAI as per the results shown by the authors.


In response to the Commentary submitted by Arora et al. [ 1], which has questioned some methodological issues in our article [2], we would like to respond item by item.


(1) Arora et al. have pointed out that we performed a study with a prevalence of orthodontic treatment need (according to the gold standard) of 91%.

However, according to Table 4 of our study, the real prevalence of orthodontic treatment need is 52%.

Despite the fact that we chose the archives of Specialization Course in Orthodontics of a Dental School, this high prevalence (52%) is similar to the results of other studies because these validation studies are usually conducted in orthodontics settings, where study models are provided and where most cases for treatment are due to the need for diagnosis [3–6].

It is important to reinforce that we have also discussed these points in the second paragraph of page 3283.


(2) Arora et al. questioned “whether 9% casts shown to be not requiring orthodontic treatment were actually of subjects who did not require orthodontic treatment?” The models evaluated (n = 131) were a sample of 198 models of oral cavity of all orthodontic patients from Universidade Federal de Minas Gerais (first paragraph, page 3279).

These models pertained to patients who required orthodontic treatment in this University.


So, the prevalence of patients without orthodontic need (48%) was not a surprise for us.


(3) Considering that the prevalence of orthodontic treatment need was not 91%, as pointed out by Arora et al., this simulation of accuracy is not relevant.


(4) To evaluate reproducibility we examined 13 models.

In the literature concerning reproducibility of occlusal indexes there is no unique option regarding sample size.

Beglin et al. [ 5] and Firestone et al. [ 8] evaluated 40 dental models in order to measure reproducibility.

Ovsenik and Primoziv [3] on the other hand studied 10 models.

In this regard, our methodology lies between these extremes presented in the literature.


(5) The last issue is related to the validity evaluation.

The ideal value of an Area Under Curve (AUC) is 1.0.

The statistical program that we have used measured the AUC for both indexes and the 95% confidence intervals (95% CI).

These confidence intervals indicated if the curve crosses the non-significant 0.5 value at any point.

Considering that 95% CI for both indexes did not cross the non-significant 0.5 value, we sustain that both indexes present reasonable (not perfect) accuracy [9].


It is important to reinforce that we discuss the limitations of our study in the last paragraph, page 3284.

However, we cannot accept the criticisms laid out in this Commentary.


A 2017 study by Goyal et al 11 showed that there was a disparity between the demand for photo- therapy and the time devoted to learning it during residency.

Responses obtained from dermatology program directors across the United States showed that a majority (67%) regarded their phototherapy training as inadequate, which was primarily attributed to time deficiency.

11 A cross-sectional survey 12 among US dermatology residents, published in 2015, showed that approxi- mately 59% did not obtain any hands-on photo- therapy training and that 42% had never observed phototherapy at all.

Fewer than half of the residents felt that they could comfortably administer NB-UVB unsupervised, and fewer than 20% were comfortable with administering other modalities (excimer laser, PUVA, and BB-UVB).


Similarly, in 2015, Anderson et al 13 found that, among US dermatology residents, 29% and 76% were not comfortable with prescribing outpatient and home phototherapy, respectively.

This discomfort stemmed from a lack of exposure and is significant because dermatology trainees who do not develop enough confidence to prescribe phototherapy during residency are less likely to do so in practice.


Phototherapy primarily uses UV radiation.

Depending on the pathogenesis of the disease being treated and the specific modality prescribed, phototherapy counteracts the pathologic changes that char- acterize inflammatory skin diseases through several key mechanisms: (1) induction of apoptosis, (2) modification of the cytokine milieu, and (3) immunosuppression.

14-16 Phototherapy has been used successfully to treat many skin diseases.

A partial list is shown in Table I.


Among these, photo- therapy for psoriasis has the most data.

The different modalities that can be used for psoriasis are BB-UVB, NB-UVB, excimer light or laser, and PUVA (oral, topical, hand-foot soak, and bath/full-body soak).

17 BB-UVB is rarely used today and has largely been replaced by NB-UVB because of the latter's better efficacy.

18 Oral PUVA has superior efficacy to NB-UVB for psoriasis.


Treatment with NB-UVB can produce substantial improvement of moderate to severe psoriasis after approximately 20 to 36 treatments, whereas oral PUVA can generate equal or better results after a median of 16 to 17 sessions.

18 In a systematic review by Almutawa et al, 17 oral PUVA achieved the highest clearance rate (CR) (79%) but caused symptomatic erythema and blistering in 17% of patients.

NB-UVB attained a 68% CR and was better tolerated (adverse effects in 7.8%), and bath PUVA was the least effective (58% CR) and least tolerated (adverse effects in 21%).

Hence, although oral PUVA is more efficacious, better tolerability makes NB-UVB a preferred first-line phototherapy modality.

17 In practice, the high cost and intermittent availability of 8-methoxypsoralen in the United States, together with well-known, long-term adverse effects of photoaging and photocarcinogenesis, have further limited the use of PUVA.


Excimer Torres et al areas such as the scalp, palms, and soles.

4 It can produce results with as few as 8 treatment sessions compared to conventional NB-UVB, thereby reducing the cumulative UV dose.

4,18 Outside the realm of psoriasis, NB-UVB is a first-line UV-based treatment option for many conditions including vitiligo, early mycosis fungoides, and atopic dermatitis.

UVA1 penetrates deeper into the dermis and has shown benefit for sclerosing skin disorders such as morphea and scleroderma.

19 NB-UVB and UVA1 are also useful for therapeutic hardening of patients with polymorphous light eruption and solar urticaria, respectively.

20 For treatment-resistant dermatitis of the hands and feet, topical PUVA is frequently used.

Oral PUVA is helpful for plaque stage mycosis fungoides, other forms of cutaneous lymphoma, and as a second-line UV-based treatment option for recalcitrant skin conditions that have failed or responded inadequately to NB-UVB.

21 The list of photoresponsive dermatoses is long, which underscores the versatility of this treatment modality.


A comparative study by Inzinger et al 25 showed that the efficacy rates of oral PUVA were comparable to those of infliximab and exceeded those of etanercept, efalizumab, alefacept, adalimumab, and ustekinumab.

Additionally, a recent trial comparing patient-reported outcomes of NB-UVB versus adalimumab versus placebo showed that improve- ments in overall health-related quality of life scores were equivalent for both active interventions after 12 weeks.


The acute adverse effects of phototherapy are relatively minor and include pruritus, tenderness, erythema, tanning, and blister formation.

18 For PUVA, the application or ingestion of a photosensi- tizer can incite skin phototoxicity, nausea, and/or vomiting.

18 The true risk of UV-induced skin cancer has long been a subject of debate and is a cause of hesitation for patients and providers alike.

PUVA has been found to significantly increase the risk of squamous cell carcinoma in a dose-dependent manner, whereas the incidences of PUVA-induced melanoma and basal cell carcinoma are much lower.

14,27 NB-UVB and UVA1 have had no evidence of increased risk of photocarcinogenesis and are considered safe with proper supervision by most practitioners.

14,27,28 These risks can be mitigated via thoughtful patient and modality selection, proper dosing and dose adjustment, protection of uninvolved areas, moni- toring of cumulative UV dose, and periodic full-body skin examinations.

14 With the exception of PUVA, phototherapy does not require any pretreatment laboratory workup and can be safely administered to pediatric, elderly, and pregnant patients.


The adverse effects and safety of biologic medi- cations should also be considered.

With proper patient screening, biologics have shown good safety, particularly when compared to traditional systemic agents 24 ; however, long-term data are scarce.

infection because of the immunomodulatory effects of the medication.

Tumor necrosis factor inhibitors have specifically been associated with an elevated risk of tuberculosis, hepatitis B, lymphoma, and other malignancies, whereas IL-17 inhibitors have been shown to induce and exacerbate inflammatory bowel disease and increase the risk for mucocutaneous candidiasis.

24,29 Depression and suicidality have been reported among patients taking brodalumab, although evidence on causation remains controversial.

24,29 Furthermore, parenteral administration of biologics makes injection site and infusion reactions a possibility.

24,29 To minimize these risks, patients who are candidates for biologic therapy require workup before initiation and at regular intervals throughout treatment.

Laboratory tests consisting of a complete blood count, liver function test, hepatitis panel, and tuberculosis screening are regularly performed.

22,30 Additional tests may be warranted depending on patient-specific or medication-specific risk factors.

Recent guidelines suggest that certain biologic agents may be safely administered to pregnant, lactating, and pediatric patients; however, information regarding this is still limited.

22,31 The COVID-19 pandemic raised numerous questions regarding biologics' safety that lack clear, evidence-based answers.

Based on experience with HIV-positive patients, and in keeping with the practice of social distancing, home phototherapy is a reasonable option during this pandemic.


The price of treatment is a reality that must be considered when formulating a management plan; and for chronic skin conditions, this may entail lifelong expenditures.

According to a study in Scotland, the average price for a course of NB-UVB is £257, whereas topical medications cost £128 annually per patient.

Implementing NB-UVB resulted in a 40% reduction in cost (£50.74 per patient annually) due to less need for medications.

32 A 2010 analysis estimated that biologics cost at least twice as much as NB-UVB and PUVA combined.

The least expensive biologic included in the study, adalimumab, was quoted at $23,538 for the first year of therapy alone, compared to $3148 and $7582 for a year of NB-UVB and PUVA, respectively.

33 Six years later, the price of adalimumab increased by more than 2-fold ($58,045) for the first year of therapy alone.


Home phototherapy units cost anywhere between $2500 and $5000.

35,36 They can offset the indirect expenses incurred from office-based treatment (transportation, lost income from work absences, inconvenience), but a disadvantage is that patients must pay for the device up front because few insurance policies offer coverage.

A comparative study showed that although in-office phototherapy was less costly at the beginning of treatment, it became 5 times more expensive than home phototherapy after 3 months.

36 Additionally, when compared to biologics, home phototherapy cost up to 36 times less over a 3-year period.


Biologics have remarkable efficacy and a good safety record to date, and they are convenient to administer.

This may provide rationale for physicians and patients to justify the high cost, especially when factoring in the intangible phototherapy expenses.

However, some patients may respond inadequately to biologic therapy and require dose escalation or more frequent administration, thereby further increasing the cost of treatment.


In practice, treatments are often combined to enhance efficacy when rapid suppression of disease activity is desired or when monotherapy is insufficient to achieve and maintain control.

Several reviews and guidelines on psoriasis treatment have reported on the combination of phototherapy with various topical drugs, traditional systemic agents, and biologics.

14,16,38,39 With each medication having its own adverse effects, enhanced toxicity from combining 2 or more modalities is a possibility.

Contrarily, some combinations of medications may lower the chances of long-term adverse effects by reducing the cumulative dose of either modality alone or have minimal to no additive toxicity because of the relatively short duration of adjunctive treatment.

38 The concomitant administration of acitretin with NB-UVB has been found to hasten clinical response, reduce the required acitretin dose, and decrease the number of phototherapy sessions by approximately 20%, thereby lowering the cumulative UV dose and theoretical risk for photocarcinogenesis.

14,38,40 Similar effects have been observed with acitretin plus PUVA, and given the established skin cancer risk with this modality, the coadministration of an oral retinoid is particularly valuable.

40-43 Because retinoids have a keratolytic effect, phototherapy dose escalations must be proceeded with cautiously when using this combination.

14 Most protocols recommend UV dose reduction by 33%.


Several studies have shown that NB-UVB in conjunction with biologics is safe, synergistic, and well tolerated, although long-term data on these combinations have not been explored.

Patients with moderate to severe psoriasis who complied with J AM ACAD DERMATOL FEBRUARY 2021 482 Torres et al thrice-weekly NB-UVB in addition to etanercept showed biopsy-proven improvement without addi- tional adverse effects compared to etanercept alone.

38,44 Moreover, a half-body comparison of adalimumab plus NB-UVB versus adalimumab alone yielded a 33% greater PASI reduction on the irradiated body half compared to the nonirradiated half. 45 Analogous results were obtained in a similar trial on ustekinumab.

46 Erythema was the most common adverse effect.

45,46 To date, there are no studies on the combination of PUVA with any biologic.


One major challenge concerning adherence to phototherapy is limited access.

In the United States, phototherapy facilities are concentrated in urban areas of the East and West Coasts and the Midwest region east of Mississippi, leaving 89% of counties without a treatment center.

6 This misdistribution appears to reflect the geographic location of dermatology providers in general and not the underuse of phototherapy per se.

Nonetheless, the indirect costs of travel and lost income from missing work can discourage patients from complying with repeated phototherapy treatments.

6 Other limiting factors are the resources needed to establish a phototherapy center (equipment, space, trained staff) and the need for prior authorization.


Home phototherapy devices can potentially address these hurdles.

Several randomized trials have shown that home-based and office-based phototherapy are equally safe and effective, with higher satisfaction and adherence in favor of home phototherapy; however, this also entails higher up-front costs for the patient and the need for a provider who is sufficiently familiar with home phototherapy.

47 In addition, photother- apy has possible long-term risks.

Therefore, in practice, it is usually administered intermittently, and patients may experience disease relapses within weeks to months of discontinuation.

Combining phototherapy with other modalities can not only decrease the likelihood or severity of a relapse but also lower the risk of adverse effects from either modality alone.

With proper patient education, home-based phototherapy is another option.


The advantages and limitations of phototherapy and biologics are summarized in Table II.


Phototherapy remains an indispensable treatment option for many cutaneous diseases.

Its versatility, cost effectiveness, and unparalleled safety makes it a viable first-line treatment or adjunct when other treatment regimens fall short.

Just as there are numerous indications for phototherapy alone, there are a wide variety of modalities with which it can be combined.

As dermatologic management becomes more individualized and costly, improved access to this treatment modality through expanding residency training curricula and prescribing of home devices will prove that even in the era of biologics, phototherapy will stand the test of time.

Dermatologists are the only group of physicians who have the knowledge and expertise to supervise the delivery of phototherapy.

Therefore, it is essential that we as a specialty continue to make sure this treatment option is available to our patients.


The article 'Role of phototherapy in the era of biologics' underscores the safety and efficacy of an established treatment among exponentially increasing options [1].

In this review, it was stated that narrowband ultraviolet B (NB-UVB) in conjunction with biologics has been considered safe.

Our findings raise questions about this opinion, particularly for antitumor necrosis factor (TNF) agents [2].


Prolonged immunosuppression by ultraviolet radiation significantly increases the risk of skin cancers.

In the transplantation literature, the cumulative incidence of skin cancers has ranged from 40% to 70% at 20 years [3].

UVB is particularly responsible for the immunomodulatory features of phototherapy; thus, the combination of immunosuppressive agents with relatively safe-considered NB-UVB should also be evaluated with caution.


TNF-α has been implicated in photocarcinogenesis [4], so it was hypothesized that anti-TNF agents might provide protection against the effects of DNA damage induced by UVB irradiation.

Faurschou et al. investigated the effects of TNF-α and infliximab in UVB-irradiated HaCaT keratinocytes and, as an interesting finding, both groups showed increased levels of DNA damage compared to that of the control group [5].


Our research group conducted a study in SKH-1 mice to test the potential of several immunosuppressive agents (infliximab, etanercept, and cyclosporine) to enhance UVB-induced carcinogenicity [2].

For the initial 10 weeks, mice were irradiated by a targeted UVB device.

The spectral output of this device peaks at 302 nm and 312 nm, with an average wavelength of 304 nm.

A predetermined minimal erythema dose was applied thrice per week before the administration of treatment agents.

The combination of the experimental drug and phototherapy was continued for an additional period of 14 weeks.

Drug doses were based on published effective doses in rodents, and the frequency of the applications was reproduced from the scheme for patients with psoriasis.

In a total experimental period of 24 weeks, higher tumor burden and keratinocytic neoplasia numbers were detected in the etanercept group.

The lack of statistical significance for other active treatment groups might be related to the small group sizes.


While the results cannot be extrapolated to the human situation, caution is advised.

Additionally, the light source and the timing of phototherapy were different from the clinical scenario because in studies evaluating this combination, NB-UVB phototherapy was initiated with the introduction of anti-TNF agents, and cessation of phototherapy was considered in the third month.


To the Editor: We read with interest and appreciation the commentary of Botsali [1], which expounds on the potential harm associated with the combination of narrowband ultraviolet (UV) B (NB-UVB) phototherapy and immunosuppressive agents, specifically, tumor necrosis factor inhibitors (anti-TNF).

Having been around for more than 4 decades, phototherapy has a well-known efficacy and long-term safety profile.

Because immunosuppression is one of the key mechanisms by which phototherapy controls inflammatory skin diseases [2], the risk of immunosuppression and subsequent cancer development is of particular concern, especially when phototherapy is combined with systemic immunosuppressive therapies.

Although it has been found that photochemotherapy with psoralen plus UVA significantly increases the risk of keratinocyte carcinomas, there has been no such evidence for NB-UVB monotherapy [2].

A retrospective study of more than 3000 patient records found no significant association between NB-UVB and the development of melanoma and keratinocyte carcinomas [3].

The median number of phototherapy sessions was 29, which approximates a typical course in clinical practice wherein substantial improvement in skin lesions is seen [2,3].

Previous human studies have also shown that the combination of NB-UVB with anti-TNF agents or other biologics is relatively safe [2].

However, given the lack of long-term data on these combinations, it is reasonable to proceed with caution.


Of note, concomitant therapy with NB-UVB and biologics is typically done in clinical practice for only a few months.

Concerns regarding immunosuppression are of special importance amidst the ongoing COVID-19 pandemic.

Patients may opt to discontinue treatment and risk having a disease flare because of fears of contracting the virus.

According to a recent study by Veenstra et al [4], patients on immunosuppressive medications (including anti-TNF agents) for various immune-mediated inflammatory conditions (including psoriasis) did not show a significantly elevated risk of acquiring COVID-19 or having a more severe disease course compared to the general population.

In addition, anti-TNF therapy was associated with a decreased likelihood of hospitalization due to COVID-19 [4].

The study did not account for patients who are on topicals or phototherapy in addition to immunosuppressive medications.

However, given that patients who are on immunosuppressive drugs often have moderate to severe skin disease, it is possible that these patients have a current or previous course of phototherapy.

Moreover, based on clinical experience with HIV-positive patients, phototherapy is a safe and reasonable option during this time [2].


Finally, care must be taken when extrapolating data obtained from in vitro or animal studies into clinical practice.

The mouse study conducted by Botsali and colleagues provided valuable insight into the potential additive carcinogenic effect of combining NB-UVB with anti-TNF agents [5].

However, as the commenting author recognized, these results cannot be readily applied to the human population.

This is because although useful information can be obtained from in vitro or animal models, they often lack the intricacies and complex interplay of human anatomy and physiology.

In summary, the aim of the review ‘‘Role of Phototherapy in the Era of Biologics’’ was to emphasize the utility, cost effectiveness, and versatility of phototherapy as a treatment option.

Because no therapeutic modality is completely devoid of risk, appropriate adjustments should be made on a case-to-case basis.


There is a statement in the introduction of the commu- nication that many breakfast cereals are 'energy-dense and nutritionally poor' (1) .

This statement cannot be sub- stantiated.

In fact, many, if not most, ready-to-eat cereals are nutrient rich, containing 25 % of the recommended dietary intake for many B vitamins, folate and Fe.

Break- fast cereal is one of the best sources of dietary Fe for both Kiwi adults and children (2,3) .

It is also well known that breakfast cereal eaters have higher nutrient intakes than those who skip or eat other foods for breakfast (4) .

They are also more likely to meet their fibre targets (4) .

Many ready- to-eat cereals, oats and muesli products are high in whole grains and often contain nuts and seeds, which further contribute to their intrinsic nutritional value.


The communication also states that many cereals in the New Zealand market are high in fat (1) .

There are very few breakfast cereal products in the market that would qualify as 'high in fat', for which there is no criterion for mea- surement in New Zealand.

The highest mean fat content in Table 1 in the communication is for muesli at 16·5 % or less than 8 g per 45 g serving.

In addition, less than one-third of the total fat is coming from saturated fat, indicating the use of non-saturated sources of fat by manufacturers.


The introduction to the communication also implies that there is no governance of breakfast cereal manufacture or labelling in New Zealand other than under Standard 1.2.7 in the Australia New Zealand Food Standards Code (1) :


'There is no regulation of formulation, labelling or promotion of breakfast cereals in NZ.

However, Food Standards Australia New Zealand (FSANZ) implemented Standard 1.2.7 in 2016, which requires that health claims be present only on foods meeting a Nutrient Profile Scoring Criterion (NPSC) score of <4.'


This is not the case for regulation in New Zealand.

The nutrition labelling of all breakfast cereals in New Zealand is strictly regulated under Standard 1.2.8 of the Code (5) .

In conjunction, the addition of vitamins and minerals is regulated in Standard 1.3.2 and there are other stipulated requirements for cereals in Standard 2.1.1.

There are fur- ther requirements which manufacturers must meet for allergen warnings and advisory statements as well as the use of novel ingredients.

What is more, not all cereals are required to meet NPSC as part of Standard 1.2.7.

Since nutrient content claims are governed under this standard, and do not require a passing of the NPSC.


The increases in mean total fat and energy in flakes and muesli shown in Table 1 of the communication would not come as a surprise to many working in food devel- opment in the cereal category.

The use of ingredients with more whole grains, nuts and seeds is evident across the category, largely due to innovation.

This would explain increases in energy, total fat and saturated fat in products that were generally low in fat and saturated fat to begin with.

In fact, when it comes to flakes, puffs and bubbles, there is little opportunity to increase fat in these products other than with the addition of nuts, seeds and wholegrain clusters.

Growth in granola and muesli, which are largely oat based and incorporate nuts and seeds, is also evident with the New Zealand market showing 30 % growth over the last 5 years (6) .

The addition of coconut to many cereals, which is also based on consumer demand, contributes to the fat and saturated fat contents.


Therefore, an increase in the mean energy, total fat or saturated fat content of the category is not a reliable indicator of a negative change in healthfulness.

In fact, if anything, the ingredient profiles of many cereals have been improved with the use of more whole ingredients having healthier fat profiles.


With respect to changes in protein in this category, it is not realistic to look for significant changes in a cate- gory that is largely grain based.

The addition of nuts can help together with soya crisps and pea protein, but this is likely to be in a small group of variants.

It is therefore unlikely that protein in this category will shift very much over time.


The discussion section of the communication explicitly states that a serving of breakfast cereal exceeds recom- mended intakes for sugar, Na and fat (1) :


'Unlike the USA, there appears to be little impetus in either Australia or New Zealand to reformulate breakfast cereal into a healthier product, despite the fact that one bowl of cereal can exceed the recom- mended dietary intake of sugar, Na and fat.'


The Daily Values (DV) or Reference Intakes used in the Food Standards Code are 90 g for sugars, 2300 mg for Na and 70 g for fat, with 24 g of that from saturated fat based on a diet of 8368 kJ/d (2000 kcal/d) (5) .

It is unlikely that a bowl of cereal would exceed these amounts, especially since one serving is likely to be between 30 and 45 g of cereal.

Even two servings of any cereal product would not reach the DV.

In fact, a breakfast cereal with 30 % sugar would provide 9 g of sugar per serving, which equates to 10 % of the 90 g DV for sugar.


The original communication references the 'added sugar or free sugars' recommendation from the WHO (10 % of energy intake from 'added or free' sugars) as the 'recommended dietary intake for total sugars' (1) .

Even in applying this recommendation correctly, it would equate to approximately 50 g of added sugars for adults, 25 g for young children and up to 50 g for teenagers.

There is currently no cereal that would reach those levels of added sugar in a serving or even in two servings.

What is more, the most recent national nutrition surveys in New Zealand showed that breakfast cereals contribute less than 3 % of total daily sugar intakes in New Zealand adults (2) and 3 % in children (3) .


This is also true for fat and saturated fat, given that the DV are 70 g and 24 g, respectively.

There is no cereal on the market that would exceed 70 g of fat or 24 g of saturated fat in a serving, particularly since the serving size is likely to be between 30 and 45 g.


The same logic would then apply to Na with a serving of cereal supplying under 200 mg, which equates to approximately 10 % of the suggested dietary target (7) .


The statement from the communication listed above is not factual and is void of reality.


The definition of reformulation according to the Oxford Dictionary is to 'formulate again or differently'.

This assumes that you are changing an already existing product.

To address reformulation, you would have to examine the same products over time.

Addressing reformulation performance based on mean nutrient intakes assumes a level of stability in the market, a stability that we cannot guarantee.

In addition, the methodology used in the communication is likely to dilute intra-individual variances over time.

This, combined with the shifting sample group, makes this assessment of reformulation problematic.


The breakfast cereal category has changed substantially over the years.

Na has been reduced across the category in line with recommendations; some of the largest brands in New Zealand have reduced their sugar content (8) and manufacturers are adding more whole ingredients such as whole grains, nuts and seeds.

All of this contributes to diverse and nutritious options becoming available to the New Zealand market.


The communication has made a number of untrue and misleading statements about the breakfast cereal category and the authors need to provide clarity for readers around their conclusions, particularly those statements regarding cereal exceeding recommended intakes.


We write this letter in response to a short communication entitled ‘The nutritional quality of New Zealand breakfast cereals: an update’, published in the December 2017 journal [1].

There are a number of misrepresentations of the New Zealand breakfast cereal category in the communication that need to be addressed by the authors.

Breakfast cereals contribute valuable nutrients to the Kiwi diet.

There is a statement in the introduction of the communication that many breakfast cereals are ‘energy-dense and nutritionally poor’ [1].

This statement cannot be substantiated.

In fact, many, if not most, ready-to-eat cereals are nutrient rich, containing 25% of the recommended dietary intake for many B vitamins, folate and Fe.

Breakfast cereal is one of the best sources of dietary Fe for both Kiwi adults and children [2,3].

It is also well known that breakfast cereal eaters have higher nutrient intakes than those who skip or eat other foods for breakfast [4].

They are also more likely to meet their fibre targets [4].

Many ready-to-eat cereals, oats and muesli products are high in whole grains and often contain nuts and seeds, which further contribute to their intrinsic nutritional value.

The communication also states that many cereals in the New Zealand market are high in fat [1].

There are very few breakfast cereal products in the market that would qualify as ‘high in fat’, for which there is no criterion for measurement in New Zealand.

The highest mean fat content in Table 1 in the communication is for muesli at 16.5% or less than 8 g per 45 g serving.

In addition, less than one-third of the total fat is coming from saturated fat, indicating the use of non-saturated sources of fat by manufacturers.

Cereal manufacture and labelling are regulated under the Australia New Zealand Food Standards Code.

The introduction to the communication also implies that there is no governance of breakfast cereal manufacture or labelling in New Zealand other than under Standard 1.2.7 in the Australia New Zealand Food Standards Code [1]: ‘There is no regulation of formulation, labelling or promotion of breakfast cereals in NZ.

However, Food Standards Australia New Zealand (FSANZ) implemented Standard 1.2.7 in 2016, which requires that health claims be present only on foods meeting a Nutrient Profile Scoring Criterion (NPSC) score of <4.

’ This is not the case for regulation in New Zealand.

The nutrition labelling of all breakfast cereals in New Zealand is strictly regulated under Standard 1.2.8 of the Code [5].

In conjunction, the addition of vitamins and minerals is regulated in Standard 1.3.2 and there are other stipulated requirements for cereals in Standard 2.1.1.

There are further requirements which manufacturers must meet for allergen warnings and advisory statements as well as the use of novel ingredients.

What is more, not all cereals are required to meet NPSC as part of Standard 1.2.7.

Since nutrient content claims are governed under this standard, and do not require a passing of the NPSC.


Mean energy, total fat and saturated fat are not reliable indicators of the healthfulness of the category.

The increases in mean total fat and energy in flakes and muesli shown in Table 1 of the communication would not come as a surprise to many working in food development in the cereal category.

The use of ingredients with more whole grains, nuts and seeds is evident across the category, largely due to innovation.

This would explain increases in energy, total fat and saturated fat in products that were generally low in fat and saturated fat to begin with.

In fact, when it comes to flakes, puffs and bubbles, there is little opportunity to increase fat in these products other than with the addition of nuts, seeds and wholegrain clusters.

Growth in granola and muesli, which are largely oat based and incorporate nuts and seeds, is also evident with the New Zealand market showing 30% growth over the last 5 years [6].

The addition of coconut to many cereals, which is also based on consumer demand, contributes to the fat and saturated fat contents.

Therefore, an increase in the mean energy, total fat or saturated fat content of the category is not a reliable indicator of a negative change in healthfulness.

In fact, if anything, the ingredient profiles of many cereals have been improved with the use of more whole ingredients having healthier fat profiles.

With respect to changes in protein in this category, it is not realistic to look for significant changes in a category that is largely grain based.

The addition of nuts can help together with soya crisps and pea protein, but this is likely to be in a small group of variants.

It is therefore unlikely that protein in this category will shift very much over time.

A bowl of breakfast cereal is physically unable to exceed daily nutrient targets.


The discussion section of the communication explicitly states that a serving of breakfast cereal exceeds recommended intakes for sugar, Na and fat [1]: ‘Unlike the USA, there appears to be little impetus in either Australia or New Zealand to reformulate breakfast cereal into a healthier product, despite the fact that one bowl of cereal can exceed the recommended dietary intake of sugar, Na and fat.’ The Daily Values (DV) or Reference Intakes used in the Food Standards Code are 90 g for sugars, 2300 mg for Na and 70 g for fat, with 24 g of that from saturated fat based on a diet of 8368 kJ/d (2000 kcal/d) [5].

It is unlikely that a bowl of cereal would exceed these amounts, especially since one serving is likely to be between 30 and 45 g of cereal.

Even two servings of any cereal product would not reach the DV.

In fact, a breakfast cereal with 30% sugar would provide 9 g of sugar per serving, which equates to 10% of the 90 g DV for sugar.

The original communication references the ‘added sugar or free sugars’ recommendation from the WHO (10% of energy intake from ‘added or free’ sugars) as the ‘recommended dietary intake for total sugars’ [1].

Even in applying this recommendation correctly, it would equate to approximately 50 g of added sugars for adults, 25 g for young children and up to 50 g for teenagers.

There is currently no cereal that would reach those levels of added sugar in a serving or even in two servings.

What is more, the most recent national nutrition surveys in New Zealand showed that breakfast cereals contribute less than 3% of total daily sugar intakes in New Zealand adults [2] and 3% in children [3].

This is also true for fat and saturated fat, given that the DV are 70 g and 24 g, respectively.

There is no cereal on the market that would exceed 70 g of fat or 24 g of saturated fat in a serving, particularly since the serving size is likely to be between 30 and 45 g. The same logic would then apply to Na with a serving of cereal supplying under 200 mg, which equates to approximately 10% of the suggested dietary target [7].

The statement from the communication listed above is not factual and is void of reality.


Reformulation in the category has been happening for decades and is still continuing.

The definition of reformulation according to the Oxford Dictionary is to ‘formulate again or differently’.

This assumes that you are changing an already existing product.

To address reformulation, you would have to examine the same products over time.

Addressing reformulation performance based on mean nutrient intakes assumes a level of stability in the market, a stability that we cannot guarantee.

In addition, the methodology used in the communication is likely to dilute intra-individual variances over time.

This, combined with the shifting sample group, makes this assessment of reformulation problematic.

The authors of the original communication do acknowledge that individual reformulations were not possible to assess, which makes the conclusion that cereal manufacturers are not doing anything to reformulate their foods a rather broad assumption [1].

Using average changes in nutrient contents across a dynamic category, to draw this conclusion, is prone to error.

In fact, it is likely that there was rapid innovation taking place in this category over the last 5 years.

This is evident from the entry of new brands, which is not recognised in the communication and could explain changes in mean intakes.

The breakfast cereal category has changed substantially over the years.

Na has been reduced across the category in line with recommendations; some of the largest brands in New Zealand have reduced their sugar content [8] and manufacturers are adding more whole ingredients such as whole grains, nuts and seeds.

All of this contributes to diverse and nutritious options becoming available to the New Zealand market.

The communication has made a number of untrue and misleading statements about the breakfast cereal category and the authors need to provide clarity for readers around their conclusions, particularly those statements regarding cereal exceeding recommended intakes.


We wish to respond to a letter by Gina Levy of Kellogg (Australia) Pty Ltd – Research and Technology, Australia entitled ‘The New Zealand breakfast cereal category is dynamic and responsive to consumer preferences’, published in Public Health Nutrition [1] in response to our published article ‘

The nutritional quality of New Zealand breakfast cereals: an update’ [2].

We thank the author for her interest in our publication and will respond to several of her comments.


The above letter suggests a misrepresentation of the nutritional quality of breakfast cereals with our wording that many cereals can be ‘energy-dense and nutritionally poor’.

While we concur with Ms Levy that some ready-to-eat cereals contain moderate levels of natural and supplemental vitamins/minerals, as well as whole grains, nuts and seeds, our comment was in relation to the proportion of nutrients in these products and their alignment with the New Zealand recommended nutrient intakes from cereals.

Many of the New Zealand breakfast cereals analysed for our original publication had up to 30% of their energy content from sugar alone, with the Na content of some cereals being up to 350 mg per 100 g of product.

With New Zealand currently having a very high prevalence of obesity [3], the 30% sugar content of cereals, particularly those marketed towards children, is nutritionally poor, despite the fact that these cereals may offer whole grains as a primary carbohydrate source.


We acknowledge an ambiguity when we described a bowl of cereal as exceeding the recommended intakes for sugar and Na. We should have clarified that these levels can be exceeded when one considers the proportion of kilojoules and nutrients normally consumed in the breakfast meal alone.

It is documented that many cereal eaters (both adults and children) regularly consume more than the manufacturer-declared serving size in one sitting [4,5] and that children will voluntarily eat more of a high-sugar cereal (up to twice the recommended serving size) compared with a lower-sugar alternative [6].

When looking at the mean sugar values of children’s breakfast cereals in our New Zealand data set, the intake from double the recommended serving size, with 200 ml of milk, would equate to approximately 25 g sugar in one breakfast meal.

Ms Levy acknowledged that a child’s maximum sugar intake should be approximately 25 g/d (based on WHO recommendations that sugar should not exceed 10% of daily energy intake [7]).

Thus, a child’s total recommended daily sugar allowance could be consumed in this one meal alone.


The literature reports that increasing breakfast energy is associated with greater overall food intake in normal-weight and obese subjects, and that those who consume a large breakfast meal do not compensate by eating smaller meals later in the day [8].

Furthermore, it has been reported that children will readily consume lower-sugar cereals if offered to them [6,9] and our argument is that New Zealand manufacturers do not currently provide adequate low-sugar cereal options.

Except for ‘Biscuits and Bites’, the mean sugar content of all breakfast cereal categories in 2017 was 20% or more, suggesting that adults are also consuming a significant amount of sugar from their breakfast cereal meal (particularly when combined with milk and any added fruits or sugar).

The ‘adequate intake’ of Na has been shown to range from 200–400 mg in children aged 2–3 years, through to 460–960 mg in youth aged 14–18 years [10].

Thus, it is concerning that one 40 g serving of dry cereal can provide up to 200 mg Na, especially as Ministry of Health data report that Na from breakfast cereals accounts for only 6% of the daily Na intake [11].

If those eating high-sugar cereals are indeed eating up to double the recommended serving size as suggested previously [6], then the Recommended Daily Intake (RDI) of Na for younger consumers is also being reached after consumption of the breakfast meal alone.


The Ministry of Health in New Zealand currently recommends that high-fat, -sugar and -salt (HFSS) foods and drinks should be consumed no more than once per week [10].

Yet, breakfast cereals are generally marketed as being foods that can be consumed on a daily basis.

Certainly, not all New Zealand breakfast cereals would be termed a HFSS food based on their sugar, fat or Na content, but a large proportion would be based on our data collected in 2017.

This is also concerning, as New Zealand television advertising shows proportionately twice as many breakfast cereals advertisements as Australia [12].

It is also a worry that promotional characters used to market cereals are more likely to feature on cereals calculated to be ‘less healthy’ when using Nutrient Profile Scoring Criterion scores [13].


We acknowledge Ms Levy’s comments about a lack of data to support our statements about product reformulation, and this was noted as a limitation in our original publication.

However, as we did not have access to individual product details from the original 2013 data set, we were unable to determine whether specific product reformulations had taken place.

Despite this, we suggest that the lack of change seen at the category level in our data is still indicative that little has been done at a manufacturer level to reduce the content of sugar and Na of many breakfast cereals.

We agree with Ms Levy that there have been new products and brands entering the breakfast cereal category during the last 5 years, and suggest a more robust study should be undertaken to evaluate whether existing products have had their nutritional profiles altered in recent years in an attempt to align with suggested New Zealand RDI.


Interestingly, Ms Levy focused on one particular in-house cereal example (Nutri-Grain®) and reported that the manufacturer (Kellogg’s) has reduced the sugar content of this product.

However, the data sourced to support this claim are from an Australian website and no evidence has been provided that this change has also occurred in the New Zealand product.

Furthermore, Nutri-Grain has previously been criticised by the Obesity Policy Coalition for its level of sugar, being the highest of ten children’s cereals surveyed [14], suggesting that this one product alone may have been targeted for reformulation.

The reformulated Nutri-Grain product discussed in the letter still contains 26.7% sugar, so the reformulation has not fully redressed the issue of excessive sugar in the cereal.

Further, Ms Levy did not provide any information that other cereals had been reformulated in recent years (either Kellogg’s or any other brand) and a search of both the Internet and published literature could not identify any other information or press releases about cereal companies in New Zealand reducing their sugar content.


We would like to acknowledge that we misrepresented that there is no regulation of food labelling in New Zealand and thank Ms Levy for clarifying this point.

However, the emphasis of this statement in the original publication was on the lack of regulation of product formulation and promotion of breakfast cereals.

Currently, there is little control of the nutritional quality of breakfast cereals in New Zealand and this appears to be largely manufacturer-driven in response to customer demand.

However, it is our belief that manufacturers should take some responsibility for the provision of healthier breakfast cereals (including cereals with a lower sugar content), particularly as the published literature demonstrates that consumers will still readily purchase and consume such products.


The precautionary approach has become a cornerstone of an improved approach to fisheries management (FAO, 1996; Richards and Maguire, 1998).

The application of precaution is founded on two principles:


 that decision-making should be more risk averse (i.e. precautionary) when uncertainty is higher than usual (where "usual" is relative, and may apply to assessments of a single stock over time, or across many stocks for a jurisdiction); and  that decision-making should be more risk averse when plausibly bad outcomes involve serious or irreversible harm.


Both principles sound reasonable, and they require fisheries science to develop new or adapt old analytical tools to make them operational.


One class of new or adapted tools are re-sampling methods that are designed to capture uncertainty more fully in the assessment process, so science advisors and decision-makers actually know how uncertain the information base for a decision is (e.g. Patterson et al., 2001; Berkson et al., 2002), and what options really are risk averse.

The other class of tools for implementation of the Precautionary Approach are simulation methods that allow the application of pre-agreed decision-rules (FAO, 1996; Garcia, 2005), whose performance characteristics have been studied through simulation and use in fisheries management (Stokes et al., 1999; Punt et al., 2001; Butterworth and Punt, 2003).

The combination of improved quantification of uncertainty and simulation-based management strategy evaluations can feed into processes for risk evaluation and decision analysis (Peterman, 2004) that are more formally structured than historical practices.

However, the reliability of the more formal risk evaluation and decision analysis depends critically on the reliability of both types of tools.

Their reliability depends in turn on the quantitative results of the re- sampling and simulations.


Without question these tools improve practice compared to ignoring uncertainty and applying ad hoc decision-making.

Unfortunately, they also create a new and subtle class of challenges to practice.

Moreover, in an attempt to increase transparency through making more assumptions in the analytic work of fisheries scientists explicit, opportunities are presented to substitute complex mathematical formulations for an admission that the experts don't really know what is going on.

The methods need to be looked at critically, with regard to both the opportunities they present for advances in practice and the new pitfalls against which practitioners must be vigilant.


The simulations consist in first building an operating model, assuming mathematical equations and parameter values for all processes from the ecology and dynamics of the resource to the dynamics of the exploiting fleets, their catch, and production.

In addition, an observation model represents the scientific observation of the system, from data collection to stock assessment.

The management procedure model includes this observation model and management decision rules.

Then, simulations are used to explore the consequences of various management strategies.

Commonly, these are then repeated for several variants of the operating model, reflecting different assumptions about key processes in the stock or fishery. .

The foundations of this approach are not novel, as models have long been used to represent knowledge, and mathematics and computers to explore and understand the consistency and hidden content of this knowledge.

What is new is the way these methods handle uncertainty.

This issue is addressed by replacing deterministic parameters with stochastic (or otherwise probabilistic) ones and a single functional relationship of interest with suites of alternative formulations of the relationships, often in various combinations.

To address parameter uncertainty, variables and parameters that are inherently variable or difficult to measure or estimate with accuracy and precision are replaced by specified probability distributions.

These are supposed to include the full variability and sampling distribution of the property which the parameter represents, not a single most likely or otherwise "expected" value.

To address model uncertainty, processes that are not well understood are represented by arrays of alternative assumptions and corresponding functional relationships.

These are intended to cover the range of processes that might underlie observed patterns, and/or the range of possible future states of nature.


Uncertainty about multiple processes requires multiple arrays of assumptions and relationships, so complexity increases multiplicatively.

These approaches then "capture" the full uncertainty by Monte-Carlo or other probabilistic simulations, drawing numbers from the probability distributions of parameters, and running multiple scenarios using all combinations of functional relationships.

Decision rules are then tested for robustness of performance in the face of the results produced by the simulations.

Robustness may be tested by combining the results of a full suite of related simulations probabilistically into a combined probability density function (pdf) of possible outcomes, and evaluating risk of an undesirable outcome from each management option.

Commonly the simulations are used to estimate the probability of violating a management benchmark such as B pa or B lim or a proxy like 20% virgin biomass (Butterworth and Punt, 1999; ICES 2005b), or the probability of stock collapse (Hilborn, 1997).

Alternatively, if managers have adopted a particular level of risk aversion to an event such as decline in SSB, then the simulations can estimate the maximum harvest that would comply with the risk tolerance (e.g. Fig 9 CSAS, 2006).

In both contexts the decision support from such stochastic simulations must be assumed to have substantial quantitative detail behind it, particularly with regard to the likelihood of uncommon events.

The multiple formulation - stochastic sampling approach also addresses the issue of estimating the severity of "plausibly serious harm", by combining the most extreme credible assumptions about processes with the most extreme credible values for parameters.

All this looks extremely quantitative and rigorous, but it is in the appearance of rigour that new challenges and pitfalls are found, because of how models may be used to represent things that are not known.


Using alternative model formulations to "cover" the range of possible realities seems a good way of acknowledging uncertainty.

However, if the undesirable outcomes have not been experienced enough times to know the conditions that cause them, and particularly if undesirable events in the past appear to have had more than one cause (Hodgson et al., 2006; Mueter et al., 2007), these formulations should be viewed with scepticism, and very rarely should be considered to bracket the range of possible functional relationships.

Predictions based on phenomenological models with a similar fit to data can be qualitatively completely different; this is a general result for models that are not entirely mechanistic (Wood and Thomas, 1999).

Unfortunately, there is no agreed method to select the formulations to be included in the simulated array, nor to assign them weights or probabilities; yet these weights may be highly influential on the conclusions regarding plausible worse-case outcomes.

Reviews such as Butterworth and Punt (1999) and Peterman (2004) have not solved any of the problems, but at least a critical assessment of their implications and associated risks is available to practitioners and users.

A similar critical assessment of the challenges and risks posed by using probability distributions rather than poorly specified parameters has not been undertaken.

We argue that stochastic modelling of uncertainty is a paradoxical and at least potentially misleading approach, also producing potentially unreliable foundations for risk averse decision-making.


Two considerations mean that the detail of the quantitative support for risk-averse decisions could be illusionary.

First, the information supporting risk-averse decision is primarily the pattern in the tails of the simulated distributions, whose shapes are largely driven by assumptions regarding the variance and skewness of the sampling distribution of the parameters.

When these have not been estimated carefully the quantitative detail in the simulations lacks exactly the rigorous scientific backing that the detail invites decision- makers to infer is present in the decision support.

Second, the support depends on the relative plausibility assumed for extreme events, often arising from the extreme model formulations.

When MSEs are used to evaluate the risk of serious or irreversible harm, except for stocks that did already collapse enough times that the causes are understood, the estimates rely on occurrences of events that were seldom or never observed.

These events often correspond to the less well-studied hypotheses and hence are least well known and least likely to be formulated accurately and precisely.


This matters because when precautionary considerations apply, risk-averse decision-making is needed.

For example in the International Council for the Exploration of the Seas (ICES) fisheries advice, B lim is the biologically defined conservation boundary, below which there is unacceptable risk of diminished stock productivity (ICES, 2005a).

B pa is used as the quantitative benchmark for harvest advice, however, serving as a risk control tool intended to ensure that the probability that true SSB is below B lim is small, given the uncertainties in the assessment.

This approach to positioning B pa relative to B lim makes advice and decisions sensitive to the shape of the tails of the parameter distribution(s) for B (and F), and less dependent on the much better specified centre of the distribution.


The use of complex mathematics and statistical tools risks giving users a false sense of rigour, whatever their relationships with the underlying knowledge.

Users should not expect probabilistic advice on unlikely or extreme events to have the same accuracy and precision as probabilistic advice about events closer to the centre of the simulated distributions, yet precautionary decision support does exactly that.

As with the concerns about multiple model formulations, we can offer no "cure" for these problems.

However we can illustrate the scale of the problems that are presented.

We also suggest that MSEs be augmented by a greater attention to alternative methods to evaluate risks or management strategies, including qualitative methods and empirical post-hoc analyses.


When a probability distribution is used because there is insufficient information (data and/or understanding) to specify the expected value for a parameter, the substitution of a probability distribution paradoxically implies knowing more about the characteristics of the parameter than using a point estimate.


To use a deterministic parameter implies knowledge of the value of the parameter.

To use the distribution of a parameter acknowledges that this value is a mean and requires additional knowledge of either its variance and skewness, or of a theoretical distribution and its parameters.


Alternatively, non parametric data-based methods like boot-strapping or density estimation can be used to avoid specifying a theoretical sampling error distribution for the simulation, or the specification of a parameterized functional relationship (e.g. Evans and Rice, 1988; Rice, 1993).

These approaches are limited to data-rich situations, and place the problem of dealing with uncertainty from infrequent events or inherently variable measurements even more centrally in the analyses.


If there is insufficient information to estimate the mean of a distribution reliably, there is little reason to expect to be able to estimate its variance.

This can be illustrated with any biological data set, for example from research survey data.


Attempts to avoid the estimation of the variance or skewness of a set of observations by fitting data to a theoretical distribution does not help.

Survey abundance data have extremely low statistical power to differentiate among common theoretical distributions such as Gamma, Normal and log-Normal (Myers and Pepin, 1990; Trenkel and Rochet, 2003), and the same is true for estimates of Spawning Stock Biomass (Figure 2).


For all stocks examined, depending on which assumption was made about the theoretical distribution from which the SSB estimates were drawn, even advice based on the modal range of estimates could be substantially different (Figure 2).

The differences in likelihood are very small, illustrating the low power to choose among the distributions, yet the probabilities of low abundances calculated from these theoretical distributions are quite different (Table 1).


A typical MSE work would of course not use a specified distribution for SSB but rather for 'true' population numbers and then go through some deterministic model (e.g. VPA) to estimate SSB and draw inferences from the percentiles of this Monte Carlo SSB series.

This may amplify the problem we outline as i) true population numbers cannot be observed at all, thus their distribution is still less well known, and ii) errors propagate through the calculations in ways that are not easy to quantify (Pelletier and Gros, 1991).


Research survey data are often a "best case" scenario for fisheries modelling, because there can be several hundred observations to help tie down the shape of the tails of the distributions.

When few data are available, common practice is that because a log-normal distribution cannot be rejected, it can be used in simulations.

In such cases even the estimates of standard deviation for the distribution can be based on a median or central value from a literature review (e.g., Punt, 1997), rather than stock-specific data.

Hence many simulations rely on parameter distributions of which neither the shape, nor the dispersion are really known.


Use of data-based boot-strap methods are not a real alternative to provide reliable quantitative support for risk-averse decision-making when there is limited contrast in the data available, when few data come from the area of interest for the management decision - (e.g., for stocks which have not yet been in the neighbourhood of B lim .), or when there might have been different causes of past stock declines.

When using resampling procedures with replacement, and weighting each sampled observation equally, the uncertainty in the region where observations are most clustered dominates the bootstrapped uncertainty, as several samples will be drawn from the region where observations are most dense (often close to the centre of the distribution) for each time a sample is drawn from the rarer observations in the tails.

As a consequence, the frequency of resampled units is increasingly biased to lie below the true percentile in the observed sample for lower percentiles (Figure 3).

That is, bootstrapping underestimates the probabilities of low values, thus also underestimating risks of undesirable outcomes.

If the bootstrap selection of observations is designed to sample preferentially in the region of interest (in the case of precautionary management, in the location where B lim is thought to lie), for stocks that have not collapsed, the probabilities are based on much less information than using the bootstrap methods to estimate probabilities of more commonly observed stock sizes.


The same type of problems arises when non-parametric density estimation methods are used to avoid specifying a functional form for poorly known relationships.

When locally weighted smoothing methods are applied near the extremes of the range of observations, the few observations of the extreme events dominate the estimated probability distribution.

Gaps between observations near the extremes of the range that are due to low sampling in the tails are treated as genuine gaps in the observations that are possible, unless very wide smoothing windows are used, and then the probabilities of the events in the tails become largely determined by the more common observations near the centre of the distribution of observations.

Once decision support requires estimating the probability of events beyond the range of historic observations, the situation becomes even more subjective.

The estimates of probabilities are increasingly determined by the assumption of what value the property converges to at zero or infinity on the independent variable.

That assumption could be convergence to the overall mean, to zero, or a linear trend from the last observations - and not any of the observations (Silverman, 1992).

Hence data-based methods for estimating both expected values and their uncertainty can also provide a spurious sense of quantitative rigour when supporting decisions about the likelihood of extreme or otherwise rare events - a type of support commonly requested by managers.


Several other approaches have been or could be used to evaluate management strategies and usefully complement simulation-based methods.

Here we scan some of them with a view to illustrating various avenues rather than offering an exhaustive review or a preferred choice.


First of all one can learn from experience and examine performance of management strategies, or of some of their pieces, based on how they succeeded or not, and the reasons for their success or failure (e.g., Hilborn, 2006).

Because estimates by some stock assessment methods are more reliable at the beginning of the data time series than in the most recent years, the performance of these methods at forecasting catch can be retrospectively examined (Jónsson and Hjörleifsson, 2000; Reeves and Pastoors, 2007).

Retrospective analysis of the performance of the subsequent advice in terms of relevance of the advised action against true status of the stock is also informative (Piet and Rice, 2004).

Analysis of time trends in indicators can also reveal performance of a management strategy.

For example, Sparholt et al. ( 2007) showed that the demersal stocks evaluated by ICES have continuously declined since the 1950's and that neither the Common Fisheries Policy introduced in 1983 nor the Precautionary Approach implemented by the late 1990s had a detectable effect on trends in fishing mortality nor stock biomasses.


Retrospective examination of management strategies performance will be much more powerful if done in a comparative approach, that is, comparing different management strategies across regions or fisheries or stocks (Patterson and Résimont, 2007; Dankel et al., 2008) or across periods with different biological or management regimes for a given stock (Simmonds, 2007).

When the outcome of a particular strategy for the management of a particular stock is examined, the reasons for success or failure might be diverse and confounded.

Bringing together outcomes of different strategies implemented in different settings, and applying explicit criteria to define successes and failures, the method will point to the characteristics of those management strategies which worked, against those of the strategies which failed.

Much can be learned about selection of cases and traits and appropriate comparison methods from the comparative method which has been developed and widely used in evolutionary biology (Harvey and Pagel, 1991).


As the experience with formalised management procedures increases, these retrospective empirical methods will be increasingly informative. But even before any implementation, the consistency of a proposed method can be checked.

One type of consistency check is simply the internal consistency of what is said and what is done.

If a management strategy is not internally consistent, it may be more prone to failure than another consistent one.

For example, Hauge et al. ( 2007) compared the definitions of limit and precautionary reference points in the ICES Precautionary Approach, and the way they are used in practice, and found that their apparent transparency is compromised by the difficulty of estimating them in a standardised way across a diversity of stocks.


Another consistency check consists in using formal mathematical analysis to test if a rule really provides the expected results under ideal conditions.

For example, De Lara et al. ( 2007) showed by analysing a dynamic stock model that in ICES Precautionary Approach, for most stocks the objective of keeping spawning stock biomass above B lim cannot be guaranteed on the long term just by keeping SSB above a limit from year to year: additional indicators of stock structure are required.


Qualitative modelling (e.g., Eisenack and Kropp, 2001) provides an attractive way of modelling vague knowledge to provide decision support.

Rather than attempting to provide arrays of formulations and parameter distributions for processes which are not well known, this approach allows a formal analysis of the consequences of qualitative assumptions, such as 'catch is a dome-shaped function of effort' or 'effort increased during the 1960-1990 period'.

A combination of formal derivation and "qualitative simulation" yields system trajectories that can be categorized in groups like 'stabilization at a low level' or 'sustainable use' depending on these qualitative assumptions.


Similarly, qualitative analysis of dynamic models e.g., loop analysis and shifts in equilibrium caused by press perturbations, allows to predict the direction (not amount) of change in some variables like stock abundance or catch induced by changes in parameters or other variables like effort or price (Dambacher et al., 2002; Dambacher et al., 2003).

For example, in a multifleet multi-species fishery targeting two trophic levels, decreasing catchability of piscivores will lead to an increase in their abundance, whether there is omnivory or technical interactions, or not, implying that a number of complex science questions about those interactions do not have to be resolved before the right type of management action can be selected (Dambacher et al., In Press).

This can obviously be extended to examine which directions of changes will be robust irrespective of model structure; when decreasing catch or increasing taxes is likely to have the expected consequences, before undertaking a costly model parameterization and simulation work.


Without question the MSE approach is a significant step forward in fisheries science.

It has all the benefits of any modelling exercise and provides a tool to help make the precautionary approach operational.

However we are seeing it being used as if small differences in analytical results are meaningful biologically and in management.


For the reasons we illustrate above, it is inherently impossible for MSE (at least with current knowledge of marine ecological processes, quality of data, and conflicts over implementation) to provide the level of quantitative accuracy and precision needed to support making finely differentiated management decisions.

For example, if the goal is to rebuild a depleted stock to a pre- selected biomass level in 10 years, when knowledge of stock productivity at low biomass is highly uncertain, MSE cannot be used to reliably estimate that the goal can be reached with a particular F, but not with an F, e.g., 20% higher.

Simulations can provide results which present themselves as having that level of precision, but it is a deception.

The implied precision may not deceive those performing the evaluations, but it is often a deception that users of the results at least may exploit, whether they are believed or not.


Our alternative is to first of all get a general admission that it is unrealistic for managers and policy experts to expect reliable, finely nuanced discriminations from simulations.

Fisheries scientists have spent 15 years admitting first to themselves and then trying to convince managers and policy makers that they cannot provide science support for annually adjusting Fs by 10% and TACs by a few thousand tonnes.

We do not want to start building a culture where either producers or users of MSEs think our tools can support discriminations of comparable precision when the analyses have to include even more processes and data sets - many more poorly specified than in traditional single-species assessments estimating only the most likely B and F each year.


In other fields where risk analysis is well developed like nuclear power safety or cost analysis of regulatory intervention, risk estimates are sometimes reported with an excessive precision that is not supported by the available data, and can lead to rankings of options which are not robust.

Unfortunately overly precise figures are picked up by users because they give confidence in the analyst's knowledge (Hassenzahl, 2006).

A useful practice might be to report with any MSE result the number of parameters that were actually estimated from data relevant to the situation, the number which were borrowed from the literature, and the number which were guessed or tuned.

This could be done separately for the parameters which determine the system state and dynamics, and for those that characterize the uncertainty.

Similarly practice could include reporting comparable information about the sources and strength of support for the functional relationships that were included, relative to alternatives that were not.

This would provide users with some appraisal of the reliability of the risk estimates.


There are valuable uses of simulation-based MSEs, such as eliminating 'bad' management strategies from consideration.

A MS that performs poorly in the full range of simulations is not likely to perform well in the real world, because the real world is always more complex and unpredictable than the simulated world.

However, this does not mean that MSEs are necessarily powerful at discriminating the best approach among ones that are not bad.

Simple, adaptive MSs may not be optimal in the face of the particular forms of uncertainties/changes that were incorporated in the operating model; but in the real world they might address better the "unknown unknowns": unforeseen changes in ecosystem factors and/or in the economy, reactions of fishers to new management plans, etc.

Overall, MSEs should be used to strengthen the use of intelligence in developing approaches to fisheries management, not as an excuse to remove it from the process.


Few operating models can simulate the act of scientists, managers, and fishers making choices in each year of an ongoing cyclic process of fishery - assessment - advice - management plan - fishery.

For example, Punt (1997) in his evaluation of VPA-based management, uses the Laurec-Shepherd estimation procedure rather than ADAPT, because the latter requires making educated choices that cannot be simulated in an operating model.

Moreover, in many cases choices of one player can be influenced by conjectures of what choices other players may make (Rice and Richards, 1996), and this might be difficult to simulate.


Particularly because another key change going on in fisheries management is making the governance process more inclusive (FAO, 2003; DFO, 2004; European Union, 2004), this is the wrong time to remove intelligence from the procedure used to evaluate the process.

If all their quantitative details are to be taken as reliable, then simulation-based MSEs can only evaluate impoverished procedures; ones without the need of human intervention.

This is not the kind of management procedures we want.


Bayesian approaches (Punt and Hilborn, 1997) amplify all our concerns about treating quantitative details as more meaningful than deserved by the quality and quantity of either the data or the theory on which the details depend.

Bayesian approaches are becoming increasingly popular in many fisheries applications, in part because they allow greater use of the information that is available, differentially emphasising data sets based on quantitative measures of their information content.

However, they are still vulnerable to the concerns that we raise here: model misspecification and underestimated errors are pervasive in Bayesian analysis and lead to false precision estimates (Small and Fishbeck, 1999).

More data and sophisticated techniques can improve this problem but not solve it.

Instead, MSEs should not oversell their ability to differentiate among strategies that perform well enough to be considered relevant to management decisions.


It has long been argued that a major benefit of MSEs is not technical development and quantitative output, but greater opportunity for stakeholder participation and acceptance (Cochrane et al., 1998; Smith et al., 1999).

The process of developing an MSE forces participants to agree about management objectives, and to think about the various components of the system, unknowns that might influence the success of a MS, appropriate criteria to evaluate an MS, and how to weight the various considerations.


Agreeing completely with the value of such careful thinking about the management problem, and wide engagement of parties in the process of developing and implementing MSs, we notice that few MSE publications are reporting how users were involved in the process (but see Cochrane et al., 1998; Pastoors et al., 2007).

On the other side of the coin, we contend that progress on those fronts does not require that everything be translated into equations and distributions for this progress to be made.

For example qualitative risk evaluation (Fletcher, 2005) basically does the same job in a transparent and shared way.


Simulations have important roles in supporting management decision-making.

These include finding out what really does not work under any plausible circumstances and avoiding it, and providing a process to define the management problem comprehensively and consolidate knowledge as much as possible.

The quantitative results certainly serve as one source of information to support decision-making.

However, they are not a substitute for applying our full intelligence in the decision support.

The simulations might really tell little more than the general magnitude of the changes to be expected from a particular set of actions and the qualitative scale of the management intervention that is needed.

Then it is necessary to bring in all the other sources of knowledge to first get confirmation that the scale of intervention really is needed, and second to get some insight into which actions are most likely to succeed in making the intervention large enough to achieve the management goals.

We know from experience that implementation uncertainty is very likely to greatly exceed any nuanced quantitative differences among simulation results.

Therefore processes that are simple and adaptive, but have high compliance are likely to be appropriate most of the time, however well structured processes may perform in MSEs.


Rochet and Rice, while recognizing management strategy evaluation (MSE) as an important step forward in ﬁsheries management, level a number of criticisms at its implementation.

Some of their points are sound, such as the need for care in representing uncertainties and for thorough documentation of the process.

However, others evidence important misunderstandings.

Although the difﬁculties in estimating tail probabilities and risks, as discussed by Rochet and Rice, are well known, their arguments that Efron’s non-parametric bootstrap re-sampling method underestimates the probabilities of low values are ﬂawed.

In any case, though, the focus of MSEs is primarily on comparing performance and robustness across alternative management procedures (MPs), rather than on estimating absolute levels of risk.

Qualitative methods can augment MSE, but their limitations also need to be recognized.

Intelligence certainly needs to play a role in ﬁsheries management, but not at the level of tinkering in the provision of annual advice, which Rochet and Rice apparently advocate, inter alia because this runs the risk of advice following noise rather than signal.

Instead, intelligence should come into play in the exercise of oversight through the process of multiannual reviews of MSE and associated MPs.

A number of examples are given of the process of interaction with stakeholders which should characterize MSE.