Article
Effect of Botulinum Toxin on Non-Motor Symptoms in
Cervical Dystonia
Matteo Costanzo 1,†, Daniele Belvisi 1,2,†, Isabella Berardelli 3, Annalisa Maraone 1, Viola Baione 1,
Gina Ferrazzano 1, Carolina Cutrona 1, Giorgio Leodori 1,2, Massimo Pasquini 1, Antonella Conte 1,2,
Giovanni Fabbrini 1,2, Giovanni Defazio 4 and Alfredo Berardelli 1,2,*
1
Department of Human Neurosciences, Sapienza University of Rome, Viale dell’Università 30, 00185 Rome,
Italy; matteo.costanzo@uniroma1.it (M.C.); daniele.belvisi@uniroma1.it (D.B.);
annalisa.maraone@uniroma1.it (A.M.); viola.baione@uniroma1.it (V.B.); gina.ferrazzano@uniroma1.it (G.F.);
carolina.cutrona@uniroma1.it (C.C.); giorgio.leodori@uniroma1.it (G.L.);
massimo.pasquini@uniroma1.it (M.P.); antonella.conte@uniroma1.it (A.C.);
giovanni.fabbrini@uniroma1.it (G.F.)
2
IRCSS Neuromed, Via Atinense 18, 86077 Pozzilli, IS, Italy
Department of Neurosciences, Mental Health and Sensory Organs, Faculty of Medicine and Psychology,
Suicide Prevention Centre, Sant’Andrea Hospital, Sapienza University of Rome,
Via di Grottarossa 1035-1039, 00185 Rome, Italy; isabella.berardelli@uniroma1.it
4 Department of Medical Sciences and Public Health, University of Cagliari, SS 554 Bivio Sestu,
09042 Monserrato, CA, Italy; giovanni.defazio@unica.it
* Correspondence: alfredo.berardelli@uniroma1.it
† Contributed equally.
3
Citation: Costanzo, M.; Belvisi, D.;
Berardelli, I.; Maraone, A.;
Baione, V.; Ferrazzano, G.;
Cutrona, C.; Leodori, G.;
Pasquini, M.; Conte, A.; et al. Effect
of Botulinum Toxin on Non-Motor
Symptoms in Cervical Dystonia.
Toxins 2021, 13, 647. https://doi.org/
10.3390/toxins13090647
Received: 18 August 2021
Accepted: 9 September 2021
Published: 12 September 2021
Publisher’s
Note:
MDPI
stays
neutral with regard to jurisdictional
claims
in
published
maps
Abstract: Patients with cervical dystonia (CD) may display non-motor symptoms, including
psychiatric disturbances, pain, and sleep disorders. Intramuscular injection of botulinum toxin type
A (BoNT-A) is the most efficacious treatment for motor symptoms in CD, but little is known about
its effects on non-motor manifestations. The aim of the present study was to longitudinally assess
BoNT-A’s effects on CD non-motor symptoms and to investigate the relationship between BoNTA-induced motor and non-motor changes. Forty-five patients with CD participated in the study.
Patients underwent a clinical assessment that included the administration of standardized clinical
scales assessing dystonic symptoms, psychiatric disturbances, pain, sleep disturbances, and
disability. Clinical assessment was performed before and one and three months after BoNT-A
injection. BoNT-A induced a significant improvement in dystonic symptoms, as well as in
psychiatric disturbances, pain, and disability. Conversely, sleep disorders were unaffected by
BoNT-A treatment. Motor and non-motor BoNT-A-induced changes showed a similar time course,
but motor improvement did not correlate with non-motor changes after BoNT-A. Non-motor
symptom changes after BoNT-A treatment are a complex phenomenon and are at least partially
independent from motor symptom improvement.
and
institutional affiliations.
Keywords: cervical dystonia; non-motor symptoms; botulinum toxin; psychiatric disorders; pain
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
Key Contribution: In this longitudinal study, we observed that botulinum toxin intramuscular
injection improves pain; feelings of anxiety and depressive symptoms in patients with cervical
dystonia. Our observations contribute to shedding light on the possible role of botulinum toxin in
the management of non-motor symptoms in patients affected by cervical dystonia.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution
(CC
BY)
license
(http://creativecommons.org/licenses
/by/4.0/).
1. Introduction
Cervical dystonia (CD), the most common form of idiopathic focal dystonia [1,2], is
characterized by involuntary muscle contractions of the cervical region causing abnormal
movements and postures of the neck and head and tremor [3–8].
Toxins 2021, 13, 647. https://doi.org/10.3390/toxins13090647
www.mdpi.com/journal/toxins
Toxins 2021, 13, 647
2 of 12
Although CD has long been considered mainly a motor disorder, in recent decades,
increased attention has been focused on the possible presence of non-motor symptoms
(NMS) throughout the disease course. In CD, the most frequent NMS are psychiatric
symptoms [9–13], as well as sleep habit changes [14–16] and pain [17,18]. Growing evidence
suggests that NMS play a pivotal role as determinants of quality of life and disability in
CD patients [19–21], and a recent cross-sectional study has demonstrated that NMS might
also intervene in determining clinical heterogeneity [22]. Finally, the relationship between
motor and NMS is still a matter of debate. Most studies have reported that NMS are
independent of CD motor manifestations [13,20,22], but it has also been suggested that
NMS may represent a secondary manifestation of motor burden [23].
Intramuscular injection of botulinum toxin type A (BoNT-A) is currently considered
the treatment of choice for CD since it is well-tolerated and effective in determining
dystonia improvement [24,25]. BoNT-A is known to inhibit acetylcholine release from
neuromuscular junctions, resulting in biochemical denervation of the treated muscle
[26,27]. In addition to a primary peripheral clinical site of action, evidence also suggests
that pharmacological properties of BoNT-A include central effects [27–29], including the
modulation of glutamate, noradrenaline, dopamine, and glycine transmission and
changes in the electrophysiologic and morphologic properties of central neurons [30]. A
few studies have investigated the effect of BoNT-A on NMS in CD, though they focused
on only one non-motor domain [31,32].
In the present study, we aimed to investigate the possible effect of BoNT-A treatment
on non-motor burden, including psychiatric symptoms, pain, and sleep disorders, in CD
patients. A further aim was to evaluate the relationship between BoNT-A-induced
changes in motor manifestations and the effect of BoNT-A on NMS. For these purposes,
we performed a longitudinal evaluation in a cohort of 45 CD patients. We tested motor
and NMS at three different timepoints—before BoNT-A injection and one and three
months after BoNT-A treatment.
2. Results
Demographic and clinical features of patients with CD are reported in Table 1.
Table 1. Demographic and clinical features of patients with CD at baseline.
Demographic and clinical data
Females (%)
Age in years (mean ± SD)
Disease duration in years (mean ± SD)
Schooling (years ± SD)
Right-handed (%)
Motor domain
Tremor (%)
TWSTRS severity Section 1 (mean ± SD)
Psychiatric domain
TWSTRS psychiatric Section 1 (mean ± SD)
HAM-A 2 (mean ± SD)
HAM-D 3 (mean ± SD)
Sleep domain
PSQI 4 (mean ± SD)
ESS 5(mean ± SD)
Pain domain
TWSTRS pain Section 1 (mean ± SD)
Disability domain
CD Patients
62%
58.5 ± 12.8
10 ± 9.2
12.3 ± 3.5
97%
31%
11.5 ± 3.4
5 ± 4.2
9 ± 7.8
8.3 ± 7.2
5.1 ± 3.7
4.1 ± 4.1
14.6 ± 10.2
Toxins 2021, 13, 647
3 of 12
TWSTRS disability Section 1 (mean ± SD)
IPDS 6 (mean ± SD)
6.8 ± 5.5
31.5 ± 16.7
TWSTRS: Toronto Western Spasmodic Torticollis Rating Scale. 2 HAM-A: Hamilton Anxiety
Rating Scale. 3 HAM-D: Hamilton Depression Rating Scale. 4 PSQI: Pittsburgh Sleep Quality Index.
5 ESS: Epworth Sleepiness Scale. 6 IPDS: Italian Perceived Disability Scale.
1
2.1. BoNT-A Treatment Parameters
Our cohort included 43 patients who were already on treatment with BoNT-A and 2
de novo patients. The mean duration of treatment was 10.5 ± 10.7 years and the number
of treatment cycles was 38.6 ± 39. At baseline evaluation, 21 subjects received aboBoNTA, 18 subjects received onaBoNT-A, and 6 subjects received incoBoNT-A. The mean total
dose of abobotulinumtoxinA was 445.8 ± 192.2 units, and the mean total doses of
incobotulinumtoxinA and AonabotulinumtoxinA were 90 ± 25.3 units and 121.8 ± 35.2
units, respectively. The most frequently injected muscle groups at baseline were the
splenius capitis, sternocleidomastoid, trapezius, and levator scapulae. No EMG or
ultrasound assessment was performed.
2.2. Effects of BoNT-A on Motor and Non-Motor Symptoms in Patients with CD
The one-month follow-up evaluation showed that BoNT-A treatment induced a
significant improvement in motor symptom severity, as shown by the significant
reduction in TWSTRS motor section score (Z = −5.46; p < 0.00001) (Figure 1). Similarly, the
TWSTRS pain section score was significantly lower one month after BoNT-A injection (Z
= −4.91; p < 0.00001) (Figure 1). Regarding psychiatric disturbances, we observed that
BoNT-A significantly reduced HAM-A (Z = −3.86; p = 0.0001) and HAM-D scores (Z = −3.1;
p = 0.001) (Figure 2) but left the TWSTRS psychiatric section score unchanged (Z = −1.75;
p= 0.07). Four of the 45 patients included in the study had a known diagnosis of depression
and were on antidepressant drugs. BoNT-A did not modify sleep disorders, as tested by
PSQI score (Z = −1.85; p = 0.06), or excessive daily sleepiness, as tested by ESS score (Z =
−0.13; p = 0.8). The TWSTRS dystonia-related disability score significantly improved after
BoNT-A (Z = −3.91; p < 0.0001), but global perceived disability, as tested by IPDS score,
was unchanged after BoNT-A (Z = −0.05; p= 0.95). At the three month follow up evaluation,
we observed a slighter but still significant improvement in TWSTRS motor section (Z =
−3.59; p = 0.0003), TWSTRS pain section (Z = −3.94; p = 0.0001), HAM A (Z = −3.28; p = 0.001)
and HAM-D scores (Z = −3-05; p = 0.002) (Figure 1).
Toxins 2021, 13, 647
4 of 12
Figure 1. BoNT-A-induced changes at one- and three-month evaluations in motor and non-motor
symptoms in CD patients. Error bars denote standard error. (TWSTRS: Toronto Western Spasmodic Torticollis Rating Scale; HAM-A: Hamilton Anxiety Rating Scale; HAM-D: Hamilton Depression Rating Scale; PSQI: Pittsburg sleep quality index; ESS: Epworth Sleepiness Scale).
Figure 2. Correlation between BoNT-A-induced changes on Hamilton anxiety rating scale (expressed as the “HAM-A total score at one month/baseline total score *100”) and Hamilton depression rating scale scores (expressed as the “HAM-D total score at 1 month/baseline total score
*100”).
2.3. Relationship between Motor and Non-Motor BoNT-A-Induced Changes
We analyzed the possible correlations between clinical variables that were significantly modified by BoNT-A treatment. These included TWSTRS motor section, TWSTRS
Toxins 2021, 13, 647
5 of 12
pain section, HAM A and HAM-D scores. Spearman’s correlation coefficient showed no
significant correlations between motor and non-motor BoNT-A-induced changes (all expressed as the ratio between one month and baseline clinical scale scores) (all p > 0.05).
Conversely, we observed a significant correlation between HAM-A and HAM-D score
changes one month after BoNT-A (r = 0.48; p = 0.001) (Figure 2). In addition, we did not
observe any correlation between psychiatric and pain scores (all p > 0.05).
Given that TWSTRS dystonia-related disability scores were also significantly
changed by BoNT-A, we investigated possible correlations between this variable and motor and non-motor variables. We observed that disability BoNT-A-induced changes did
not correlate with motor changes but significantly correlated with HAM-D score after
BoNT-A changes (r = 0.38; p = 0.01) (Figure 3). No significant correlations between pain
score and other non-motor symptoms scores were found (all p > 0.05). Similarly, we did
not observe any significant correlation between site of injection and BoNT-A-induced effects on motor and non-motor symptoms.
Figure 3. Correlation between BoNT-A-induced changes on Hamilton Depression Rating Scale
scores (expressed as the “HAM-D total score at one month/baseline total score *100”) and TWSTRS
dystonia-related disability score (expressed as the “TWSTRS disability total score at one
month/baseline total score *100”).
3. Discussion
In the present paper, we longitudinally evaluated the effect of BoNT-A intramuscular
injection on motor and non-motor manifestations in a cohort of 45 CD patients. Beyond
the expected improvement in motor symptom severity and dystonia-related disability, we
observed that BoNT-A intramuscular injection improved pain and psychiatric disturbances but not sleep disorders. We did not observe any association between motor and
non-motor symptom improvements after BoNT-A treatment. Finally, we found that the
reduction in dystonia-related disability after BoNT-A treatment was correlated with psychiatric disturbance changes but not with motor symptom severity reduction.
To exclude methodological biases that could have affected our results, we took several precautions. To minimize selection bias, we consecutively recruited CD patients who
were diagnosed according to international clinical diagnostic recommendations in a single
center setting [33,34], thus providing a case series resembling the general population. To
Toxins 2021, 13, 647
6 of 12
exclude any confounding effects due to previous BoNT injections, a baseline clinical assessment was performed at least four months after the last BoNT treatment. Finally, to
avoid observer bias, baseline and follow-up clinical examinations were performed by a
single neurologist expert in movement disorders who administered standardized scales
to evaluate motor and NMS. Similarly, psychiatric evaluation was performed by a single
psychiatrist expert in psychiatric conditions in patients with movement disorders. The
possibility that a previous history of depression might have affected our results is highly
unlikely since at the time of evaluation only four of 45 patients tested suffered from depression and were on antidepressant treatment.
In our study, we observed that BoNT-A injection was associated with a reduction in
anxiety and depressive symptom severity, as indicated by the decrement in HAM-A and
HAM-D scores, respectively. To our knowledge, no studies have specifically evaluated
the effect of BoNT-A on psychiatric symptoms in CD. One study demonstrated an improvement in anxiety and depression following BoNT-A intramuscular injection in another frequent form of focal dystonia, blepharospam (BSP) [35]. Differently from our
study, the authors did not investigate the extent of motor symptom improvement in BSP
and therefore it is not possible to establish whether psychiatric improvement paralleled
motor improvement or not. The most intuitive explanation to interpret BoNT-A-induced
changes on psychiatric measures in CD is that the symptomatic improvement in motor
impairment, dystonia-related disability, and pain contributes to reducing feelings of depression and anxiety in these patients. In line with this hypothesis, we observed a similar
time course for motor and psychiatric symptom improvement after BoNT-A injection.
This is also consistent with findings by Jahanshahi and Marsden, who hypothesized that
the occurrence of depressive symptoms in CD is related to negative body image, secondary to abnormal postures of the head [36–38]. Contrary to the hypothesis that BoNT-A effects on psychiatric disturbances mainly depend on motor symptom improvement, we
did not find any correlation between psychiatric and motor domain changes after BoNTA injection. The mechanism linking motor and neuropsychiatric features is unclear, but it
has been suggested that CD is a network disorder in which motor and NMS depend on
the activity of independent but integrated nodes [22], including the limbic fronto-striatal
circuitry, which connects the anterior cingulate and orbitofrontal cortex with ventral striatal areas, the ventral pallidum, and the medial thalamus [39–41]. Recent imaging studies
have suggested that BoNT treatment determines a rebalancing of connectivity in the sensorimotor network in CD patients [41–43]. From this perspective, it is possible that improvement in psychiatric disturbances depends on a rebalance in connectivity between
motor and non-motor structures involved in the CD brain network [44–46]. Future neuroimaging and neurophysiological studies investigating functional connectivity between
brain network nodes underlying motor and psychiatric manifestations are, however,
needed to confirm this hypothesis.
In the present study, we also observed that BoNT-A treatment may temporarily relieve pain in CD patients, as shown by the significant reduction in TWSTRS pain section
score at the one-month evaluation, with a gradual return to baseline values at the threemonth assessment. The short- and long-term effects of BoNT-A in reducing pain in CD
patients have been consistently reported in previous studies [32,47,48]. It has been hypothesized that BoNT-A reduces pain via muscular relaxation with a reduction in painful
ischemia in hypercontracted muscles or, alternatively, via inhibition of neurogenic inflammation and peripheral sensitization (see Marciniec et al., 2019 for a review) [48]. The present observation that BoNT-A-induced motor improvement did not correlate with
changes in pain intensity suggests that the antinociceptive effect and muscle relaxation
mechanisms of BoNT-A could be distinct phenomena.
In addition, it is known that psychiatric disorders and pain strongly interact in patients affected by chronic conditions [49]. It is therefore possible that the improvement in
pain after BoNT-A treatment may partly depend on the improvement of anxiety and de-
Toxins 2021, 13, 647
7 of 12
pression observed in our patients and vice versa. Against this hypothesis we did not observe any correlation between pain and psychiatric symptom improvement after BoNT-A
treatment. This suggests that BoNT-A induces a NMS improvement in CD throughout
mechanisms that are at least partially distinct and symptom-specific.
Unlike what was observed for psychiatric disorders and pain, we did not observe
any effect of BoNT-A treatment on sleep quality and daytime sleepiness. Sleep habit disturbances represent conditions that are usually persistent or long-lasting and that are due
to several variables, including environmental factors, occupational factors, physiologic
changes, medical disorders, and psychiatric disorders. Due to the chronic nature of the
condition and to the large number of variables that influence this non-motor symptom, it
is possible that the short-term prospective design of our study did not allow us to obtain
reliable results. Future longitudinal studies may shed new light on this topic in CD.
Similar to previous studies [25,50], we found that BoNT-A intramuscular injection
determines a reduction in dystonia-related disability. We found that disability domain
scores did not correlate with motor symptom severity, but significantly correlated with
psychiatric domain scores. Most studies that have investigated the relationship between
motor burden and non-motor features reported similar results, indicating that neuropsychiatric disturbances may play a role in disability in CD [22,51]. It is therefore plausible
that disability in CD is a result of the contribution of motor and NMS, including pain and
neuropsychiatric features. This finding highlights the importance of a multidisciplinary
approach to dystonia management that uses treatment strategies aimed at reducing the
severity of motor and non-motor domains.
In order to assess NMS, we used a large number of clinical scales. It is important to
point out that we obtained different results when we administered clinical scales assessing
similar non-motor domains. For instance, we found that the TWSTRS psychiatric section
score was unchanged following BoNT-A treatment, whereas HAM-A and HAM-D scores
were significantly reduced. These contrasting results may be explained by the different
clinimetric properties of these assessment scales in capturing various aspects of anxiety
and depression. The TWSTRS psychiatric section is a screening tool used to rate several
categories of psychiatric disturbance that are associated with CD and identify psychiatric
disorders that may require further investigation and treatment [52,53]. HAM-A and HAMD, in contrast, are assessment tools designed to provide a more complete picture and to
rate the severity of general anxiety disorder and depression, respectively [54–56]. It is
therefore possible that HAM-A and HAM-D might be more suitable than the TWSTRS in
documenting the results of intervention programs, including pharmacotherapy or psychotherapy.
We acknowledge some limitations. The short-term prospective design of our study,
with three evaluations (baseline and one and three months after treatment) did not allow
us to observe BoNT-induced changes beyond this time. It is, however, likely that BoNTA may determine a long-term effect on non-motor symptoms in CD. Therefore, future
long-term studies simultaneously assessing BoNT-A’s effects on NMS will be necessary.
Similarly, we did not include a two-month time point because one-month and threemonth evaluations are more informative since they represent the peak and end phase of
BoNT-A therapeutic effects, respectively. The use of only the BoNT-A type of botulinum
toxin does not allow us to generalize the conclusions of this study to other types of botulinum toxins. Future studies directly comparing the effects of BoNT-A and BoNT-B treatment on non-motor symptoms in CD are warranted to clarify this aspect. Finally, we did
not include a placebo-treated group for ethical reasons, and we cannot fully exclude a
repetition bias during the administration of clinical scales. It is, however, important to
point that after BoNT-A treatment we observed a similar time course between motor
symptom severity, which was directly evaluated by the examinator, and NMS severity,
which was assessed by the administration of clinical scales.
4. Conclusions
Toxins 2021, 13, 647
8 of 12
In conclusion, our findings suggest that BoNT-A intramuscular injection in CD patients is able to improve non-motor variables (psychiatric symptoms and pain) in patients
affected by CD. Non-motor and motor symptom improvement after BoNT-A injection
show a similar time course. BoNT-A effect on CD NMS may reflect a global improvement
secondary to dystonic symptom reduction. On the other hand, the lack of correlation between motor and non-motor manifestation improvement after BoNT-A suggests that
other non-motor domain-specific mechanisms intervene in determining non-motor improvement in CD. Our study underscores the importance of the detection and treatment
of NMS in the management of CD patients and sheds light on the possible therapeutic role
of BoNT-A on NMS in patients affected by CD.
5. Materials and Methods
5.1. Study Participants and Ethics
Forty-five patients with primary CD were consecutively enrolled from among outpatients at the movement disorder clinic of the Department of Human Neurosciences, Sapienza University of Rome (Table 1). Patients were included in the study if they were
diagnosed with idiopathic CD according to validated criteria [1,33–34]. Exclusion criteria
were a known genetic form of dystonia, secondary forms of CD, other neurological disorders, a history of exposure to drugs known to negatively impact cognitive and behavioral
functions, and previous or current alcohol abuse. Demographic and clinical data, including gender, age, age at disease onset, disease duration, and BoNT-A dose, were collected.
The experimental procedure was approved by the local institutional review board (Sapienza University of Rome Ethics Committee, No. 5941) and was conducted in accordance
with the Declaration of Helsinki. All patients were informed of the purpose of the study
and written informed consent was obtained from all participants.
5.2. Motor and Non-Motor Assessment
All CD patients underwent three evaluations of motor symptoms: before (baseline)
and one and three months after BoNT-A treatment.
The baseline assessment was performed 16 weeks after the last BoNT-A injection and
immediately before a new BoNT-A treatment. The one-month evaluation was chosen because this time point corresponds to the peak effect of BoNT-A induced motor improvement [25,57–59]. The three-month evaluation was chosen to assess whether BoNT-A-induced changes in motor and non-motor symptoms show a similar temporal trajectory.
Indeed, it is known that BoNT-A’s effects wear off at 16 weeks but they are still present,
to a small extent, after 12 weeks [57–59].
A full neurological examination was performed by a neurologist expert in movement
disorders. CD severity was assessed according to the revised Toronto Western Spasmodic
Torticollis Rating Scale (TWSTRS) [52]. The scale includes 21 items divided into four sections. The first evaluates the amplitude of excursion of abnormal neck posture without
opposing any movement while performing tasks indicated by the examiner. The second
section evaluates the degree of disability of patients and their ability to work or perform
housework, perform daily living activities (such as feeding, dressing, washing, shaving,
etc.), drive, read, or watch television, or perform other activities outside the house (such
as shopping, going to the movies, dining, or other recreational activities). The third section
assesses severity and duration of pain and its contribution to patient disability. The fourth
section evaluates the presence of psychiatric symptoms in the month preceding the evaluation. It is composed of six items and evaluates the presence of depressed feelings, loss
of interest in things the patient used to enjoy, fear of performing activities (such as speaking, eating, or writing) in front of other people, feelings of anxiety, presence of panic attacks, and fear of going out of the house alone, being in crowds, standing in line, or traveling on buses or trains. To better evaluate motor symptom severity, all patients also underwent a standardized video recording [22,60–62].
Toxins 2021, 13, 647
9 of 12
NMS severity of CD patients was also assessed at three different time-points: before
and one and three months after BoNT-A. The Hamilton Anxiety Rating Scale (HAM-A)
was used to evaluate the presence and severity of anxiety [55], whereas the Hamilton Depression Rating Scale (HAM-D) was used to assess the presence and severity of depression [56]. To evaluate pain, we used the TWSTRS, section III, which evaluates the severity
and duration of pain and its contribution to disability [52]. The presence of sleep disturbance was evaluated using the Pittsburgh Sleep Quality Index (PSQI) and the Epworth
Sleepiness Scale (ESS) [63,64]. The PSQI is a self-rated questionnaire composed of 19 questions that assess sleep quality and disturbances in the month preceding its administration.
The ESS is a self-administered questionnaire to measure daytime sleepiness.
Finally, we investigated dystonia-related and global disability. Global disability was
evaluated by administering the Italian Perceived Disability Scale (IPDS), a 20-item selfreport instrument that assesses global disability on a five-point Likert scale (completely
false to completely true) [65]. The IPDS is based on items that investigate people’s beliefs
regarding autonomy/disability in different situations in life and has previously been used
to assess perceived disability in patients affected by movement disorders [66,67].
5.3. Statistical Analysis
Data are expressed as mean ± standard deviation (SD) unless otherwise specified.
Statistical analysis was performed by using IBM SPSS Statistics for Windows, Version
25.0. Armonk, NY, USA: IBM Corp. 25. The Shapiro–Wilk test was used to evaluate
whether distribution was Gaussian or not. Parametric and non-parametric tests were used
to compare clinical scale scores before and one and three months after BoNT-A, accordingly. Spearman’s rank correlation coefficient was used to evaluate possible correlations
between BoNT-A-induced changes in motor and non-motor domains. For each patient,
we calculated the ratio between one month and baseline clinical scale scores. In addition,
we investigated possible correlations between disability and clinical scales by calculating
the ratio between one month and baseline TWSTRS dystonia-related disability scores. A
value of p < 0.05 indicated statistical significance. FDR correction was used to correct for
multiple comparisons where appropriate.
Author Contributions: Conceptualization, M.C., D.B., I.B., M.P., A.C., G.F. (Giovanni Fabbrini),
G.D., and A.B.; methodology, M.C., D.B., I.B., and G.L.; formal analysis, M.C., D.B., A.C., and G.L.;
investigation, M.C., D.B., A.M., V.B., G.F. (Gina Ferrrazzano), C.C., and G.L.; data curation, M.C.,
D.B., I.B., V.B., G.F. (Gina Ferrazzano), and C.C.; writing—original draft preparation, M.C., D.B.,
I.B., G.D., and A.B.; writing—review and editing, M.C., D.B., I.B., M.P., A.C., G.F. (Giovanni Fabbrini), G.D., and A.B.; supervision, M.P., A.C., G.F. (Giovanni Fabbrini), G.D., and A.B.; funding
acquisition, A.B. All Authors have read and agreed to the published version of the manuscript.
Funding: This study was supported by an unrestricted grant from Ipsen S.p.A.
Institutional Review Board Statement: The study was conducted according to the guidelines of the
Declaration of Helsinki and was approved by the local institutional review board (Sapienza University of Rome Ethics Committee, No. 5941, approval date 23 July 2020).
Informed Consent Statement: Informed consent was obtained from all subjects involved in the
study.
Data Availability Statement: Data are available upon reasonable request.
Acknowledgments: We thank Melissa Kerr for English language editing.
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the
design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Toxins 2021, 13, 647
10 of 12
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
Jinnah, H.A.; Berardelli, A.; Comella, C.; Defazio, G.; Delong, M.R.; Factor, S.; Galpern, W.R.; Hallett, M.; Ludlow, C.L.;
Perlmutter, J.S.; et al. The focal dystonias: Current views and challenges for future research. Mov. Disord. 2013, 28, 926–943,
https://doi.org/10.1002/mds.25567.
Chan, J.; Brin, M.F.; Fahn, S. Idiopathic cervical dystonia: Clinical characteristics. Mov. Disord. 1991, 6, 119–126,
https://doi.org/10.1002/mds.870060206.
Dauer, W.T.; Burke, R.E.; Greene, P.; Fahn, S. Current concepts on the clinical features, aetiology and management of idiopathic
cervical dystonia. Brain 1998, 121, 547–560, https://doi.org/10.1093/brain/121.4.547.
Reichel, G. Cervical dystonia: A new phenomenological classification for botulinum toxin therapy. Basal Ganglia 2011, 1, 5–12,
https://doi.org/10.1016/j. baga.2011.01.001.
Jankovic, J.; Leder, S.; Warner, D.; Schwartz, K. Cervical dystonia: Clinical findings and associated movement disorders.
Neurology 1991, 41, 1088–1088, https://doi.org/10.1212/WNL.41.7.1088.
Erro, R.; Rubio-Agusti, I.; Saifee, T.A.; Cordivari, C.; Ganos, C.; Batla, A.; Bhatia, K.P. Rest and other types of tremor in adultonset primary dystonia. J. Neurol. Neurosurg. Psychiatry 2014, 85, 965–968, https://doi.org/10.1136/jnnp-2013-305876.
Defazio, G.; Gigante, A.F.; Abbruzzese, G.; Bentivoglio, A.R.; Colosimo, C.; Esposito, M.; Fabbrini, G.; Guidubaldi, A.; Girlanda,
P.; Liguori, R.; et al. Tremor in primary adult-onset dystonia: Prevalence and associated clinical features. J. Neurol. Neurosurg.
Psychiatry 2013, 84, 404–408, https://doi.org/10.1136/jnnp-2012-303782.
Defazio, G.; Conte, A.; Gigante, A.F.; Fabbrini, G.; Berardelli, A. Is tremor in dystonia a phenotypic feature of dystonia? Neurology 2015, 84, 1053–1059, https://doi.org/10.1212/WNL.0000000000001341.
Berardelli, I.; Ferrazzano, G.; Pasquini, M.; Biondi, M.; Berardelli, A.; Fabbrini, G. Clinical course of psychiatric disorders in
patients with cervical dystonia. Psychiatry Res. 2015, 229, 583–585, https://doi.org/10.1016/j.psychres.2015.07.076.
Gündel, H.; Wolf, A.; Xidara, V.; Busch, R.; Ladwig, K.H.; Jacobi, F.; von Rad, M.; Ceballos-Baumann, A.O. High psychiatric
comorbidity in spasmodic torticollis: A controlled study. J. Nerv. Ment. Dis. 2003, 191, 465–473,
https://doi.org/10.1097/01.NMD.0000081667.02656.21.
Moraru, E.; Schnider, P.; Wimmer, A.; Wenzel, T.; Birner, P.; Griengl, H.; Auff, E. Relation between depression and anxiety in
dystonic patients: Implications for clinical management. Depress. Anxiety 2002, 16, 100–103, https://doi.org/10.1002/da.10039.
Lencer, R.; Steinlechner, S.; Stahlberg, J.; Rehling, H.; Orth, M.; Baeumer, T.; Rumpf, H.J.; Meyer, C.; Klein, C.; Muenchau, A.; et
al. Primary focal dystonia: Evidence for distinct neuropsychiatric and personality profiles. J. Neurol. Neurosurg. Psychiatry 2009,
80, 1176–1179, https://doi.org/10.1136/jnnp.2008.170191.
Fabbrini, G.; Berardelli, I.; Moretti, G.; Pasquini, M.; Bloise, M.; Colosimo, C.; Biondi, M.; Berardelli, A. Psychiatric disorders in
adult-onset focal dystonia: A case-control study. Mov. Disord. 2010, 25, 459–465, https://doi.org/10.1002/mds.22983.
Trotti, L.M.; Esper, C.D.; Feustel, P.J.; Bliwise, D.L.; Factor, S.A. Excessive daytime sleepiness in cervical dystonia. Parkinsonism
Relat. Disord. 2009, 15, 784–786, https://doi.org/10.1016/j.parkreldis.2009.04.007.
Paus, S.; Gross, J.; Moll-Müller, M.; Hentschel, F.; Spottke, A.; Wabbels, B.; Klockgether, T.; Abele, M. Impaired sleep quality
and restless legs syndrome in idiopathic focal dystonia: A controlled study. J. Neurol. 2011. 258, 1835–1840,
https://doi.org/10.1007/s00415-011-6029-6.
Avanzino, L.; Martino, D.; Marchese, R.; Aniello, M.S.; Minafra, B.; Superbo, M.; Defazio, G.; Abbruzzese, G. Quality of sleep in
primary focal dystonia: A case-control study. Eur. J. Neurol. 2010, 17, 576–581, https://doi.org/10.1111/j.1468-1331.2009.02884.x.
Tinazzi, M.; Fiorio, M.; Fiaschi, A.; Rothwell, J.C.; Bhatia, K.P. Sensory functions in dystonia: Insights from behavioral studies.
Mov. Disord. 2009, 24, 1427–1436, https://doi.org/10.1002/mds.22490.
Tinazzi, M.; Squintani, G.M.; Bhatia, K.P.; Segatti, A.; Donato, F.; Valeriani, M.; Erro, R. Pain in cervical dystonia: Evidence of
abnormal inhibitory control. Parkinsonism Relat. Disord. 2019, 65, 252–255, https://doi.org/10.1016/j.parkreldis.2019.06.009.
Stamelou, M.; Edwards, M.J.; Hallett, M.; Bhatia, K.P. The non-motor syndrome of primary dystonia: Clinical and
pathophysiological implications. Brain 2012, 135, 1668–1681, https://doi.org/10.1093/brain/awr224.
Han, V.; Skorvanek, M.; Smit, M.; Turcanova Koprusakova, M.; Hoekstra, T.; van Dijk, J.P.; Tijssen, M.A.J.; Gdovinova, Z.;
Reijneveld, S.A. Prevalence of non-motor symptoms and their association with quality of life in cervical dystonia. Acta Neurol.
Scand. 2020, 142, 613–622, https://doi.org/10.1111/ane.13304.
Ortiz, R.M.; Scheperjans, F.; Mertsalmi, T.; Pekkonen, E. Comorbidity and retirement in cervical dystonia. J. Neurol. 2019, 266,
2216–2223, https://doi.org/10.1007/s00415-019-09402-0.
Costanzo, M.; Belvisi, D.; Berardelli, I.; Maraone, A.; D’Antonio, F.; Baione, V.; Toscano, A.; Ferrazzano, G.; Pasquini, M.; Conte,
A.; et al. Motor and non-motor subtypes of cervical dystonia. Parkinsonism Relat. Disord. 2021, 88, 108–113,
https://doi.org/10.1016/j.parkreldis.2021.06.008.
Zurowski, M.; McDonald, W.M.; Fox, S.; Marsh, L. Psychiatric comorbidities in dystonia: Emerging concepts. Mov. Disord. 2013,
28, 914–920, https://doi.org/10.1002/mds.25501.
Simpson, D.M.; Hallett, M.; Ashman, E.J.; Comella, C.L.; Green, M.W.; Gronseth, G.S.; Armstrong, M.J.; Gloss, D.; Potrebic, S.;
Jankovic, J.; et al. Practice guideline update summary: Botulinum neurotoxin for the treatment of blepharospasm, cervical dystonia, adult spasticity, and headache: Report of the Guideline Development Subcommittee of the American Academy of Neurology. Neurology 2016, 86, 1818–1826, https://doi.org/10.1212/WNL.0000000000002560.
Toxins 2021, 13, 647
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
11 of 12
Rodrigues, F.B.; Duarte, G.S.; Marques, R.E.; Castelão, M.; Ferreira, J.; Sampaio, C.; Moore, A.P.; Costa, J. Botulinum toxin type
A
therapy
for
cervical
dystonia.
Cochrane
Database
Syst.
Rev.
2020,
11,
CD003633,
https://doi.org/10.1002/14651858.CD003633.pub4.
Schiavo, G.; Matteoli, M.; Montecucco, C. Neurotoxins Affecting Neuroexocytosis. Physiol. Rev. 2000, 80, 717–766,
https://doi.org/10.1152/physrev.2000.80.2.717.
Caleo, M.; Restani, L. Direct central nervous system effects of botulinum neurotoxin. Toxicon 2018, 147, 68–72,
https://doi.org/10.1016/j.toxicon.2017.10.027.
Hok, P.; Veverka, T.; Hluštík, P.; Nevrlý, M.; Kaňovský, P. The Central Effects of Botulinum Toxin in Dystonia and Spasticity.
Toxins 2021, 13, 155, https://doi.org/10.3390/toxins13020155.
Curra, A.; Berardelli, A. Do the unintended actions of botulinum toxin at distant sites have clinical implications? Neurology 2009,
72, 1095–1099, https://doi.org/10.1212/01.wnl.0000345010.98495.fc.
Bozzi, Y.; Costantin, L.; Antonucci, F.; Caleo, M. Action of botulinum neurotoxins in the central nervous system: Antiepileptic
effects. Neurotox Res. 2006, 9, 197–203, https://doi.org/10.1007/BF03033939.
Eichenseer, S.R.; Stebbins, G.T.; Comella, C.L. Beyond a motor disorder: A prospective evaluation of sleep quality in cervical
dystonia. Parkinsonism Relat. Disord. 2014, 20, 405–408, https://doi.org/10.1016/j.parkreldis.2014.01.004.
Comella, C.L.; Jankovic, J.; Truong, D.D.; Hanschmann, A.; Grafe, S.; U.S. XEOMIN Cervical Dystonia Study Group. Efficacy
and safety of incobotulinumtoxinA (NT 201, XEOMIN®, botulinum neurotoxin type A, without accessory proteins) in patients
with cervical dystonia. J. Neurol. Sci. 2011; 308, 103–109, https://doi.org/10.1016/j.jns.2011.05.041.
Defazio, G.; Albanese, A.; Pellicciari, R.; Scaglione, C.L.; Esposito, M.; Morgante, F.; Abbruzzese, G.; Bentivoglio, A.R.; Bono, F.;
Coletti Moja, M.; et al. Expert recommendations for diagnosing cervical, oromandibular, and limb dystonia. Neurol. Sci. 2019,
40, 89–95, https://doi.org/10.1007/s10072-018-3586-9.
Albanese, A.; Bhatia, K.; Bressman, S.B.; Delong, M.R.; Fahn, S.; Fung, V.S.; Hallett, M.; Jankovic, J.; Jinnah, H.A.; Klein, C.; et
al. Phenomenology and classification of dystonia: A consensus update. Mov. Disord. 2013, 28, 863–873.
https://doi.org/10.1002/mds.25475.
Dong, H.; Fan, S.; Luo, Y.; Peng, B. Botulinum toxin relieves anxiety and depression in patients with hemifacial spasm and
blepharospasm. Neuropsychiatr Dis Treat. 2018, 15, 33–36, https://doi.org/10.2147/NDT.S181820.
Jahanshahi, M.; Marsden, C.D. Depression in torticollis: A controlled study. Psychol. Med. 1988. 18, 925–933,
https://doi.org/10.1017/S0033291700009855.
Jahanshahi, M., Marsden, C.D. A Longitudinal Follow-up Study of Depression, Disability, and Body Concept in Torticollis.
Behav. Neurol. 1990, 3, 233–246, https://doi.org/10.1155/1990/873983.
Jahanshahi, M.; Marsden, C.D. Body Concept, Disability, and Depression in Patients with Spasmodic Torticollis. Behav. Neurol.
1990, 3, 117–131, https://doi.org/10.1155/1990/764203.
Egger, K.; Mueller, J.; Schocke, M.; Brenneis, C.; Rinnerthaler, M.; Seppi, K.; Trieb, T.; Wenning, G.K.; Hallett, M.; Poewe, W.
Voxel based morphometry reveals specific gray matter changes in primary dystonia. Mov. Disord. 2007, 22, 1538–1542,
https://doi.org/10.1002/mds.21619.
Ceballos-Baumann, A.O.; Passingham, R.E.; Warner, T.; Playford, E.D.; Marsden, C.D.; Brooks, D.J. Overactive prefrontal and
underactive motor cortical areas in idiopathic dystonia. Ann. Neurol. 1995, 37, 363–372, https://doi.org/10.1002/ana.410370313.
Delnooz, C.C.; Pasman, J.W.; Beckmann, C.F.; van de Warrenburg, B.P. Task-free functional MRI in cervical dystonia reveals
multi-network changes that partially normalize with botulinum toxin. PLoS ONE 2013, 8, e62877, https://doi.org/10.1371/journal.pone.0062877.
Opavský, R.; Hluštík, P.; Otruba, P.; Kaňovský, P. Sensorimotor network in cervical dystonia and the effect of botulinum toxin
treatment: A functional MRI study. J. Neurol. Sci. 2011, 306, 71–75, https://doi.org/10.1016/j.jns.2011.03.040.
Nevrlý, M.; Hluštík, P.; Hok, P.; Otruba, P.; Tüdös, Z.; Kaňovský, P. Changes in sensorimotor network activation after botulinum toxin type A injections in patients with cervical dystonia: A functional MRI study. Exp. Brain. Res. 2018, 236, 2627–2637,
https://doi.org/10.1007/s00221-018-5322-3.
Jinnah, H.A.; Neychev, V.; Hess, E.J. The Anatomical Basis for Dystonia: The Motor Network Model. Tremor Other Hyperkinet.
Mov. 2017, 7, 506, https://doi.org/10.5334/tohm.383.
Corp, D.T.; Joutsa, J.; Darby, R.R.; Delnooz, C.C.S.; van de Warrenburg, B.P.C.; Cooke, D.; Prudente, C.N.; Ren, J.; Reich, M.M.;
Batla, A.; et al. Network localization of cervical dystonia based on causal brain lesions. Brain 2019, 142, 1660–1674,
https://doi.org/10.1093/brain/awz112.
Shaikh, A.G.; Zee, D.S.; Crawford, J.D.; Jinnah, H.A. Cervical dystonia: A neural integrator disorder. Brain 2016, 139, 2590–2599,
doi:10.1093/brain/aww141.
Camargo, C.; Cattai, L.; Teive, H.A. Pain Relief in Cervical Dystonia with Botulinum Toxin Treatment. Toxins 2015, 7, 2321–
2335, https://doi.org/10.3390/toxins7062321.
Marciniec, M.; Szczepańska-Szerej, A.; Kulczyński, M.; Sapko, K.; Popek-Marciniec, S.; Rejdak, K. Pain in cervical dystonia and
the antinociceptive effects of botulinum toxin: What is currently known? Rev. Neurosci. 2019, 30, 771–779,
https://doi.org/10.1515/revneuro-2018-0119.
Velly, A.M.; Mohit, S. Epidemiology of pain and relation to psychiatric disorders. Prog. Neuropsychopharmacol. Biol. Psychiatry
2018, 87, 159–167, https://doi.org/10.1016/j.pnpbp.2017.05.012.
Toxins 2021, 13, 647
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
12 of 12
Brans, J.W.; Lindeboom, R.; Snoek, J.W.; Zwarts, M.J.; van Weerden, T.W.; Brunt, E.R.; van Hilten, J.J.; van der Kamp, W.; Prins,
M.H.; Speelman, J.D. Botulinurn Toxin Versus Trihexyphenidyl in Cervical Dystonia: A Prospective, Randomized, DoubleBlind Controlled Trial. Neurology 1996, 46, 1066–1072, https://doi.org/10.1212/wnl.46.4.1066.
Van Den Dool, J.; Tijssen, M.A.; Koelman, J.H.; Engelbert, R.H.; Visser, B. Determinants of disability in cervical dystonia. Parkinsonism Relat. Disord. 2016, 32, 48–53, https://doi.org/10.1016/j.parkreldis.2016.08.014.
Comella, C.L.; Fox, S.H.; Bhatia, K.P.; Perlmutter, J.S.; Jinnah, H.A.; Zurowski, M.; McDonald, W.M.; Marsh, L.; Rosen, A.R.;
Waliczek, T.; et al. Development of the Comprehensive Cervical Dystonia Rating Scale: Methodology. Mov. Disord. Clin. Pract.
2015, 2, 135–141, https://doi.org/10.1002/mdc3.12131.
Comella, C.L.; Perlmutter, J.S.; Jinnah, H.A.; Waliczek, T.A.; Rosen, A.R.; Galpern, W.R.; Adler, C.A.; Barbano, R.L.; Factor, S.A.;
Goetz, C.G.; et al. Clinimetric testing of the comprehensive cervical dystonia rating scale: Comprehensive Cervical Dystonia
Rating Scale. Mov. Disord. 2016, 31, 563–569, https://doi.org/10.1002/mds.26534.
Trajković, G.; Starčević, V.; Latas, M.; Leštarević, M.; Ille, T.; Bukumirić, Z.; Marinković, J. Reliability of the Hamilton Rating
Scale for Depression: A meta-analysis over a period of 49years. Psychiatry Res. 2011, 189, 1–9, https://doi.org/10.1016/j.psychres.2010.12.007.
Hamilton, M. The assessment of anxiety states by rating. Br. J. Med. Psychol. 1959, 32, 50–55, https://doi.org/10.1111/j.20448341.1959.tb00467.x.
Hamilton, M. A rating scale for depression. J. Neurosurg. Psychiat. 1960, 23, 56, doi:10.1136/jnnp.23.1.56.
Marsh, W.A.; Monroe, D.M.; Brin, M.F.; Gallagher, C.J. Systematic review and meta-analysis of the duration of clinical effect of
onabotulinumtoxinA in cervical dystonia. BMC Neurol. 2014, 14, 91, https://doi.org/10.1186/1471-2377-14-91.
Dressler, D.; Tacik, P.; Saberi, F.A. Botulinum toxin therapy of cervical dystonia: Duration of therapeutic effects. J. Neural.
Transm. 2015, 122, 297–300, doi:10.1007/s00702-014-1253-8.
Wissel, J.; Kanovsky, P.; Ruzicka, E.; Bares, M.; Hortova, H.; Streitova, H.; Jech, R.; Roth, J.; Brenneis, C.; Müller, J.; et al. Efficacy
and safety of a standardised 500 unit dose of Dysport (clostridium botulinum toxin type A haemaglutinin complex) in a heterogeneous cervical dystonia population: Results of a prospective, multicentre, randomised, double-blind, placebo-controlled,
parallel group study. J. Neurol. 2001, 248, 1073–1078, https://doi.org/10.1007/s004150170028.
Ferrazzano, G.; Berardelli, I.; Belvisi, D.; De Bartolo, M.I.; Di Vita, A.; Conte, A.; Fabbrini, G. Awareness of Dystonic Posture in
Patients With Cervical Dystonia. Front. Psychol. 2020, 11, 1434, https://doi.org/10.3389/fpsyg.2020.01434.
Ferrazzano, G.; Conte, A.; Fabbrini, G.; Bologna, M.; Macerollo, A.; Defazio, G.; Hallett, M.; Berardelli, A. Botulinum toxin and
blink rate in patients with blepharospasm and increased blinking. J. Neurol. Neurosurg. Psychiatry 2015, 86, 336–340,
https://doi.org/10.1136/jnnp-2014-307821.
Ferrazzano, G.; Berardelli, I.; Conte, A.; Baione, V.; Concolato, C.; Belvisi, D.; Fabbrini, G.; Defazio, G.; Berardelli, A. Motor and
non-motor symptoms in blepharospasm: Clinical and pathophysiological implications. J. Neurol. 2019, 266, 2780–2785,
https://doi.org/10.1007/s00415-019-09484-w.
Buysse, D.J.; Reynolds, C.F.; Monk, T.H.; Berman, S.R.; Kupfer, D.J. The Pittsburgh sleep quality index: A new instrument for
psychiatric practice and research. Psychiatry Res. 1989, 28, 193–213, https://doi.org/10.1016/0165-1781(89)90047-4.
Johns, M.W. A New Method for Measuring Daytime Sleepiness: The Epworth Sleepiness Scale. Sleep 1991, 14, 540–545,
https://doi.org/10.1093/sleep/14.6.540.
Innamorati, M.; Pompili, M.; De Filippis, S.; Gentili, F.; Erbuto, D.; Lester, D.; Tamburello, A.; Iacorossi, G.; Cuomo, I.; Dominici,
G.; et al. The validation of the Italian Perceived Disability Scale (IPDS) in chronic daily headache sufferers. J. Headache Pain 2009,
10, 21–26, https://doi.org/10.1007/s10194-008-0076-0.
Berardelli, I.; Belvisi, D.; Corigliano, V.; Costanzo, M.; Innamorati, M.; Fabbrini, G.; Berardelli, A.; Pompili, M. Suicidal ideation,
perceived disability, hopelessness and affective temperaments in patients affected by Parkinson's disease. Int. J. Clin. Pract. 2018,
19, e13287, https://doi.org/10.1111/ijcp.13287.
Belvisi, D.; Berardelli, I.; Ferrazzano, G.; Costanzo, M.; Corigliano, V.; Fabbrini, G.; Berardelli, A.; Pompili, M. The clinical correlates
of
suicidal
ideation
in
Parkinson's
disease.
Parkinsonism
Relat
Disord.
2019,
63,
54–59,
https://doi.org/10.1016/j.parkreldis.2019.02.047.